metadata

language:
  - ar
tags:
  - pytorch
  - text-generation
  - causal-lm
  - rwkv
license: apache-2.0

RWKV-4-World-7b-Arabic

Model Description

RWKV-4-World-7b-Arabic is a pretrinaed version of RWKV-4-world that finetuned on Arabic datasets mc4, wikipedia, and abulkhair.

How to use:

use https://github.com/josStorer/RWKV-Runner for GUI
use latest rwkv pip package (0.8.0+)
use https://github.com/BlinkDL/ChatRWKV/blob/main/v2/benchmark_world.py and https://github.com/BlinkDL/ChatRWKV/blob/main/API_DEMO_WORLD.py to test it

NOTE: the new greedy tokenizer (https://github.com/BlinkDL/ChatRWKV/blob/main/tokenizer/rwkv_tokenizer.py) will tokenize '\n\n' as one single token instead of ['\n','\n']

QA prompt (replace \n\n in xxx to \n):

Question: xxx

Answer:

and

Instruction: xxx

Input: xxx

Response:

A good chat prompt (replace \n\n in xxx to \n):

User: hi

Assistant: Hi. I am your assistant and I will provide expert full response in full details. Please feel free to ask any question and I will always answer it.

User: xxx

Assistant:

Reference

@article{BlinkDL@rwkv-4-world,
  title={RWKV-4 World },
  URL={https://huggingface.co/BlinkDL/rwkv-4-world},
  year={2023}
}