README.md

---
license: apache-2.0
language:
- fa
- en
library_name: transformers
pipeline_tag: text-generation
datasets:
- myrkur/persian-alpaca-deep-clean
---

# Paya (aya 23 8B Instruction Tuned on Farsi)

<a href="https://ibb.co/fHmCngh"><img src="https://i.ibb.co/jD7LWNc/paya.png" alt="paya" border="0"></a>


Welcome to PAYA, a powerful Persian text generation model built upon the foundations of Aya 23 8B, a multilingual language model. PAYA has been fine-tuned using the supervised finetuning technique, employing the DORA method for efficient refinement on Persian datasets, particularly leveraging the [persian-alpaca-deep-clean](https://huggingface.co/datasets/myrkur/persian-alpaca-deep-clean) dataset.

## Features

- **Advanced Text Generation**: Generate coherent and contextually relevant Persian text with ease.
- **Efficient Fine-Tuning**: Utilizes the DORA method for streamlined fine-tuning on Persian datasets.
- **Optimized Tokenization**: The model's tokenizer ensures accurate representation of Persian words, enhancing the quality of generated text.

## Usage

You can quickly get started with PAYA using the following sample code:

```python
import transformers
import torch

model_id = "myrkur/paya"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

messages = [
    {"role": "user", "content": "علم بهتر است یا ثروت؟"},
]

prompt = pipeline.tokenizer.apply_chat_template(
        messages, 
        tokenize=False, 
        add_generation_prompt=True
)

terminators = [
    pipeline.tokenizer.eos_token_id,
]

outputs = pipeline(
    prompt,
    max_new_tokens=512,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.4,
    top_p=0.9,
    repetition_penalty=1.1
)
print(outputs[0]["generated_text"][len(prompt):])
```

## Why PAYA?

PAYA stands out for its exceptional tokenization capabilities, accurately capturing the nuances of the Persian language. Additionally, its fine-tuned parameters and efficient training methodology ensure remarkable results in text generation tasks.

## Contributions

Contributions to PAYA are welcome! Whether it's enhancing the model's capabilities, improving its performance on specific tasks, or evaluating its performance, your contributions can help advance Persian natural language processing.

## Contact
For questions or further information, please contact:

- Amir Masoud Ahmadi: [[email protected]](mailto:[email protected])