README.md

---
language: fa
license: apache-2.0
---
This repository is created with the aim to provide better models for NLI in persian, with the transparent codes for training I hope you guys find it inspiring and build better model in the future. for more details about the task and methods used for training check the [medium post](https://haddadhesam.medium.com/) and notebooks.


# Dataset

The dataset used for training is Wiki D/Similar dataset (wiki-d-similar.zip), obtained from [Sentence Transformers](https://github.com/m3hrdadfi/sentence-transformers) repository.

# Model

The proposed model is published at HuggingFace Hub with the name of ``demoversion/bert-fa-base-uncased-haddad-wikinli``. You can download and use the model from [HuggingFace Website](https://huggingface.co/demoversion/bert-fa-base-uncased-haddad-wikinli) or directly in transformers library like this:

    from transformers import pipeline
    model = pipeline("zero-shot-classification", model="demoversion/bert-fa-base-uncased-haddad-wikinli")
    labels = ["ورزشی",
			    "سیاسی",
			    "علمی",
			    "فرهنگی"]
		template_str = "این یک متن {} است."
		str_sentence = "مرحله مقدماتی جام جهانی حاشیه‌های زیادی داشت."
		model(str_sentence, labels, hypothesis_template=template_str)
The result of this code snippet is:

    Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. Default to no truncation.
    {'labels': ['فرهنگی', 'علمی', 'سیاسی', 'ورزشی'],
     'scores': [0.25921085476875305,
       0.25713297724723816,
       0.24884170293807983,
       0.23481446504592896],
       'sequence': 'مرحله مقدماتی جام جهانی حاشیه\u200cهای زیادی داشت.'}
Yep, the right label (highest score) without training.
# Results

The result comparing to the original model published for this dataset is available in the table bellow.


|Model|dev_accuracy| dev_f1|test_accuracy|test_f1|
|--|--|--|--|--|
|[m3hrdadfi/bert-fa-base-uncased-wikinli](https://huggingface.co/m3hrdadfi/bert-fa-base-uncased-wikinli)|77.88|77.57|76.64|75.99|
|[demoversion/bert-fa-base-uncased-haddad-wikinli](https://huggingface.co/demoversion/bert-fa-base-uncased-haddad-wikinli)|**78.62**|**79.74**|**77.04**|**78.56**|

# Notebooks
Notebooks used for training and evaluation are available below.

[Training ![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/DemoVersion/persian-nli-trainer/blob/main/notebooks/training.ipynb)

[Evaluation ![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/DemoVersion/persian-nli-trainer/blob/main/notebooks/evaluation.ipynb)