|
--- |
|
license: apache-2.0 |
|
--- |
|
<h1> |
|
<a alt="Ask2Democracy project" href="https://github.com/jorge-henao/ask2democracy">Ask2Democracy project</a> |
|
</h1> |
|
<hr> |
|
|
|
## What's baizemocracy-lora-7B-cfqa model? |
|
|
|
This model is an open-source chat model fine-tuned with [LoRA](https://github.com/microsoft/LoRA) inspired by [Baize project](https://github.com/project-baize/baize-chatbot/tree/main/). It was trained with the Baize datasets and the ask2democracy-cfqa-salud-pension dataset, wich contains almost 4k instructions to answers questions based on a context relevant to citizen concerns and public debate in spanish. |
|
|
|
- **Developed by:** |
|
- 🇨🇴 [Jorge Henao](https://huggingface.co/jorge-henao) |
|
- 🇨🇴 [David Torres ](https://github.com/datorresb) |
|
|
|
## Training Parameters |
|
|
|
- Base Model: [LLaMA-7B](https://arxiv.org/pdf/2302.13971.pdf) |
|
- Training Epoch: 1 |
|
- Batch Size: 16 |
|
- Maximum Input Length: 512 |
|
- Learning Rate: 2e-4 |
|
- LoRA Rank: 8 |
|
- Updated Modules: All Linears |
|
|
|
## Training Dataset |
|
|
|
- [Ask2Democracy-cfqa-salud-pension](https://huggingface.co/datasets/hackathon-somos-nlp-2023/ask2democracy-cfqa-salud-pension) (3,806) |
|
- [Standford Alpaca](https://github.com/tatsu-lab/stanford_alpaca) (51,942) |
|
- [Quora Dialogs](https://github.com/project-baize/baize) (54,456): |
|
- [StackOverflow Dialogs](https://github.com/project-baize/baize) (57,046) |
|
- [Alpacaca chat Dialogs](https://github.com/project-baize/baize) |
|
- [Medical chat Dialogs](https://github.com/project-baize/baize) |
|
|
|
More details can be found in the Ask2Democracy [GitHub](https://github.com/jorge-henao/ask2democracy) |
