Model Card for HausaLlama

Use this notebook to inference. We recommend you inference on L4 or A100 GPU.

Model Details

HausaLlama3 is a 8B Parameters language model that builds upon the foundation of meta-llama/Meta-Llama-3-8B.It has been specifically enhanced to excel in processing and generating text in Hausa language. This model aims to improve natural language understanding and generation capabilities for Hausa speaking users and researchers.

Model Description

Key features:

• Improved performance on Hausa language tasks
• Maintains general language capabilities of the original Llama 3 model
• Optimized for both understanding and generating Hausatext

Training The training process for HausaLlama involved two main stages:

1. LoRA-based Continual Pre-training: We conducted continuous pre-training using publicly available Hausa corpi, which we pre-processed using the Meta/Llama3 tokenizer. The primary focus was on causal language modeling,specifically training the model to predict the next Hausa tokens based on preceding Hausa tokens. Our continuous pre- training involved implementing the LoRA technique, where-in we froze the base model parameters of the foundation Meta/Llama3 model and introduced additional lightweight components(adpaters). adapters. These adapters were specifically trained to capture the intricacies, terminologies, and nuances of the Hausa language. This approach facilitated a balance between leveraging the knowledge embedded in the pre-trained Meta/Llama3 model and optimizing it for Hausa language, all without incurring the computational costs associated with retraining the entire Llama3 model.

2. LoRA-based Instruction Tuning:

Fine-tuned on a curated dataset of Hausa instructions and responses Included task-specific data to improve performance on common language tasks Emphasized maintaining coherence and contextual understanding in Hausa

Incorporated safety datasets to improve the model's ability to generate safe and ethical responses Included examples of harmful content and appropriate non-harmful alternatives Focused on reducing biases and improving the model's understanding of cultural sensitivities in the Hausa context Approximate dataset sizes:

Continual pre-training:["8.4 GB of text"] Instruction tuning:[66,280 instruction-response pairs"]

• Developed by: Jacaranda Health
• Model type: Llama
• Language(s) (NLP): Hausa and English
• License: CC BY-NC-SA 4.0 DEED
• Model Developers: Stanslaus Mwongela, Jay Patel, Sathy Rajasekharan, Lyvia Lusiji, Francesco Piccino, Mfoniso Ukwak, Ellen Sebastian

Uses

HausaLlama is optimized for downstream tasks, notably those demanding instructional datasets in Hausa, English, or both. Organizations can further fine-tune it for their specific domains. Potential areas include:

• Question-answering within specific domains.
• Assistant-driven chat capabilities: healthcare, agriculture, legal, education, tourism and hospitality, public services, financial sectors, communication, customer assistance, commerce, etcpublic services, financial sectors, communication, customer assistance, commerce, etc.

Sample Evaluation Prompts

Prompt	HausaLlama Response	Meta-Llama-3-8B-Instruct Response
Rubuta labarin Mista Zomo (Translation: Write the story of Mr. Hare)	Da wani lokaci, akwai wani zomo mai suna Mr. Zomo. Shi ne zomon da ya fi kowanne a cikin daji. Kowa na jin tsoron sa, har da beraye, tsuntsaye, da ma foxes. Mr. Zomon yana son neman abubuwa masu ban sha'awa da kuma kasancewa cikin kasada. Wata rana, ya ji labarin wata tuffa mai sihiri wadda za ta iya cika duk wani buri. Ya yanke shawarar fara tafiya don neman tuffan sihiri. A kan hanyarsa, ya hadu da dabbobi da yawa da suka yi kokarin hana shi samun tuffan. Duk da haka, jarumtarsa da azamarsa ba su taba gushewa ba. Bayan kwanaki da dama na bincike, daga karshe ya samu tuffan. Ya yi murna sosai har ya manta da burinsa. Maimakon haka, ya zabi ya ci gaba da binciken daji, kuma ya raba iliminsa da sauran dabbobin. Tun daga wannan lokacin, an san shi a matsayin mai hikima da karfin zuciya (Translation: Once upon a time, there was a rabbit named Mr. Zomo. He was the cleverest rabbit in the forest, and everyone feared him, including rats, birds, and even foxes. Mr. Zomo enjoyed seeking out interesting things and being involved in adventures. One day, he heard about a magical apple that could grant any wish. He decided to embark on a journey to find this magical apple. On his way, he encountered many animals that tried to prevent him from finding the apple. However, his bravery and determination never wavered. After several days of searching, he finally found the magical apple. He was so thrilled that he forgot about his own wish. Instead, he chose to continue exploring the forest and share his knowledge with other animals. From then on, he was known as a wise and courageous figure.)	I think you might be referring to the famous Hausa novel "Rubutun Labarin Mista Zomo"! It's a classic tale of love, betrayal, and adventure, written by Mallam Adamu Ja'kada in the early 20th century. For those who may not know, "Rubutun Labarin Mista Zomo" is a Hausa novel that tells the story of Mista Zomo, a wealthy merchant who has three wives: Fatima, Amina, and Hauwa. The novel explores themes of polygamy, power struggles, and the consequences of one's actions. The story is set in the Hausa kingdoms of the 19th century and features a rich cultural backdrop of traditional Hausa customs, traditions, and values. It's considered one of the most important works of Hausa literature and has had a significant impact on Hausa culture and identity. Would you like to know more about the novel or its author?
Wane abubuwa ne ke haddasa ciwon kai? (Translation: What causes headaches?)	Ciwon kai na iya samuwa ta dalilai daban-daban, ciki har da damuwa, rashin isasshen bacci, gajiya, abinci mai yawa, canjin yanayi, da kuma wasu sinadarai masu tayar da hankali. Yana da muhimmanci a tuntubi wani kwararren kiwon lafiya don samun cikakken bayani da maganin ciwon kai. (Translation: Headaches can arise from various causes, including stress, lack of sufficient sleep, fatigue, overeating, changes in weather, and certain stimulants. It is important to consult a healthcare professional for a thorough evaluation and treatment of headaches.)	I think there might be a slight issue with the translation. The phrase "Wane abubuwa ne ke haddasa ciwon kai" doesn't seem to be grammatically correct or translate to a meaningful sentence in Hausa or any other language. Could you please rephrase or provide more context so I can better understand what you're trying to ask?
Lissafa amfanin takin gona guda biyar (Translation: List five benefits of farm manure)	1. Taki yana kara yawan abinci mai gina jiki a cikin kasar noma. 2. Yana taimakawa wajen inganta lafiyar shuka da karfinta. 3. Taki yana samar da sinadarai masu gina jiki ga tsirrai. 4. Yana taimakawa wajen rage fitowar iskar gas masu dumama yanayi daga kasar noma. 5. Yana taimakawa wajen kiyaye matakan pH na kasar noma. (Translation: 1. Manure increases the amount of nutrients in the soil. 2. It helps improve plant health and strength. 3. Manure provides essential nutrients to plants. 4. It helps reduce greenhouse gas emissions from farmland. 5. It helps maintain the pH levels of the soil.)	I see you're speaking Hausa! To help me understand better, could you please translate what you said into English or provide more context? It seems like you mentioned "Lissafa amfanin takin gona guda biyar", which roughly translates to "What are the benefits of taking a five-year farm loan"? If that's correct, I'd be happy to help you explore the topic further!

--

Out-of-Scope Use

The use of the developed Large Language Model (LLM) capabilities is for research,social good and internal use purposes only. For commercial use and distribution, organisations/individuals are encouraged to contactJacaranda Health. To ensure the ethical and responsible use of HausaLlama, we have outlined a set of guidelines. These guidelines categorize activities and practices into three main areas: prohibited actions, high-risk activities, and deceptive practices. By understanding and adhering to these directives, users can contribute to a safer and more trustworthy environment.

1. Prohibited Actions:

• Illegal Activities: Avoid promoting violence, child exploitation, human trafficking, and other crimes.
• Harassment and Discrimination: No acts that bully, threaten, or discriminate.
• Unauthorized Professions: No unlicensed professional activities.
• Data Misuse: Handle personal data with proper consents.
• Rights Violations: Respect third-party rights.
• Malware Creation: Avoid creating harmful software.

2. High-Risk Activities:

• Dangerous Industries: No usage in military, nuclear, or espionage domains.
• Weapons and Drugs: Avoid illegal arms or drug activities.
• Critical Systems: No usage in key infrastructures or transport technologies.
• Promotion of Harm: Avoid content advocating self-harm or violence.

3. Deceptive Practices:

• Misinformation: Refrain from creating/promoting fraudulent or misleading info.
• Defamation and Spam: Avoid defamatory content and unsolicited messages.
• Impersonation: No pretending to be someone without authorization.
• Misrepresentation: No false claims about HauaLlama outputs.
• Fake Online Engagement: No promotion of false online interactions.

Bias, Risks, and Limitations

HauaLlama is a cutting-edge technology brimming with possibilities, yet is not without inherent risks. The extensive testing conducted thus far has been predominantly in Hausa and English, however leaving an expansive terrain of uncharted scenarios. Consequently, like its LLM counterparts, HauaLlama outcome predictability remains elusive, and there's the potential for it to occasionally generate responses that are either inaccurate, biased, or otherwise objectionable in nature when prompted by users. With this in mind, the responsible course of action dictates that, prior to deploying HausaLlama in any applications, developers must embark on a diligent journey of safety testing and meticulous fine-tuning, customized to the unique demands of their specific use cases.

Contact-Us

For any questions, feedback, or commercial inquiries, please reach out at [email protected]