Create README.md

README.md

@@ -0,0 +1,31 @@
+---
+license: llama2
+---
+# Taga-llama-v0.3a:
+* Test Tagalog model, particularly with using a partially synthetic dataset (observing the Tagalog capability of LLaMA-2)
+* Fine tuned on dataset of ~1k items (Tagalog/Taglish dataset, based off Tagalog sentences augmented by LLaMA-2-13b base to create a 3-turn dialogue dataset between Human and Assistant); dataset to be further refined
+* Trained on llama 7b chat
+* May still switch to Taglish or English: please see usage
+* [QLoras (hf and GGML)](https://huggingface.co/922-Narra/tagalog-lm-lora-tests/tree/main/llama-2-7b-chat-tagalog-0.3a)
+
+### USAGE
+This is meant to be mainly a chat model.
+
+Use "Human" and "Assistant" and prompt with Tagalog. Example:
+
+"Ito ay isang chat log sa pagitan ng AI Assistant na nagta-Tagalog at isang Pilipino. Magsimula ng chat:\nHuman: Hello po?\nAssistant:"
+
+### HYPERPARAMS
+* Trained for 2 epochs
+* rank: 16
+* lora alpha: 32
+* lora dropout: 0.5
+* lr: 2e-4
+* batch size: 2
+* warmup ratio: 0.075
+* grad steps: 4
+
+### WARNINGS AND DISCLAIMERS
+Note that aside from formatting and other minor edits, dataset used is mostly as is generated by LM. As such, while this version may be better at coherency or chatting than previous Tagalog ones, conversations may still switch between languages or easily derail.
+
+Finally, this model is not guaranteed to output aligned or safe outputs nor is it meant for production use - use at your own risk!