|
--- |
|
language: |
|
- hi |
|
- gu |
|
- pa |
|
- as |
|
- ta |
|
- mr |
|
- bn |
|
- te |
|
- ml |
|
- kn |
|
--- |
|
Indic-Sentence-Completion |
|
--- |
|
license: other |
|
--- |
|
|
|
# Details |
|
The model cannot be commercially used. It's a fine-tuned Bloom-3B in several Indian languages: |
|
- Gujarati |
|
- Marathi |
|
- Bangali |
|
- Punjabi |
|
- Kannada |
|
- Malayalam |
|
- Telugu |
|
- Tamil |
|
- Hindi |
|
|
|
# Architecture |
|
Same as Bloom-3B, the model is decoder only. |
|
|
|
# Motivation behind the model fine-tuning |
|
- The model can be fine-tuned for any downstream task that requires the use of the aforementioned Indian languages |
|
- PEFT LoRA is advised. |
|
- Can be stacked with an Encoder if needed for any Sequence to Sequence task that requires aforementioned Indian languages |
|
|
|
# Example of getting inference from the model |
|
from transformers import AutoModel, AutoConfig, AutoModelForCausalLM, AutoTokenizer |
|
|
|
# Path to the directory containing the model files |
|
model_directory = "autopilot-ai/Indic-sentence-completion" |
|
tokenizer = AutoTokenizer.from_pretrained(model_directory) |
|
model = AutoModelForCausalLM.from_pretrained( |
|
model_directory, |
|
load_in_8bit=True, |
|
device_map="auto", |
|
) |
|
|
|
# Load the model configuration |
|
config = AutoConfig.from_pretrained(model_directory) |
|
|
|
# Load the model |
|
model = AutoModel.from_pretrained(model_directory, config=config) |
|
batch = tokenizer("હેલો કેમ છો?", return_tensors='pt') |
|
|
|
with torch.cuda.amp.autocast(): |
|
output_tokens = model.generate(**batch, max_new_tokens=10) |
|
|
|
print('\n\n', tokenizer.decode(output_tokens[0], skip_special_tokens=True)) |
|
|
|
## To run the above code snippet (in 8 bits), make sure to install the following |
|
pip install accelerate bitsandbytes |
|
|
