|
--- |
|
license: apache-2.0 |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
# Bigram Language Model |
|
|
|
## Overview |
|
This repository contains a simple Bigram Language Model implemented in PyTorch. The model is trained to predict the next character in a sequence, given the current character. It's a character-level model and can be used for tasks like text generation. |
|
|
|
## Model Details |
|
- **Model Type**: Character-level Language Model |
|
- **Architecture**: Simple lookup table for character bigrams |
|
- **Training Data**: [https://huggingface.co/datasets/csebuetnlp/xlsum/viewer/bengali] |
|
|
|
## Requirements |
|
- Python 3.x |
|
- PyTorch |
|
- JSON (for loading the tokenizer) |
|
|
|
## Installation |
|
First, clone this repository: |
|
|
|
|
|
## Loading the Model |
|
To load the model, you need to initialize it with the vocabulary size and load the pre-trained weights: |
|
|
|
```python |
|
import torch |
|
from model import BigramLanguageModel |
|
|
|
vocab_size = 225 |
|
model = BigramLanguageModel(vocab_size) |
|
|
|
model.load_state_dict(torch.load('path_to_your_model.pth', map_location=torch.device('cpu'))) |
|
model.eval() |
|
|
|
import json |
|
|
|
with open('tokenizer_mappings.json', 'r', encoding='utf-8') as f: |
|
mappings = json.load(f) |
|
stoi = mappings['stoi'] |
|
itos = mappings['itos'] |
|
|
|
# Example usage |
|
encode = lambda s: [stoi[c] for c in s] |
|
decode = lambda l: ''.join([itos[i] for i in l]) |
|
|
|
|
|
context = torch.tensor([encode("Your initial text")], dtype=torch.long) |
|
generated_text_indices = model.generate(context, max_new_tokens=100) |
|
print(decode(generated_text_indices[0].tolist())) |
|
|
|
|
|
|
