Update README.md

README.md

@@ -1,23 +1,123 @@
 ---
 base_model: ibm-granite/granite-3.1-2b-instruct
 tags:
-- text-generation-inference
+- text-generation
 - transformers
-- unsloth
+- safetensors
+- english
 - granite
+- text-generation-inference
 - trl
 - grpo
+- conversational
+- inference-endpoints
+- 4-bit precision
+- bitsandbytes
 license: apache-2.0
 language:
 - en
 ---
 
-# Uploaded  model
+# Granite-3.1-2B-Reasoning-4bit (Quantized for Efficiency)
+
+## Model Overview
+
+This is a **4-bit quantized version** of **ruslanmv/granite-3.1-2b-Reasoning**, which is fine-tuned from **ibm-granite/granite-3.1-2b-instruct**. The quantization allows for significantly reduced memory usage while maintaining strong reasoning capabilities.  
+
+- **Developed by:** [ruslanmv](https://huggingface.co/ruslanmv)  
+- **License:** Apache 2.0  
+- **Base Model:** [ibm-granite/granite-3.1-2b-instruct](https://huggingface.co/ibm-granite/granite-3.1-2b-instruct)  
+- **Fine-tuned for:** Logical reasoning, structured problem-solving, long-context tasks  
+- **Quantized with:** **bitsandbytes (4-bit precision)**  
+- **Supported Languages:** English  
+- **Tensor Type:** **BF16**  
+- **Parameter Size:** **2.53B params**  
+
+---
+
+## Why Use the Quantized Version?
+
+This **4-bit quantized model** is ideal for users who require **fast inference speeds and reduced memory usage** while still benefiting from **Granite's advanced reasoning capabilities**.  
+
+✅ **2x Faster Training** compared to standard methods  
+✅ **Lower VRAM usage**, ideal for consumer GPUs  
+✅ **Optimized for inference**, making it more efficient for deployment  
+
+---
+
+## Installation & Usage  
+
+To run the quantized model, install the required dependencies:  
+
+```bash
+pip install torch torchvision torchaudio
+pip install accelerate
+pip install transformers
+pip install bitsandbytes
+```
+
+### Running the Model  
+
+Use the following Python snippet to load and generate text with the **4-bit quantized** model:  
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+import bitsandbytes as bnb
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+model_path = "ruslanmv/granite-3.1-2b-Reasoning-4bit"
+
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+model = AutoModelForCausalLM.from_pretrained(
+    model_path, 
+    device_map="auto",
+    load_in_4bit=True,  # Load model in 4-bit precision
+    quantization_config=bnb.QuantizationConfig(llm_int8_threshold=6.0)
+)
+model.eval()
+
+input_text = "Can you explain the difference between inductive and deductive reasoning?"
+input_tokens = tokenizer(input_text, return_tensors="pt").to(device)
+
+output = model.generate(**input_tokens, max_length=4000)
+output_text = tokenizer.batch_decode(output)
+
+print(output_text)
+```
+
+---
+
+## Intended Use  
+
+Granite-3.1-2B-Reasoning-4bit is designed for tasks requiring structured **reasoning**, including:  
+
+- **Logical and analytical problem-solving**  
+- **Text-based reasoning tasks**  
+- **Mathematical and symbolic reasoning**  
+- **Advanced instruction-following**  
+
+This model is particularly useful for users needing a **lightweight, high-performance** version of **Granite-3.1-2B-Reasoning** without sacrificing too much accuracy.
+
+---
+
+## License & Acknowledgments  
+
+This model is released under the **Apache 2.0** license. It is fine-tuned from IBM’s **Granite 3.1-2B-Instruct** model and **quantized using bitsandbytes** for optimal efficiency. Special thanks to the **IBM Granite Team** for developing the base model.  
+
+For more details, visit the [IBM Granite Documentation](https://huggingface.co/ibm-granite).  
+
+---
 
-- **Developed by:** ruslanmv
-- **License:** apache-2.0
-- **Finetuned from model :** ibm-granite/granite-3.1-2b-instruct
+### Citation  
 
-This granite model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+If you use this model in your research or applications, please cite:  
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+```
+@misc{ruslanmv2025granite,
+  title={Fine-Tuning and Quantizing Granite-3.1 for Advanced Reasoning},
+  author={Ruslan M.V.},
+  year={2025},
+  url={https://huggingface.co/ruslanmv/granite-3.1-2b-Reasoning-4bit}
+}
+```