File size: 8,927 Bytes
2dd362e e19b228 1795ffe d3f9c5c 2dd362e 167d2a6 ab1a1d7 167d2a6 d3f9c5c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 |
---
license: llama3.1
language:
- en
pipeline_tag: text-generation
library_name: transformers
tags:
- phi-4
- LlamaForCausalLM
- xElite
- 14B
model-index:
- name: Blaze-14B-xElite
results:
- task:
type: text-generation
name: Text Generation
dataset:
name: IFEval (0-Shot)
type: wis-k/instruction-following-eval
split: train
args:
num_few_shot: 0
metrics:
- type: inst_level_strict_acc and prompt_level_strict_acc
value: 3.63
name: averaged accuracy
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FBlaze-14B-xElite
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: BBH (3-Shot)
type: SaylorTwift/bbh
split: test
args:
num_few_shot: 3
metrics:
- type: acc_norm
value: 51.57
name: normalized accuracy
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FBlaze-14B-xElite
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MATH Lvl 5 (4-Shot)
type: lighteval/MATH-Hard
split: test
args:
num_few_shot: 4
metrics:
- type: exact_match
value: 35.88
name: exact match
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FBlaze-14B-xElite
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: GPQA (0-shot)
type: Idavidrein/gpqa
split: train
args:
num_few_shot: 0
metrics:
- type: acc_norm
value: 19.24
name: acc_norm
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FBlaze-14B-xElite
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MuSR (0-shot)
type: TAUR-Lab/MuSR
args:
num_few_shot: 0
metrics:
- type: acc_norm
value: 17.68
name: acc_norm
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FBlaze-14B-xElite
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MMLU-PRO (5-shot)
type: TIGER-Lab/MMLU-Pro
config: main
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 45.68
name: accuracy
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FBlaze-14B-xElite
name: Open LLM Leaderboard
---
# **Blaze-14B-xElite**
[Blaze-14B-xElite finetuned] is a state-of-the-art open model built on the LLaMA-based model architecture. It has been fine-tuned using a blend of synthetic datasets, data from filtered public domain websites, and acquired academic books and Q&A datasets. The goal of this approach is to ensure that small yet capable models are trained with high-quality data focused on advanced reasoning.
Blaze-14B-xElite has adopted a robust safety post-training approach. This approach leverages a variety of both open-source and in-house generated synthetic datasets. The overall technique employed to achieve safety alignment combines SFT (Supervised Fine-Tuning) and iterative DPO (Direct Preference Optimization), including publicly available datasets focusing on helpfulness and harmlessness as well as various questions and answers targeted at multiple safety categories.
# **Dataset Info**
Blaze-14B-xElite is fine-tuned on a synthetic dataset curated through a pipeline explicitly built for this purpose. The data is primarily based on the Chain of Thought (CoT) or Chain of Continuous Flow methodologies. This approach ensures that the dataset is rich in reasoning, problem-solving, and step-by-step breakdowns of complex tasks. The model is specifically designed to excel in reasoning, mathematics, and breaking down problems into logical, manageable steps.
# **Run with Transformers**
```python
# pip install accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
tokenizer = AutoTokenizer.from_pretrained("prithivMLmods/Blaze-14B-xElite")
model = AutoModelForCausalLM.from_pretrained(
"prithivMLmods/Blaze-14B-xElite",
device_map="auto",
torch_dtype=torch.bfloat16,
)
input_text = "Write me a poem about Machine Learning."
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
outputs = model.generate(**input_ids, max_new_tokens=32)
print(tokenizer.decode(outputs[0]))
```
You can ensure the correct chat template is applied by using `tokenizer.apply_chat_template` as follows:
```python
messages = [
{"role": "user", "content": "Write me a poem about Machine Learning."},
]
input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True).to("cuda")
outputs = model.generate(**input_ids, max_new_tokens=256)
print(tokenizer.decode(outputs[0]))
```
# **Intended Use**
The Blaze-14B-xElite model is designed for a wide range of applications, particularly those requiring advanced reasoning, high-quality text generation, and multilingual capabilities. Below are some of the intended use cases:
1. **Complex Reasoning Tasks**:
- Solving intricate problems in mathematics, logic, and science.
- Assisting in academic research by providing detailed explanations and summaries.
2. **Multilingual Applications**:
- Translating text across multiple languages while preserving context and nuance.
- Generating content in various languages for global audiences.
3. **Content Creation**:
- Assisting writers, marketers, and creators with high-quality text generation.
- Generating creative ideas, stories, and technical documentation.
4. **Educational Tools**:
- Providing explanations, tutoring, and Q&A support for students and educators.
- Generating practice questions and answers for learning purposes.
5. **Customer Support**:
- Automating responses to customer queries with accurate and helpful information.
- Handling complex customer service scenarios with advanced reasoning.
6. **Safety-Critical Applications**:
- Ensuring responses are aligned with safety guidelines, making it suitable for sensitive domains.
- Providing harmlessness-focused interactions in public-facing applications.
# **Limitations**
While Blaze-14B-xElite is a powerful and versatile model, it has certain limitations that users should be aware of:
1. **Bias and Fairness**:
- Despite rigorous training and safety alignment, the model may still exhibit biases present in the training data. Users should critically evaluate outputs, especially in sensitive contexts.
2. **Contextual Understanding**:
- The model may occasionally misinterpret complex or ambiguous prompts, leading to inaccurate or irrelevant responses.
3. **Real-Time Knowledge**:
- The model's knowledge is limited to the data it was trained on and does not include real-time or post-training updates. It may not be aware of recent events or developments.
4. **Safety and Harmlessness**:
- While extensive efforts have been made to align the model with safety guidelines, it may still generate outputs that are inappropriate or harmful in certain contexts. Continuous monitoring and human oversight are recommended.
5. **Resource Requirements**:
- Running the model efficiently may require significant computational resources, especially for large-scale or real-time applications.
6. **Ethical Considerations**:
- The model should not be used for malicious purposes, such as generating harmful content, misinformation, or spam. Users are responsible for ensuring ethical use.
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/prithivMLmods__Blaze-14B-xElite-details)!
Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=prithivMLmods%2FBlaze-14B-xElite&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)!
| Metric |Value (%)|
|-------------------|--------:|
|**Average** | 28.95|
|IFEval (0-Shot) | 3.63|
|BBH (3-Shot) | 51.57|
|MATH Lvl 5 (4-Shot)| 35.88|
|GPQA (0-shot) | 19.24|
|MuSR (0-shot) | 17.68|
|MMLU-PRO (5-shot) | 45.68|
|