Daemontatox
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -1,96 +1,77 @@
|
|
1 |
-
|
2 |
-
|
3 |
-
|
4 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
5 |
|
6 |
-
|
7 |
-
- **Architecture**: Transformer-based
|
8 |
-
- **Base Model**: Qwen-0.5B
|
9 |
-
- **Fine-Tuning Dataset**: MATH and other reasoning-intensive corpora
|
10 |
-
- **Performance**:
|
11 |
-
- Ranked #2 on Hugging Face for the MATH dataset
|
12 |
-
- Real-time text-based reasoning capabilities
|
13 |
|
14 |
-
|
15 |
-
![image](./image.webp)
|
16 |
-
## Intended Use
|
17 |
-
### Primary Use Cases
|
18 |
-
- **Education**: Provide detailed, step-by-step solutions to mathematical and logical problems.
|
19 |
-
- **Research**: Assist with hypothesis generation, logical inference, and data-driven research.
|
20 |
-
- **Business**: Support complex decision-making processes, financial modeling, and strategic planning.
|
21 |
-
- **Legal and Policy Analysis**: Simplify legal texts, analyze policy documents, and construct logical arguments.
|
22 |
-
- **Healthcare**: Aid in diagnostic reasoning and structured decision support.
|
23 |
|
24 |
-
|
25 |
-
- Generating harmful, biased, or unethical content.
|
26 |
-
- Applications involving real-time physical world interactions (e.g., robotics or autonomous systems).
|
27 |
|
28 |
-
|
|
|
|
|
|
|
29 |
|
30 |
-
|
31 |
-
- **Model Type**: Fine-tuned Transformer
|
32 |
-
- **Training Objective**: Optimized for reasoning and logical problem solving using chain-of-thought prompts.
|
33 |
-
- **Language Support**: Multilingual, with an emphasis on English reasoning tasks.
|
34 |
-
- **Quantization**: Supports 8-bit and 4-bit quantization for deployment on resource-constrained devices (e.g., Raspberry Pi 5).
|
35 |
-
- **Key Features**:
|
36 |
-
- Advanced chain-of-thought (CoT) reasoning
|
37 |
-
- Enhanced multi-step problem-solving ability
|
38 |
-
- Scalable deployment on edge devices
|
39 |
|
40 |
---
|
41 |
|
42 |
-
##
|
43 |
-
The model was fine-tuned on:
|
44 |
-
- **MATH Dataset**: A large-scale dataset of mathematical problems with diverse complexity levels.
|
45 |
-
- **Additional Corpora**: Custom curated datasets focusing on logical reasoning, structured decision-making, and multi-domain inference.
|
46 |
|
47 |
-
|
48 |
-
|
|
|
|
|
49 |
|
50 |
---
|
51 |
|
52 |
-
##
|
53 |
-
### Benchmark Results
|
54 |
-
- **MATH Dataset**: Achieved a near-perfect accuracy of X% (specific score) on challenging problem sets.
|
55 |
-
- **Hugging Face Leaderboard**: Secured second place among nearly 2,000 models.
|
56 |
|
57 |
-
###
|
58 |
-
-
|
59 |
-
-
|
|
|
|
|
60 |
|
61 |
### Limitations
|
62 |
-
- May
|
63 |
-
-
|
64 |
|
65 |
---
|
66 |
|
67 |
-
##
|
68 |
-
### Inference
|
69 |
-
PathfinderAI 32b can be deployed using:
|
70 |
-
- **Hugging Face Transformers Library**:
|
71 |
-
```python
|
72 |
-
from transformers import AutoModelForCausalLM, AutoTokenizer
|
73 |
-
|
74 |
-
model_name = "PathfinderAI/32b-reasoning-model"
|
75 |
-
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
76 |
-
model = AutoModelForCausalLM.from_pretrained(model_name)
|
77 |
|
78 |
-
|
79 |
-
|
80 |
-
|
81 |
-
|
82 |
-
```
|
83 |
|
84 |
-
|
85 |
-
Ethical Considerations
|
86 |
-
Bias Mitigation: While efforts have been made to reduce biases during fine-tuning, users should be cautious of potential unintended outputs.
|
87 |
-
Safe Usage: The model should not be used for applications promoting harm, misinformation, or unethical practices.
|
88 |
|
|
|
|
|
|
|
89 |
|
|
|
|
|
|
|
90 |
|
91 |
-
|
92 |
-
|
93 |
-
|
94 |
-
|
95 |
-
year={2025}
|
96 |
-
}
|
|
|
1 |
+
---
|
2 |
+
base_model:
|
3 |
+
- Qwen/QwQ-32B-Preview
|
4 |
+
tags:
|
5 |
+
- text-generation-inference
|
6 |
+
- transformers
|
7 |
+
- unsloth
|
8 |
+
- qwen2
|
9 |
+
- trl
|
10 |
+
license: apache-2.0
|
11 |
+
language:
|
12 |
+
- en
|
13 |
+
metrics:
|
14 |
+
- accuracy
|
15 |
+
new_version: Daemontatox/CogitoZ
|
16 |
+
library_name: transformers
|
17 |
+
---
|
18 |
|
19 |
+
# CogitoZ - Qwen2
|
|
|
|
|
|
|
|
|
|
|
|
|
20 |
|
21 |
+
## Model Overview
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
22 |
|
23 |
+
CogitoZ - Qwen2 is a state-of-the-art large language model fine-tuned to excel in advanced reasoning and real-time decision-making tasks. This enhanced version was trained using [Unsloth](https://github.com/unslothai/unsloth), achieving a 2x faster training process. Leveraging Hugging Face's TRL (Transformers Reinforcement Learning) library, CogitoZ combines efficiency with exceptional reasoning performance.
|
|
|
|
|
24 |
|
25 |
+
- **Developed by**: Daemontatox
|
26 |
+
- **License**: Apache 2.0
|
27 |
+
- **Base Model**: [Qwen/QwQ-32B-Preview](https://huggingface.co/Qwen/QwQ-32B-Preview)
|
28 |
+
- **Finetuned from**: [Daemontatox/CogitoZ](https://huggingface.co/Daemontatox/CogitoZ)
|
29 |
|
30 |
+
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
31 |
|
32 |
---
|
33 |
|
34 |
+
## Key Features
|
|
|
|
|
|
|
35 |
|
36 |
+
1. **Fast Training**: Optimized with Unsloth, achieving a 2x faster training cycle without compromising model quality.
|
37 |
+
2. **Enhanced Reasoning**: Utilizes advanced chain-of-thought (CoT) reasoning for solving complex problems.
|
38 |
+
3. **Quantization Ready**: Supports 8-bit and 4-bit quantization for deployment on resource-constrained devices.
|
39 |
+
4. **Scalable Inference**: Seamless integration with text-generation-inference tools for real-time applications.
|
40 |
|
41 |
---
|
42 |
|
43 |
+
## Intended Use
|
|
|
|
|
|
|
44 |
|
45 |
+
### Primary Use Cases
|
46 |
+
- **Education**: Real-time assistance for complex problem-solving, especially in mathematics and logic.
|
47 |
+
- **Business**: Supports decision-making, financial modeling, and operational strategy.
|
48 |
+
- **Healthcare**: Enhances diagnostic accuracy and supports structured clinical reasoning.
|
49 |
+
- **Legal Analysis**: Simplifies complex legal documents and constructs logical arguments.
|
50 |
|
51 |
### Limitations
|
52 |
+
- May produce biased outputs if the input prompts contain prejudicial or harmful content.
|
53 |
+
- Should not be used for real-time, high-stakes autonomous decisions (e.g., robotics or autonomous vehicles).
|
54 |
|
55 |
---
|
56 |
|
57 |
+
## Technical Details
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
58 |
|
59 |
+
- **Training Framework**: Hugging Face's Transformers and TRL libraries.
|
60 |
+
- **Optimization Framework**: Unsloth for faster and efficient training.
|
61 |
+
- **Language Support**: English.
|
62 |
+
- **Quantization**: Compatible with 8-bit and 4-bit inference modes for deployment on edge devices.
|
|
|
63 |
|
64 |
+
### Deployment Example
|
|
|
|
|
|
|
65 |
|
66 |
+
#### Using Hugging Face Transformers:
|
67 |
+
```python
|
68 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
69 |
|
70 |
+
model_name = "Daemontatox/CogitoZ"
|
71 |
+
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
72 |
+
model = AutoModelForCausalLM.from_pretrained(model_name)
|
73 |
|
74 |
+
prompt = "Explain the Pythagorean theorem step-by-step:"
|
75 |
+
inputs = tokenizer(prompt, return_tensors="pt")
|
76 |
+
outputs = model.generate(**inputs)
|
77 |
+
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
|
|
|