README.md

---
base_model:
- Qwen/QwQ-32B-Preview
tags:
- text-generation-inference
- transformers
- unsloth
- qwen2
- trl
- Chain-of-thought
- Reasoning
license: apache-2.0
language:
- en
new_version: Daemontatox/CogitoZ
library_name: transformers
datasets:
- PJMixers/Math-Multiturn-100K-ShareGPT
model-index:
- name: CogitoZ
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: IFEval (0-Shot)
      type: wis-k/instruction-following-eval
      split: train
      args:
        num_few_shot: 0
    metrics:
    - type: inst_level_strict_acc and prompt_level_strict_acc
      value: 39.67
      name: averaged accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=Daemontatox%2FCogitoZ
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: BBH (3-Shot)
      type: SaylorTwift/bbh
      split: test
      args:
        num_few_shot: 3
    metrics:
    - type: acc_norm
      value: 53.89
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=Daemontatox%2FCogitoZ
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MATH Lvl 5 (4-Shot)
      type: lighteval/MATH-Hard
      split: test
      args:
        num_few_shot: 4
    metrics:
    - type: exact_match
      value: 46.3
      name: exact match
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=Daemontatox%2FCogitoZ
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GPQA (0-shot)
      type: Idavidrein/gpqa
      split: train
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 19.35
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=Daemontatox%2FCogitoZ
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MuSR (0-shot)
      type: TAUR-Lab/MuSR
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 19.94
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=Daemontatox%2FCogitoZ
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU-PRO (5-shot)
      type: TIGER-Lab/MMLU-Pro
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 51.03
      name: accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=Daemontatox%2FCogitoZ
      name: Open LLM Leaderboard
---
![image](./image.webp)
# CogitoZ - 32B

## Model Overview

CogitoZ - 32B is a state-of-the-art large language model fine-tuned to excel in advanced reasoning and real-time decision-making tasks. This enhanced version was trained using [Unsloth](https://github.com/unslothai/unsloth), achieving a 2x faster training process. Leveraging Hugging Face's TRL (Transformers Reinforcement Learning) library, CogitoZ combines efficiency with exceptional reasoning performance.

- **Developed by**: Daemontatox
- **License**: Apache 2.0
- **Base Model**: [Qwen/QwQ-32B-Preview](https://huggingface.co/Qwen/QwQ-32B-Preview)
- **Finetuned To**: [Daemontatox/CogitoZ](https://huggingface.co/Daemontatox/CogitoZ)

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)

---

## Key Features

1. **Fast Training**: Optimized with Unsloth, achieving a 2x faster training cycle without compromising model quality.
2. **Enhanced Reasoning**: Utilizes advanced chain-of-thought (CoT) reasoning for solving complex problems.
3. **Quantization Ready**: Supports 8-bit and 4-bit quantization for deployment on resource-constrained devices.
4. **Scalable Inference**: Seamless integration with text-generation-inference tools for real-time applications.

---

## Intended Use

### Primary Use Cases
- **Education**: Real-time assistance for complex problem-solving, especially in mathematics and logic.
- **Business**: Supports decision-making, financial modeling, and operational strategy.
- **Healthcare**: Enhances diagnostic accuracy and supports structured clinical reasoning.
- **Legal Analysis**: Simplifies complex legal documents and constructs logical arguments.

### Limitations
- May produce biased outputs if the input prompts contain prejudicial or harmful content.
- Should not be used for real-time, high-stakes autonomous decisions (e.g., robotics or autonomous vehicles).

---

## Technical Details

- **Training Framework**: Hugging Face's Transformers and TRL libraries.
- **Optimization Framework**: Unsloth for faster and efficient training.
- **Language Support**: English.
- **Quantization**: Compatible with 8-bit and 4-bit inference modes for deployment on edge devices.

### Deployment Example

#### Using Hugging Face Transformers:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Daemontatox/CogitoZ"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "Explain the Pythagorean theorem step-by-step:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Optimized Inference:
Install the transformers and text-generation-inference libraries.
Deploy on servers or edge devices using quantized models for optimal performance.
Training Data
The fine-tuning process utilized reasoning-specific datasets, including:


 **MATH Dataset**: Focused on logical and mathematical problems.
 
 **Custom Corpora**: Tailored datasets for multi-domain reasoning and structured problem-solving.


## Ethical Considerations
**Bias Awareness** **->** The model reflects biases present in the training data. Users should carefully evaluate outputs in sensitive contexts.

**Safe Deployment** **->**  Not recommended for generating harmful or unethical content.

## Acknowledgments
This model was developed with contributions from Daemontatox and the Unsloth team, utilizing state-of-the-art techniques in fine-tuning and optimization.
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/Daemontatox__CogitoZ-details)!
Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=Daemontatox%2FCogitoZ&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)!

|      Metric       |Value (%)|
|-------------------|--------:|
|**Average**        |    38.36|
|IFEval (0-Shot)    |    39.67|
|BBH (3-Shot)       |    53.89|
|MATH Lvl 5 (4-Shot)|    46.30|
|GPQA (0-shot)      |    19.35|
|MuSR (0-shot)      |    19.94|
|MMLU-PRO (5-shot)  |    51.03|