--- license: apache-2.0 language: - en base_model: - Qwen/Qwen2.5-7B-Instruct pipeline_tag: text-generation library_name: transformers tags: - LCoT - Qwen - v2 datasets: - PowerInfer/QWQ-LONGCOT-500K - AI-MO/NuminaMath-CoT - prithivMLmods/Math-Solve - amphora/QwQ-LongCoT-130K - prithivMLmods/Deepthink-Reasoning model-index: - name: QwQ-LCoT2-7B-Instruct results: - task: type: text-generation name: Text Generation dataset: name: IFEval (0-Shot) type: wis-k/instruction-following-eval split: train args: num_few_shot: 0 metrics: - type: inst_level_strict_acc and prompt_level_strict_acc value: 55.76 name: averaged accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-LCoT2-7B-Instruct name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: BBH (3-Shot) type: SaylorTwift/bbh split: test args: num_few_shot: 3 metrics: - type: acc_norm value: 34.37 name: normalized accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-LCoT2-7B-Instruct name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MATH Lvl 5 (4-Shot) type: lighteval/MATH-Hard split: test args: num_few_shot: 4 metrics: - type: exact_match value: 22.21 name: exact match source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-LCoT2-7B-Instruct name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GPQA (0-shot) type: Idavidrein/gpqa split: train args: num_few_shot: 0 metrics: - type: acc_norm value: 6.38 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-LCoT2-7B-Instruct name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MuSR (0-shot) type: TAUR-Lab/MuSR args: num_few_shot: 0 metrics: - type: acc_norm value: 15.75 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-LCoT2-7B-Instruct name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU-PRO (5-shot) type: TIGER-Lab/MMLU-Pro config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 37.13 name: accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-LCoT2-7B-Instruct name: Open LLM Leaderboard --- # **QwQ-LCoT2-7B-Instruct** The *QwQ-LCoT2-7B-Instruct* is a fine-tuned language model designed for advanced reasoning and instruction-following tasks. It leverages the Qwen2.5-7B base model and has been fine-tuned on the chain of thought reasoning datasets, focusing on chain-of-thought (CoT) reasoning for problems. This model is optimized for tasks requiring logical reasoning, detailed explanations, and multi-step problem-solving, making it ideal for applications such as instruction-following, text generation, and complex reasoning tasks. # **Quickstart with Transformers** Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents. ```python from transformers import AutoModelForCausalLM, AutoTokenizer model_name = "prithivMLmods/QwQ-LCoT2-7B-Instruct" model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype="auto", device_map="auto" ) tokenizer = AutoTokenizer.from_pretrained(model_name) prompt = "How many r in strawberry." messages = [ {"role": "system", "content": "You are a helpful and harmless assistant. You are Qwen developed by Alibaba. You should think step-by-step."}, {"role": "user", "content": prompt} ] text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) model_inputs = tokenizer([text], return_tensors="pt").to(model.device) generated_ids = model.generate( **model_inputs, max_new_tokens=512 ) generated_ids = [ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids) ] response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] ``` # **Intended Use** The QwQ-LCoT2-7B-Instruct model is designed for advanced reasoning and instruction-following tasks, with specific applications including: 1. **Instruction Following**: Providing detailed and step-by-step guidance for a wide range of user queries. 2. **Logical Reasoning**: Solving problems requiring multi-step thought processes, such as math problems or complex logic-based scenarios. 3. **Text Generation**: Crafting coherent, contextually relevant, and well-structured text in response to prompts. 4. **Problem-Solving**: Analyzing and addressing tasks that require chain-of-thought (CoT) reasoning, making it ideal for education, tutoring, and technical support. 5. **Knowledge Enhancement**: Leveraging reasoning datasets to offer deeper insights and explanations for a wide variety of topics. # **Limitations** 1. **Data Bias**: As the model is fine-tuned on specific datasets, its outputs may reflect inherent biases from the training data. 2. **Context Limitation**: Performance may degrade for tasks requiring knowledge or reasoning that significantly exceeds the model's pretraining or fine-tuning context. 3. **Complexity Ceiling**: While optimized for multi-step reasoning, exceedingly complex or abstract problems may result in incomplete or incorrect outputs. 4. **Dependency on Prompt Quality**: The quality and specificity of the user prompt heavily influence the model's responses. 5. **Non-Factual Outputs**: Despite being fine-tuned for reasoning, the model can still generate hallucinated or factually inaccurate content, particularly for niche or unverified topics. 6. **Computational Requirements**: Running the model effectively requires significant computational resources, particularly when generating long sequences or handling high-concurrency workloads. # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/prithivMLmods__QwQ-LCoT2-7B-Instruct-details)! Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=prithivMLmods%2FQwQ-LCoT2-7B-Instruct&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)! | Metric |Value (%)| |-------------------|--------:| |**Average** | 28.60| |IFEval (0-Shot) | 55.76| |BBH (3-Shot) | 34.37| |MATH Lvl 5 (4-Shot)| 22.21| |GPQA (0-shot) | 6.38| |MuSR (0-shot) | 15.75| |MMLU-PRO (5-shot) | 37.13|