|
--- |
|
base_model: Pinkstack/PARM-V1.5-QwQ-Qwen-2.5-o1-3B-VLLM |
|
tags: |
|
- text-generation-inference |
|
- transformers |
|
- unsloth |
|
- qwen2 |
|
- gguf |
|
- Reasoning |
|
- o1 |
|
- qwq |
|
license: apache-2.0 |
|
language: |
|
- en |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
 |
|
|
|
⚠️ it may think it's name is Claude due to the training data, we are sorry for this issue but is shouldn't effect the quality of the responses. |
|
|
|
🧀 Which quant is right for you? |
|
|
|
- ***Q4:*** This model should be used on edge devices like high end phones or laptops due to its very compact size, quality is okay but fully usable. |
|
- ***Q8:*** This model should be used on most high end modern devices like rtx 3080, Responses are very high quality, but its noticeably slower than q4 |
|
|
|
This Parm v2 is based on Qwen 2.5 3B which has gotten many extra reasoning training parameters so it would have similar outputs to qwen QwQ / O.1 mini (only much, smaller.). We've trained it using the datasets [here](https://huggingface.co/collections/Pinkstackorg/pram-v2-67612d3c542b9121bf15891c) |
|
|
|
This is a pretty heavy to run model if you want on device ai's for phones I'd recommend using the 0.5B version of this model (coming soon) |
|
|
|
|
|
To use this model, you must use a service which supports the GGUF file format. |
|
Additionaly, this is the Prompt Template: it uses the qwen2 template. |
|
``` |
|
{{ if .System }}<|system|> |
|
{{ .System }}<|end|> |
|
{{ end }}{{ if .Prompt }}<|user|> |
|
{{ .Prompt }}<|end|> |
|
{{ end }}<|assistant|> |
|
{{ .Response }}<|end|> |
|
``` |
|
|
|
Or if you are using an anti prompt: <|end|><|assistant|> |
|
|
|
Highly recommended to use with a system prompt. |
|
|
|
|
|
# Uploaded model |
|
|
|
- **Developed by:** Pinkstack |
|
- **License:** apache-2.0 |
|
- **Finetuned from model :** Pinkstack/PARM-V1.5-QwQ-Qwen-2.5-o1-3B-VLLM |
|
|
|
This ai model was trained with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library. |
