noneUsername
/

Rombos-LLM-V2.6-Qwen-14b-W8A8-Dynamic-Per-Token

8-bit precision

Model card Files Files and versions Community

Rombos-LLM-V2.6-Qwen-14b-W8A8-Dynamic-Per-Token / README.md

noneUsername's picture

Create README.md

1b4797a verified 3 months ago

|

history blame contribute delete

1.05 kB

	---
	base_model:
	- rombodawg/Rombos-LLM-V2.6-Qwen-14b
	---
	vllm (pretrained=/root/autodl-tmp/Rombos-LLM-V2.6-Qwen-14b,add_bos_token=true,tensor_parallel_size=2,max_model_len=2048,dtype=bfloat16), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: auto
	\|Tasks\|Version\| Filter \|n-shot\| Metric \| \|Value\| \|Stderr\|
	\|-----\|------:\|----------------\|-----:\|-----------\|---\|----:\|---\|-----:\|
	\|gsm8k\| 3\|flexible-extract\| 5\|exact_match\|↑ \|0.656\|± \|0.0301\|
	\| \| \|strict-match \| 5\|exact_match\|↑ \|0.864\|± \|0.0217\|

	vllm (pretrained=/root/autodl-tmp/output87,add_bos_token=true,tensor_parallel_size=2,max_model_len=2048,dtype=bfloat16), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: auto
	\|Tasks\|Version\| Filter \|n-shot\| Metric \| \|Value\| \|Stderr\|
	\|-----\|------:\|----------------\|-----:\|-----------\|---\|----:\|---\|-----:\|
	\|gsm8k\| 3\|flexible-extract\| 5\|exact_match\|↑ \|0.692\|± \|0.0293\|
	\| \| \|strict-match \| 5\|exact_match\|↑ \|0.876\|± \|0.0209\|