README.md

---
base_model:
- InferenceIllusionist/Excalibur-7b
library_name: transformers
tags:
- finetune
- dpo
- chatml
license: apache-2.0
datasets:
- Intel/orca_dpo_pairs
---


# Excalibur-7b-DPO

<img src="https://i.imgur.com/pbPbqq0.jpeg" width="550"/>

An initial foray into the world of fine-tuning. The goal of this release was to amplify the quality of the original model's responses, in particular for vision use cases*

<b>GGUFs available [here](https://huggingface.co/InferenceIllusionist/Excalibur-7b-DPO-GGUF)</b>

## Notes & Methodology
* [Excalibur-7b](https://huggingface.co/InferenceIllusionist/Excalibur-7b) fine-tuned with Direct Preference Optimization (DPO) using Intel/orca_dpo_pairs
* This is a quick experiment to determine the impact of DPO finetuning on the original base model
* Ran for a little over an hour on a single A100
* Internal benchmarks showed improvement over base model, awaiting final results
* Precision: bfloat16


## Sample Question - Vision
<img src="https://i.imgur.com/7aRWtzU.jpeg" width="425"/>

*<b>Requires additional mmproj file. You have two options for vision functionality (available inside this repo):</b>
 * [Quantized - Limited VRAM Option (197mb)](https://huggingface.co/InferenceIllusionist/Excalibur-7b-DPO-GGUF/resolve/main/mistral-7b-mmproj-v1.5-Q4_1.gguf?download=true)
 * [Unquantized - Premium Option / Best Quality (596mb)](https://huggingface.co/InferenceIllusionist/Excalibur-7b-DPO-GGUF/resolve/main/mmproj-model-f16.gguf?download=true)

Select the gguf file of your choice in [Koboldcpp](https://github.com/LostRuins/koboldcpp/releases/) as usual, then make sure to choose the mmproj file above in the LLaVA mmproj field of the model submenu:
<img src="https://i.imgur.com/x8vqH29.png" width="425"/>

## Prompt Format
* For best results please use ChatML for the prompt format. Alpaca may also work.