metadata

base_model:
  - ISTA-DASLab/Mixtral-8x7B-Instruct-v0_1-AQLM-2Bit-1x16-hf
tags:
  - text-generation-inference
  - transformers
language:
  - en
pipeline_tag: text-generation
license: apache-2.0

Model Card for ORANSight Mistral-8x7B

This model belongs to the first release of the ORANSight family of models.

Developed by: NextG lab@ NC State
License: apache-2.0
Context Window: 32K
Fine Tuning Framework: AQLM

Note: This only contains the LoRA adapters and the Tokenizer obtained by leveraging QLoRA with AQLM's 2-Bit implementation.

Generate with Transformers

Below is a quick example of how to use the model with Hugging Face Transformers:

from transformers import pipeline

# Example query
messages = [
    {"role": "system", "content": "You are an O-RAN expert assistant."},
    {"role": "user", "content": "Explain the E2 interface."},
]

# Load the model
chatbot = pipeline("text-generation", model="NextGLab/ORANSight_Mixtral_8x7B_Instruct_AQLM")
result = chatbot(messages)
print(result)

Coming Soon

A detailed paper documenting the experiments and results achieved with this model will be available soon. Meanwhile, if you try this model, please cite the below mentioned paper to acknowledge the foundational work that enabled this fine-tuning.

@article{gajjar2024oran,
  title={Oran-bench-13k: An open source benchmark for assessing llms in open radio access networks},
  author={Gajjar, Pranshav and Shah, Vijay K},
  journal={arXiv preprint arXiv:2407.06245},
  year={2024}
}