---
library_name: transformers
pipeline_tag: text-generation
tags:
- lora
license: cc-by-nc-4.0
---

# Model Card for LoRA Adapters

This model card describes LoRA adapters fine-tuned from Llama-3.1-8B-Instruct to incorporate new knowledge while trying to maintain previously learned information. It explores the limitations of LoRA-based LLM updates.

## Model Details

*   **Developed by:** [Sergey Pletenev, Maria Marina, Daniil Moskovskiy, Vasily Konovalov, Pavel Braslavski, Alexander Panchenko, Mikhail Salnikov]
*   **Model type:** Causal language model, LoRA adapters
*   **Language(s) (NLP):** English
*   **License:** CC-BY-NC-4.0
*   **Finetuned from model:** [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)

### Model Sources

*   **Repository:** This repository.
*   **Paper:** [How Much Knowledge Can You Pack into a LoRA Adapter without Harming LLM?](https://arxiv.org/abs/2502.14502)

## Uses

### Direct Use

These LoRA adapters are intended to be used with the base Llama-3.1-8B-Instruct model for text generation tasks, particularly in scenarios where incorporating new knowledge is desired.

### Downstream Use

These LoRA adapters can be integrated into question-answering systems, chatbots, or other applications that require up-to-date information. However, caution is advised due to potential performance degradation on external question-answering benchmarks and a tendency towards biased answers.

### Out-of-Scope Use

The model should not be used in applications where biased or inaccurate information could have serious consequences, such as medical or legal advice.

## Bias, Risks, and Limitations

The model exhibits the following biases, risks, and limitations:

*   **Performance Degradation:** Performance on external question-answering benchmarks may decline after fine-tuning.
*   **Answer Bias:** The model may regress to few overrepresented answers when the training data is biased towards certain entities.
*   **Overconfidence:** The model becomes more confident and may refuse to provide an answer in fewer cases, even when uncertain.

### Recommendations

Users should be aware of the risks, biases, and limitations of the model. When incorporating new knowledge, ensure that the training data contains a balanced mixture of known and new facts. Carefully tune parameters to balance new knowledge integration and general model capabilities.

## How to Get Started with the Model

1.  Install the necessary libraries:

```bash
pip install transformers peft accelerate
```

2.  Load the base model and LoRA adapter:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

model_name = "meta-llama/Meta-Llama-3.1-8B-Instruct"
adapter_name = "YOUR_ADAPTER_NAME"  # Replace with the actual adapter name

model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.float16)
model = PeftModel.from_pretrained(model, adapter_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

model.eval()

prompt = "What is the capital of France?"
input_ids = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(**input_ids, max_new_tokens=50)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Training Details

### Training Data

The model was fine-tuned using LoRA on the base Llama-3.1-8B-instruct model. The training data consists of a mixture of known and new facts. The new facts were created using a head-to-tail pipeline, generating questions and answers using templates and information extracted from a Dbpedia dump.

### Training Procedure

The model was trained using LoRA (Low-Rank Adaptation). The training data composition and tuning parameters are crucial for balancing new knowledge integration and general model capabilities.

#### Training Hyperparameters

*   **Training regime:** LoRA fine-tuning
*   **Base Model:** Llama-3.1-8B-instruct

## Evaluation

### Testing Data, Factors & Metrics

*   **Testing Data:** External question-answering benchmarks were used to evaluate the model's performance on previously learned knowledge.
*   **Metrics:** The primary metric was accuracy on the question-answering benchmarks. Additionally, the model's confidence and refusal rate were analyzed.

### Results

Experiments have shown that fine-tuning with LoRA can lead to a decline in performance on external question-answering benchmarks. The best results are obtained when the training data contains a mixture of known and new facts, but this approach is still potentially harmful.

#### Summary

The model's performance is sensitive to the composition of the training data. Fine-tuning with biased data can lead to a regression towards overrepresented answers and a decline in overall performance.

## Citation

[How Much Knowledge Can You Pack into a LoRA Adapter without Harming LLM?](https://arxiv.org/abs/2502.14502)

**BibTeX:**

```
@misc{pletenev2025knowledgepackloraadapter,
      title={How Much Knowledge Can You Pack into a LoRA Adapter without Harming LLM?}, 
      author={Sergey Pletenev and Maria Marina and Daniil Moskovskiy and Vasily Konovalov and Pavel Braslavski and Alexander Panchenko and Mikhail Salnikov},
      year={2025},
      eprint={2502.14502},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2502.14502}, 
}
```