File size: 2,916 Bytes
aaf57ee |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 |
# Model Details
**Model Name:** Fine-Tuned BART for Customer Support Resolution Generation
**Base Model:** facebook/bart-base
**Dataset:** bitext/Bitext-customer-support-llm-chatbot-training-dataset
**Training Device:** CUDA (GPU)
---
# Dataset Information
**Dataset Structure:**
DatasetDict({
train: Dataset({
features: ['input_text', 'target_text'],
num_rows: 24184
})
validation: Dataset({
features: ['input_text', 'target_text'],
num_rows: 2688
})
})
**Available Splits:**
- **Train:** 24,184 examples
- **Validation:** 2,688 examples
**Feature Representation:**
- **input_text:** Customer issue text (e.g., "Customer: How do I cancel my order?")
- **target_text:** Resolution text (e.g., "Log into the portal and cancel it there.")
---
# Training Details
**Training Process:**
- Fine-tuned for 3 epochs
- Loss reduced progressively across epochs
**Hyperparameters:**
- Epochs: 3
- Learning Rate: 2e-5
- Batch Size: 8
- Weight Decay: 0.01
- Mixed Precision: FP16
**Performance Metrics:**
- Final Training Loss: ~0.0140
- Final Validation Loss: ~0.0121
---
# Inference Example
```python
import torch
from transformers import BartForConditionalGeneration, BartTokenizer
def load_model(model_path):
tokenizer = BartTokenizer.from_pretrained(model_path)
model = BartForConditionalGeneration.from_pretrained(model_path).half() # FP16
model.eval()
return model, tokenizer
def generate_resolution(issue, model, tokenizer, device="cuda"):
input_text = f"Customer: {issue}"
inputs = tokenizer(
input_text,
max_length=512,
padding="max_length",
truncation=True,
return_tensors="pt"
).to(device)
outputs = model.generate(
inputs["input_ids"],
max_length=128,
num_beams=4,
early_stopping=True
)
return tokenizer.decode(outputs[0], skip_special_tokens=True)
# Example usage
if __name__ == "__main__":
model_path = "your-username/bart-resolution-summarizer-fp16" # Replace with your HF repo
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model, tokenizer = load_model(model_path)
model.to(device)
issue = "How do I cancel my order?"
resolution = generate_resolution(issue, model, tokenizer, device)
print(f"Issue: {issue}")
print(f"Resolution: {resolution}")
Expected Output:
Issue: How do I cancel my order?
Resolution: Log into the portal and cancel it there.
```
# Limitations
Model may struggle with issues requiring specific resolutions not well-represented in the training data (e.g., time-related queries like "When can I call support?").
Resolution extraction relied on heuristics, potentially missing nuanced answers in verbose responses.
# Future Improvements
Refine resolution extraction with more advanced NLP techniques or manual curation.
Fine-tune on additional customer support datasets for broader coverage.
|