Hugging Face's logo

amang1802
/
Llama3.2-1B-summary-length-exp6

Text Generation
Transformers
Safetensors
llama
conversational
text-generation-inference
Inference Endpoints
Model card Files Community

Model Card for Model ID

  • Summary Length PPO experiment #5
  • No KL divergence in loss

Model Details

  • Dataset size: 1024
  • Epochs: 1
  • Batch Size: 4 * 4 (w/ 4 GPUs) * 8 (w/ Gradient Accumulation)

Optimizer args: Torch AdamW default, except

  • LR = 0.00001
Downloads last month
6
Safetensors
Model size
1.24B params
Tensor type
BF16
·
Inference Providers NEW
Text Generation
This model is not currently available via any of the supported Inference Providers.

Collection including amang1802/Llama3.2-1B-summary-length-exp6