README.md

metadata

license: apache-2.0
base_model: google/mt5-small
tags:
  - summarization
  - generated_from_trainer
  - music
  - song-lyrics
metrics:
  - rouge
model-index:
  - name: mt5-small-finetuned-genius
    results: []
pipeline_tag: summarization
datasets:
  - miscjose/genius-music
widget:
  - text: >
      Thought I'd end up with Sean  But he wasn't a match \n  Wrote some songs
      about Ricky Now I listen and laugh Even almost got married And for Pete,
      I'm so thankful Wish I could say, "Thank you" to Malcolm 'Cause he was an
      angel One taught me love One taught me patience And one taught me pain
      Now, I'm so amazing Say I've loved and I've lost But that's not what I see
      So, look what I got Look what you taught me And for that, I say Thank you,
      next (Next) Thank you, next (Next) Thank you, next I'm so fuckin' grateful
      for my ex Thank you, next (Next) Thank you, next (Next) Thank you, next
      (Next) I'm so fuckin'— Spend more time with my friends I ain't worried
      'bout nothin' Plus, I met someone else We havin' better discussions I know
      they say I move on too fast But this one gon' last 'Cause her name is Ari
      And I'm so good with that (So good with that) She taught me love (Love)
      She taught me patience (Patience) How she handles pain (Pain) That shit's
      amazing (Yeah, she's amazing) I've loved and I've lost (Yeah, yeah) But
      that's not what I see (Yeah, yeah) 'Cause look what I've found (Yeah,
      yeah, I've found) Ain't no need for searching, and for that, I say Thank
      you, next (Thank you, next) Thank you, next (Thank you, next) Thank you,
      next (Thank you) I'm so fuckin' grateful for my ex Thank you, next (Thank
      you, next) Thank you, next (Said thank you, next) Thank you, next (Next)
      I'm so fuckin' grateful for my ex Thank you, next Thank you, next Thank
      you, next I'm so fuckin'— One day I'll walk down the aisle Holding hands
      with my mama I'll be thanking my dad 'Cause she grew from the drama Only
      wanna do it once, real bad Gon' make that shit last God forbid something
      happens Least this song is a smash (Song is a smash) I've got so much love
      (Love) Got so much patience (Patience) And I've learned from the pain
      (Pain) I turned out amazing (Turned out amazing) Say I've loved and I've
      lost (Yeah, yeah) But that's not what I see (Yeah, yeah) 'Cause look what
      I've found (Yeah, yeah) Ain't no need for searching And for that, I say
      Thank you, next (Thank you, next) Thank you, next (Thank you, next) Thank
      you, next I'm so fuckin' grateful for my ex Thank you, next (Thank you,
      next) Thank you, next (Said thank you, next) Thank you, next (Next) I'm so
      fuckin' grateful for my ex Thank you, next Thank you, next Thank you, next
      Yeah, yee Thank you, next Thank you, next Thank you, next Yeah, yee

mt5-small-finetuned-genius

This model is a fine-tuned version of google/mt5-small on the Genius Music dataset found here. The song lyrics and song titles were preprocessed and used for fine-tuning.

You can view more examples of this model's inference on the following Space.

Model description

Please visit: google/mt5-small

Intended uses & limitations

• Intended Uses: Given song lyrics, generate a summary.
• Limitations: Due to the nature of music, the model can generate summaries containing hate speech.

Training and evaluation data

• 27.6K Training Samples
• 3.45 Validation Samples

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
7.9304	1.0	863	3.5226	14.235	6.78	14.206	14.168
3.8394	2.0	1726	3.0382	22.97	13.166	22.981	22.944
3.3799	3.0	2589	2.9010	24.932	14.54	24.929	24.919
3.2204	4.0	3452	2.8441	26.678	15.587	26.624	26.665
3.1498	5.0	4315	2.8363	26.827	15.696	26.773	26.793

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.14.1
• Tokenizers 0.13.3