--- datasets: - samsum language: - en metrics: - rouge library_name: transformers pipeline_tag: summarization tags: - summarization - conversational - seq2seq - bart large widget: - text: | Hannah: Hey, do you have Betty's number? Amanda: Lemme check Amanda: Sorry, can't find it. Amanda: Ask Larry Amanda: He called her last time we were at the park together Hannah: I don't know him well Amanda: Don't be shy, he's very nice Hannah: If you say so.. Hannah: I'd rather you texted him Amanda: Just text him 🙂 Hannah: Urgh.. Alright Hannah: Bye Amanda: Bye bye model-index: - name: bart-large-xsum-samsum-conversational_summarizer results: - task: name: Abstractive Text Summarization type: abstractive-text-summarization dataset: name: "SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization" type: samsum metrics: - name: Validation ROUGE-1 type: rouge-1 value: 54.3921 - name: Validation ROUGE-2 type: rouge-2 value: 29.8078 - name: Validation ROUGE-L type: rouge-l value: 45.1543 - name: Test ROUGE-1 type: rouge-1 value: 53.3059 - name: Test ROUGE-2 type: rouge-2 value: 28.355 - name: Test ROUGE-L type: rouge-l value: 44.0953 --- ## Usage ```python from transformers import pipeline summarizer_pipe = pipeline("summarization", model="yashugupta786/bart_large_xsum_samsum_conv_summarizer") conversation_data = '''Hannah: Hey, do you have Betty's number? Amanda: Lemme check Amanda: Sorry, can't find it. Amanda: Ask Larry Amanda: He called her last time we were at the park together Hannah: I don't know him well Amanda: Don't be shy, he's very nice Hannah: If you say so.. Hannah: I'd rather you texted him Amanda: Just text him 🙂 Hannah: Urgh.. Alright Hannah: Bye Amanda: Bye bye ''' summarizer_pipe(conversation_data) ``` ## Results | key | value | | --- | ----- | | eval_rouge1 | 54.3921 | | eval_rouge2 | 29.8078 | | eval_rougeL | 45.1543 | | eval_rougeLsum | 49.942 | | test_rouge1 | 53.3059 | | test_rouge2 | 28.355 | | test_rougeL | 44.0953 | | test_rougeLsum | 48.9246 | All the metric Rouge1,2,L are computed using precison and recall then computed the F measure for these Rouge recall= no of overlaping words/total no of referenced humman annotated words Rouge precision= no of overlaping words/total no of candidate machine predicted words