paulagarciaserrano/roberta-depression-detection

Roberta for depression signs detection

This model is a fine-tuned version the cardiffnlp/twitter-roberta-base model. It has been trained using a recently published corpus: Shared task on Detecting Signs of Depression from Social Media Text at LT-EDI 2022-ACL 2022.

The obtained macro f1-score is 0.54, on the development set of the competition.

Intended uses

This model is trained to classify the given text into one of the following classes: moderate, severe, or not depression. It corresponds to a multiclass classification task.

How to use

You can use this model directly with a pipeline for text classification:

>>> from transformers import pipeline
>>> classifier = pipeline("text-classification", model="paulagarciaserrano/roberta-depression-detection")
>>> your_text = "I am very sad."
>>> classifier (your_text)

Training and evaluation data

The train dataset characteristics are:

Class	Nº sentences	Avg. document length (in sentences)	Nº words	Avg. sentence length (in words)
not depression	7,884	4	153,738	78
moderate	36,114	6	601,900	100
severe	9,911	11	126,140	140

Similarly, the evaluation dataset characteristics are:

Class	Nº sentences	Avg. document length (in sentences)	Nº words	Avg. sentence length (in words)
not depression	3,660	2	10,980	6
moderate	66,874	29	804,794	349
severe	2,880	8	75,240	209

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• evaluation_strategy: epoch
• save_strategy: epoch
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 8
• num_train_epochs: 5
• seed: 10
• weight_decay: 0.01
• metric_for_best_model: macro-f1