Hugging Face's logo

mihalca
/
bert_model_ro_fake_news

Text Classification
Transformers.js
ONNX
Romanian
bert
romanian
fake news
BERT
Model card Files Community
bert_model_ro_fake_news
File size: 4,176 Bytes 
431b297
 
 
 
 
 
5c62cc8
431b297
 
 
 
 
 
5c62cc8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
431b297
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
 
---
license: mit
language:
- ro
metrics:
- accuracy
base_model: google-bert/bert-base-multilingual-cased
library_name: transformers.js
tags:
- romanian
- fake news
- BERT
---
# Model Card for BERT 

This model classifies Romanian fake news in ["fake_news", "misinformation", "propaganda", "real_news", "satire"]

## Model Details

### Model Description

This model is a BERT-based model fine-tuned for the task of detecting fake news in Romanian text. It is designed to classify text into one of several predefined labels indicating the likelihood of the text being fake news.


- **Developed by:** Bogdan Mihalca
- **Model type:** BERT
- **License:** MIT
- **Finetuned from model:** bert-base-multilingual-cased

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]

## Uses

### Direct Use
This model can be directly used to classify Romanian text into five categories: fake news, misinformation, propaganda, real news, and satire.
It can be handy for media monitoring platforms, news verifiers, and research on misinformation.


### Downstream Use

Potential applications include integration into larger systems for real-time news validation or being part of a research toolkit for studying misinformation.

### Out-of-Scope Use

This model is not intended for classifying non-Romanian text, and its accuracy may diminish significantly if used on text outside of the Romanian language.
It is also not suitable for detailed sentiment analysis or non-news related content.


## Bias, Risks, and Limitations

This model may exhibit bias due to the nature of the training data, which could lead to overfitting on certain types of news or propaganda specific to Romanian contexts.
It might not generalize well to new, unseen forms of misinformation or satire.


### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
Users should evaluate the model's performance on their specific use case before deployment, particularly if applied in contexts different from those in which the training data was collected.


## How to Get Started with the Model

To get started with this model, load it using HuggingFace's Transformers library and ensure you have the appropriate tokenizer
. Example code for loading and using the model is provided in the Python script.

## Training Details

### Training Data
The model was trained on a combined dataset of Romanian fake news, including data scraped from Veridica.ro and the Fakerom dataset.
The dataset includes categories such as fake news, misinformation, propaganda, real news, and satire.
### Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

#### Preprocessing [optional]

[More Information Needed]


#### Training Hyperparameters

- **Training regime:** The model was trained using fp32 precision.
- **Number of epochs:** 3
- **Batch size:** 16
- **Warmup steps:** 500
- **Weight decay:** 0.01
- **Learning rate scheduler:** Warmup with cosine decay.


## Evaluation


### Testing Data, Factors & Metrics

#### Testing Data

The model was tested on a split of the combined dataset, with 20% of the data used for evaluation.

#### Factors
The evaluation was performed across all five classes of fake news categories, with results disaggregated by each category.


#### Metrics

The primary evaluation metrics were accuracy, precision, recall, F1 score, and log loss. ROC AUC scores were also computed for each class.


### Results

Results

    Accuracy: 93.51%
    Precision: 93.80%
    Recall: 93.51%
    F1 Score: 93.56%
    Log Loss: 0.225
    ROC AUC Per Class:
        Fake News: 98.50%
        Misinformation: 98.89%
        Propaganda: 98.98%
        Real News: 99.55%
        Satire: 99.99%



#### Summary
The model demonstrates strong performance across all classes, with particularly high ROC AUC scores indicating good separability between classes.