Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# SecRoBERTa
|
| 2 |
+
|
| 3 |
+
This is the pretrained model presented in [SecBERT: A Pretrained Language Model for Cyber Security Text](https://github.com/jackaduma/SecBERT/), which is a SecRoBERTa model trained on cyber security text.
|
| 4 |
+
|
| 5 |
+
The training corpus was papers taken from [APTnotes](https://github.com/kbandla/APTnotes), [Stucco-Data: Cyber security data sources](https://stucco.github.io/data/),
|
| 6 |
+
[CASIE: Extracting Cybersecurity Event Information from Text](https://ebiquity.umbc.edu/_file_directory_/papers/943.pdf), [SemEval-2018 Task 8: Semantic Extraction from CybersecUrity REports using Natural Language Processing (SecureNLP)](https://competitions.codalab.org/competitions/17262).
|
| 7 |
+
|
| 8 |
+
SecRoBERTa has its own wordpiece vocabulary (secvocab) that's built to best match the training corpus. We trained [BERT](https://huggingface.co/jackaduma/SecBERT) and [SecRoBERTa](https://huggingface.co/jackaduma/SecRoBERTa) versions.
|
| 9 |
+
|
| 10 |
+
Available models include:
|
| 11 |
+
* `SecBERT`
|
| 12 |
+
* `SecRoBERTa`
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
The original repo can be found [here](https://github.com/jackaduma/SecBERT).
|