|
--- |
|
language: en |
|
thumbnail: https://github.com/jackaduma |
|
tags: |
|
- exbert |
|
- security |
|
- cyber security |
|
license: apache-2.0 |
|
datasets: |
|
- APTnotes |
|
- Stucco-Data |
|
- CASIE |
|
--- |
|
|
|
# SecRoBERTa |
|
|
|
This is the pretrained model presented in [SecBERT: A Pretrained Language Model for Cyber Security Text](https://github.com/jackaduma/SecBERT/), which is a SecRoBERTa model trained on cyber security text. |
|
|
|
The training corpus was papers taken from |
|
* [APTnotes](https://github.com/kbandla/APTnotes) |
|
* [Stucco-Data: Cyber security data sources](https://stucco.github.io/data/) |
|
* [CASIE: Extracting Cybersecurity Event Information from Text](https://ebiquity.umbc.edu/_file_directory_/papers/943.pdf) |
|
* [SemEval-2018 Task 8: Semantic Extraction from CybersecUrity REports using Natural Language Processing (SecureNLP)](https://competitions.codalab.org/competitions/17262). |
|
|
|
SecRoBERTa has its own wordpiece vocabulary (secvocab) that's built to best match the training corpus. |
|
|
|
We trained [SecBERT](https://huggingface.co/jackaduma/SecBERT) and [SecRoBERTa](https://huggingface.co/jackaduma/SecRoBERTa) versions. |
|
|
|
Available models include: |
|
* [`SecBERT`](https://huggingface.co/jackaduma/SecBERT) |
|
* [`SecRoBERTa`](https://huggingface.co/jackaduma/SecRoBERTa) |
|
|
|
|
|
The original repo can be found [here](https://github.com/jackaduma/SecBERT). |
