ConvBERT base pre-trained on large_spanish_corpus
The ConvBERT architecture is presented in the "ConvBERT: Improving BERT with Span-based Dynamic Convolution" paper.
Metrics on evaluation set
disc_accuracy = 0.9488542
disc_auc = 0.8833056
disc_loss = 0.15933733
disc_precision = 0.79224133
disc_recall = 0.27443287
global_step = 1000000
loss = 9.658503
masked_lm_accuracy = 0.6177698
masked_lm_loss = 1.7050561
sampled_masked_lm_accuracy = 0.5379228
Usage
from transformers import AutoModel, AutoTokenizer
model_name = "mrm8488/convbert-base-spanish"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)
Created by Manuel Romero/@mrm8488 with the support of Narrativa
Made with ♥ in Spain
- Downloads last month
- 461
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.