ahmeddbahaa

/

xlmroberta2xlmroberta-finetune-summarization-ar

like 0

could you please provide the source code? I tried and trained the model but the output was much worst.
I have used :
tokenizer = XLMRobertaTokenizerFast.from_pretrained("xlm-roberta-base")
model = EncoderDecoderModel.from_encoder_decoder_pretrained("xlm-roberta-base", "xlm-roberta-base", tie_encoder_decoder=True)
hyperparameters: ( do_train=True, do_eval=True, eval_steps=20, num_train_epochs=5, dataloader_num_workers=2, optim="adamw_torch",
learning_rate=5e-5, warmup_steps=500, fp16=True)

Have you solved the issue? I am trying for a seq-2-seq model and also having some issues.