README.md

metadata

language: en
license: apache-2.0
tags:
  - financial-sentiment-analysis
  - sentiment-analysis
  - language-perceiver
datasets:
  - financial_phrasebank
widget:
  - text: INDEX100 fell sharply today.
  - text: ImaginaryJetCo bookings hit by Omicron variant as losses total £1bn.
  - text: Q1 ImaginaryGame's earnings beat expectations.
  - text: Should we buy IMAGINARYSTOCK today?
metrics:
  - recall
  - f1
  - accuracy
  - precision
model-index:
  - name: fin-perceiver
    results:
      - task:
          name: Text Classification
          type: text-classification
        dataset:
          name: financial_phrasebank
          type: financial_phrasebank
          args: sentences_50agree
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.8624
          - name: F1
            type: f1
            value: 0.8416
            args: macro
          - name: Precision
            type: precision
            value: 0.8438
            args: macro
          - name: Recall
            type: recall
            value: 0.8415
            args: macro

FINPerceiver

FINPerceiver is a fine-tuned Perceiver IO language model for financial sentiment analysis. More details on the training process of this model are available on the GitHub repository.

Weights & Biases was used to track experiments.

We achieved the following results with 10-fold cross validation.

eval/accuracy  0.8624 (stdev 0.01922)
eval/f1        0.8416 (stdev 0.03738)
eval/loss      0.4314 (stdev 0.05295)
eval/precision 0.8438 (stdev 0.02938)
eval/recall    0.8415 (stdev 0.04458)

The hyperparameters used are as follows.

per_device_train_batch_size  16
per_device_eval_batch_size   16
num_train_epochs             4
learning_rate                2e-5

Datasets

This model was trained on the Financial PhraseBank (>= 50% agreement)