---
library_name: transformers
language:
- en
base_model:
- microsoft/deberta-v3-base
---
### Model Performance
On a macro-level, this model performs quite poorly. Do use the other model which takes only text data as input. This model takes a lot of other features.

- Better Text-only model: https://huggingface.co/wanadzhar913/debertav3-finetuned-banking-transaction-classification-text-only
- Training notebook: https://github.com/wanadzhar913/bank-transaction-classification-debertav3/blob/master/notebooks/02_model_building_deberta_with_non_text_features.ipynb

| Model                        | Epoch | Learning Rate | Grad Norm   | Training Loss | Validation Loss | Accuracy  | F1 Score (Weighted) | F1 Score (Macro) | Precision (Weighted) | Precision (Macro) | Recall (Weighted) | Recall (Macro) |
|------------------------------|-------|---------------|-------------|---------------|-----------------|-----------|---------------------|------------------|----------------------|-------------------|-------------------|----------------|
| DeBERTaV3 (Text & Non-text)  | 15    | 0.0000100     | 264.891     | 0.441         | 0.494           | 0.883     | 0.908               | 0.604            | 0.948                | 0.593             | 0.883             | 0.758          |

### How To Use

```bash
mkdir data/
cd data/ && wget https://raw.githubusercontent.com/wanadzhar913/bank-transaction-classification/refs/heads/master/data/test.csv -q
```

```python
import pandas as pd
from datasets import load_dataset, Dataset

import torch
from transformers import AutoTokenizer

from classifier import CustomSequenceClassification

### Load sample dataset ###
test = pd.read_csv('data/test.csv')
df_test = Dataset.from_pandas(test)

cols_to_remove = ['client_id', 'bank_id', 'account_id', 'txn_id', 'txn_date',  'description_stem', 'description_clean_len', 'category']
cols_to_preprocess = sorted(list(set(df_test.column_names) - set(cols_to_remove)))

def transform_row(row):
    """This is a function to extract the text data and convert our non-text features into a list."""

    # Separate description
    description = row["description"]

    # Collect remaining features and convert to a FloatTensor
    extra_data = [row[key] for key in cols_to_preprocess if key != "description"]

    return {
        "description": description,
        "extra_data": extra_data, # for all the extra columns
    }

test_X = df_test.remove_columns(cols_to_remove) \
                .map(transform_row, num_proc=2) \
                .select_columns(['description', 'extra_data'])

### Load Model ###
model = CustomSequenceClassification.from_pretrained(
    "wanadzhar913/debertav3-finetuned-banking-transaction-classification",
    num_labels=33,
    num_extra_dims=44,
)

_ = model.eval().cuda()

tokenizer = AutoTokenizer.from_pretrained('wanadzhar913/debertav3-finetuned-banking-transaction-classification')

### Inference ###
padded = tokenizer(test_X['description'][0], padding = 'longest', return_tensors = 'pt')
for k in padded.keys():
    padded[k] = padded[k].cuda()

with torch.no_grad():
    pred = model(
        **padded,
        extra_data=torch.FloatTensor([test_X['extra_data'][0]]).cuda(),
        return_dict = False
    )
    print(pred[0].argmax(axis = 1).detach().cpu().numpy()[0])
>>> 19 # Restaurants (refer to `config.json`)
```