FLAN-T5 small-GeoNames

This model is a fine-tuned version of flan-t5-small on the GeoNames dataset.

Model description

The model is trained to classify terms into one of 660 category classes related to geographical locations.

The model also works well as part of a Retrieval-and-Generation (RAG) pipeline by leveraging an external knowledge source, specifically GeoNames Semantic Primes.

Intended uses and limitations

This model is intended to be used to generate a type (class) for an input term.

Training and evaluation data

The training and evaluation data can be found here.

The train size is 8078865.

The test size is 702510.

Example

Here's an example of the model capabilities:

input:
- Lexical Term L: Pic de Font Blanca
output:
- Type: peak
input:
- Lexical Term L: Roc Mele
output:
- Type: mountain
input:
- Lexical Term L: Estany de les Abelletes
output:
- Type: lake

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss
2.6223	1.0	1000	1.5223
2.1430	2.0	2000	1.3764
1.9100	3.0	3000	1.2825
1.7642	4.0	4000	1.2102
1.6607	5.0	5000	1.1488

@misc{akl2024dstillms4ol2024task,
      title={DSTI at LLMs4OL 2024 Task A: Intrinsic versus extrinsic knowledge for type classification}, 
      author={Hanna Abi Akl},
      year={2024},
      eprint={2408.14236},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2408.14236}, 
}