pascalhuerten commited on
Commit
b740a5b
1 Parent(s): c2d27e4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -9
README.md CHANGED
@@ -29,15 +29,6 @@ The **isy-thl/multilingual-e5-base-course-skill-tuned** is a finetuned version o
29
  - **Scalability:**
30
  - The model can handle input sequences up to 512 tokens in length, making it suitable for processing comprehensive course descriptions.
31
 
32
- ## Limitations and Considerations
33
-
34
- - **Language Limitation:**
35
- - The finetuning was specifically targeted at German language content. While the base model supports multiple languages, this particular finetuned version may not perform as well on non-German texts without additional training.
36
- - **Data Bias:**
37
- - The performance and reliability of the model are dependent on the quality of the annotated data in the training dataset. Any biases present in the training data may affect the model's output.
38
- - **Retrieval Scope:**
39
- - The model is optimized for educational contexts and may not generalize as effectively to other domains without further finetuning.
40
-
41
  ## Performance
42
 
43
  To evaluate the model, all ESCO (x=13895) and GRETA (x=23) skills were embedded using the model under assessment and stored in a vector database. For each query in the evaluation dataset, the top 30 most relevant candidates were retrieved based on cosine similarity. Metrics such as accuracy, precision, recall, NDCG, MRR, and MAP were then calculated. For reranker evaluation, the reranker was used to re-rank the top 30 candidates chosen by the fine-tuned bi-encoder model. The evaluation results were split for the ESCO and GRETA use cases:
 
29
  - **Scalability:**
30
  - The model can handle input sequences up to 512 tokens in length, making it suitable for processing comprehensive course descriptions.
31
 
 
 
 
 
 
 
 
 
 
32
  ## Performance
33
 
34
  To evaluate the model, all ESCO (x=13895) and GRETA (x=23) skills were embedded using the model under assessment and stored in a vector database. For each query in the evaluation dataset, the top 30 most relevant candidates were retrieved based on cosine similarity. Metrics such as accuracy, precision, recall, NDCG, MRR, and MAP were then calculated. For reranker evaluation, the reranker was used to re-rank the top 30 candidates chosen by the fine-tuned bi-encoder model. The evaluation results were split for the ESCO and GRETA use cases: