ChenWeiLi commited on
Commit
f2de2ca
1 Parent(s): 52b2a9e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -0
README.md CHANGED
@@ -24,6 +24,30 @@ The following models were included in the merge:
24
  * [sethuiyer/Medichat-Llama3-8B](https://huggingface.co/sethuiyer/Medichat-Llama3-8B)
25
  * [johnsnowlabs/JSL-MedLlama-3-8B-v2.0](https://huggingface.co/johnsnowlabs/JSL-MedLlama-3-8B-v2.0)
26
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
  ### Configuration
28
 
29
  The following YAML configuration was used to produce this model:
 
24
  * [sethuiyer/Medichat-Llama3-8B](https://huggingface.co/sethuiyer/Medichat-Llama3-8B)
25
  * [johnsnowlabs/JSL-MedLlama-3-8B-v2.0](https://huggingface.co/johnsnowlabs/JSL-MedLlama-3-8B-v2.0)
26
 
27
+ ### Evaluation
28
+
29
+ - multimedq (0 shot)
30
+ | Tasks |Version|Filter|n-shot| Metric |Value | |Stderr|
31
+ |-------------------------------|-------|------|-----:|--------|-----:|---|-----:|
32
+ | - medmcqa |Yaml |none | 0|acc |0.5728|± |0.0076|
33
+ | | |none | 0|acc_norm|0.5728|± |0.0076|
34
+ | - medqa_4options |Yaml |none | 0|acc |0.5923|± |0.0138|
35
+ | | |none | 0|acc_norm|0.5923|± |0.0138|
36
+ | - anatomy (mmlu) | 0|none | 0|acc |0.7111|± |0.0392|
37
+ | - clinical_knowledge (mmlu) | 0|none | 0|acc |0.7547|± |0.0265|
38
+ | - college_biology (mmlu) | 0|none | 0|acc |0.7917|± |0.0340|
39
+ | - college_medicine (mmlu) | 0|none | 0|acc |0.6647|± |0.0360|
40
+ | - medical_genetics (mmlu) | 0|none | 0|acc |0.8200|± |0.0386|
41
+ | - professional_medicine (mmlu)| 0|none | 0|acc |0.7426|± |0.0266|
42
+ |stem |N/A |none | 0|acc_norm|0.5773|± |0.0067|
43
+ | | |none | 0|acc |0.6145|± |0.0057|
44
+ | - pubmedqa | 1|none | 0|acc |0.7400|± |0.0196|
45
+
46
+ |Groups|Version|Filter|n-shot| Metric |Value | |Stderr|
47
+ |------|-------|------|-----:|--------|-----:|---|-----:|
48
+ |stem |N/A |none | 0|acc_norm|0.5773|± |0.0067|
49
+ | | |none | 0|acc |0.6145|± |0.0057|
50
+
51
  ### Configuration
52
 
53
  The following YAML configuration was used to produce this model: