Update README.md
Browse files
README.md
CHANGED
@@ -2,7 +2,7 @@
|
|
2 |
license: cc-by-4.0
|
3 |
datasets:
|
4 |
- Salesforce/xlam-function-calling-60k
|
5 |
-
- MadeAgents/
|
6 |
base_model:
|
7 |
- Qwen/Qwen2.5-Coder-1.5B-Instruct
|
8 |
---
|
@@ -13,24 +13,26 @@ base_model:
|
|
13 |
We're excited to release lightweight Hammer 2.0 models ([0.5B](https://huggingface.co/MadeAgents/Hammer2.0-0.5b) , [1.5B](https://huggingface.co/MadeAgents/Hammer2.0-1.5b) , [3B](https://huggingface.co/MadeAgents/Hammer2.0-3b) , and [7B](https://huggingface.co/MadeAgents/Hammer2.0-7b)) with strong function calling capability, which empower developers to build personalized, on-device agentic applications.
|
14 |
|
15 |
## Model Details
|
16 |
-
Hammer2.0 finetuned based on [Qwen 2.5 series](https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e) and [Qwen 2.5 coder series](https://huggingface.co/collections/Qwen/qwen25-coder-66eaa22e6f99801bf65b0c2f) using function masking techniques. It's trained using the [APIGen Function Calling Datasets](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k) containing 60,000 samples, supplemented by [
|
17 |
|
18 |
## Evaluation
|
19 |
-
The evaluation results of Hammer 2.0
|
20 |
<div style="text-align: center;">
|
21 |
<img src="v2_figures/bfcl.PNG" alt="overview" width="1000" style="margin: auto;">
|
22 |
</div>
|
23 |
|
|
|
|
|
|
|
24 |
|
25 |
-
In addition, we evaluated Hammer2.0 on other academic benchmarks to further show our model's generalization ability:
|
26 |
<div style="text-align: center;">
|
27 |
<img src="v2_figures/others.PNG" alt="overview" width="1000" style="margin: auto;">
|
28 |
</div>
|
29 |
|
30 |
-
|
31 |
|
32 |
## Requiements
|
33 |
-
The code of
|
34 |
|
35 |
## How to Use
|
36 |
This is a simple example of how to use our model.
|
|
|
2 |
license: cc-by-4.0
|
3 |
datasets:
|
4 |
- Salesforce/xlam-function-calling-60k
|
5 |
+
- MadeAgents/xlam-irrelevance-7.5k
|
6 |
base_model:
|
7 |
- Qwen/Qwen2.5-Coder-1.5B-Instruct
|
8 |
---
|
|
|
13 |
We're excited to release lightweight Hammer 2.0 models ([0.5B](https://huggingface.co/MadeAgents/Hammer2.0-0.5b) , [1.5B](https://huggingface.co/MadeAgents/Hammer2.0-1.5b) , [3B](https://huggingface.co/MadeAgents/Hammer2.0-3b) , and [7B](https://huggingface.co/MadeAgents/Hammer2.0-7b)) with strong function calling capability, which empower developers to build personalized, on-device agentic applications.
|
14 |
|
15 |
## Model Details
|
16 |
+
Hammer2.0 finetuned based on [Qwen 2.5 series](https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e) and [Qwen 2.5 coder series](https://huggingface.co/collections/Qwen/qwen25-coder-66eaa22e6f99801bf65b0c2f) using function masking techniques. It's trained using the [APIGen Function Calling Datasets](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k) containing 60,000 samples, supplemented by [xlam-irrelevance-7.5k](https://huggingface.co/datasets/MadeAgents/xlam-irrelevance-7.5k) we generated. Hammer2.0 has achieved exceptional performances across numerous function calling benchmarks. For more details, please refer to [Hammer: Robust Function-Calling for On-Device Language Models via Function Masking](https://arxiv.org/abs/2410.04587) and [Hammer GitHub repository](https://github.com/MadeAgents/Hammer) .
|
17 |
|
18 |
## Evaluation
|
19 |
+
The evaluation results of Hammer 2.0 models on the Berkeley Function-Calling Leaderboard (BFCL-v3) are presented in the following table:
|
20 |
<div style="text-align: center;">
|
21 |
<img src="v2_figures/bfcl.PNG" alt="overview" width="1000" style="margin: auto;">
|
22 |
</div>
|
23 |
|
24 |
+
Our Hammer 2.0 series consistently achieves corresponding best performance at comparable scales. The 7B model outperforms most function calling enchanced models, and the 1.5B model also achieves unexpected performance.
|
25 |
+
|
26 |
+
In addition, we evaluated the Hammer 2.0 models on other academic benchmarks to further demonstrate the generalization ability of our models.
|
27 |
|
|
|
28 |
<div style="text-align: center;">
|
29 |
<img src="v2_figures/others.PNG" alt="overview" width="1000" style="margin: auto;">
|
30 |
</div>
|
31 |
|
32 |
+
Hammer 2.0 models showcase highly stable performance, suggesting the robustness of Hammer 2.0 series. In contrast, the baseline approaches display varying levels of effectiveness.
|
33 |
|
34 |
## Requiements
|
35 |
+
The code of Hammer 2.0 models have been in the latest Hugging face transformers and we advise you to install `transformers>=4.37.0`.
|
36 |
|
37 |
## How to Use
|
38 |
This is a simple example of how to use our model.
|