Update README.md

README.md

@@ -1,199 +1,229 @@
 ---
 library_name: transformers
-tags: []
+base_model:
+- Qwen/Qwen2.5-7B-Instruct
 ---
 
-# Model Card for Model ID
+# Model Card for EdgeRunner-Comand-Nested-FC-v3
 
-<!-- Provide a quick summary of what the model is/does. -->
+EdgeRunner-Comand-Nested is an advanced large language model designed specifically for handling complex nested function calls within personal domains. Initialized from Qwen2.5-7B-Instruct, further enhanced by the integration of the Hermes function call template and additional training on a specialized dataset (based on TinyAgent). This extra dataset focuses on personal domain applications, providing the model with a robust understanding of nested function scenarios that are typical in complex user interactions.
 
+### Data Processing Process
 
+#### 1. **Data Augmentation:**
+   - **TinyAgent Augmentation:**
+     TinyAgent consists of 16 core functions, but each sample only requires a subset of these functions to work. To ensure diversity and avoid overfitting, we used an augmentation technique where each sample is provided with a random superset of functions. This superset covers all the functions that the specific sample requires but is always a subset of the 16 total functions available in TinyAgent. By doing so, we allow the model to generalize across various combinations of functions while still providing the necessary context for each sample to execute correctly.
 
-## Model Details
+#### 2. **Additional Data Samples:**
+   - **GPT-4o Augmented Samples:**
+     To further enhance the dataset, we used GPT-4 for function synthesis in the personal assistant domain. This was done by using a pipeline that automatically generates function definitions, corresponding queries, and execution plans. After generating the function data, each sample was verified for validity to ensure the correctness of the functional outputs. Through this pipeline, we produced approximately 2,000 additional samples. These samples follow the TinyAgent format but introduce a broader range of new and more complex functions. This expanded set of functions enriches the model's understanding of diverse scenarios, allowing it to handle more varied function calls beyond the synthesis-based functions of TinyAgent, thereby improving its ability to generalize across tasks. 
 
-### Model Description
+#### 3. **Data Conversion:**
+   - **Hermes Format Conversion:**
+     All function call data, including those from TinyAgent, the additional samples generated, and the function call data we had previously, were converted to the Hermes format. In TinyAgent, the Hermes format uses the `$ + number` format to indicate nested use cases. This ensures consistency in data representation while also clearly defining nested function use cases for enhanced interpretability. Standardizing all data in Hermes format allows the model to efficiently process and understand function calls across a variety of scenarios.
 
-<!-- Provide a longer summary of what this model is. -->
+### Example Code
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+model_id = "edgerunner-ai/EdgeRunner-Command-Nested"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
 
-### Model Sources [optional]
+# You can find the full tools list here https://jsonblob.com/1290566717196328960
+tools = [{
+    "name": "open_and_get_file_path",
+    "description": "Opens the file and returns its path.",
+    "parameters": {
+      "type": "object",
+      "properties": {
+        "file_name": {
+          "type": "str",
+          "description": "The name of the file to open"
+        }
+      },
+      "required": [
+        "file_name"
+      ]
+    }
+  }, ...]
 
-<!-- Provide the basic links for the model. -->
+conversation = [{
+        "role": "user",
+        "content": "Send an email to Andrea attached with the file 'report.pdf' and ask when he is available for a meeting."
+    }]
+
+# Render the tool use prompt as a string:
+tool_use_prompt = tokenizer.apply_chat_template(
+            conversation,
+            tools=tools,
+            tokenize=False,
+            add_generation_prompt=True,
+)
+
+inputs = tokenizer(tool_use_prompt, return_tensors="pt")
+
+model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")
+
+outputs = model.generate(**inputs, max_new_tokens=1000)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+# Examples
+
+## Simple function
+
+### Single turn
+
+```markdown
+**User:**  
+How is the weather today in New York?
+
+**Assistant:**  
+I don't have direct access to real-time data like current weather conditions. However, I can help you find out by performing a web search. Let me do that now.
+
+<tool_call>  
+{"name": "web_search", "arguments": {"query": "weather today New York"}}  
+</tool_call>
+```
+
+### Multi turn
+
+```markdown
+**User:**  
+I want to search for some information about music, can you help me with that?
+
+**Assistant:**  
+Of course! What specific information about music are you looking for?
+
+**User:**  
+I'm looking for information about the history of rock music.
+
+**Assistant:**  
+<tool_call>  
+{"name": "web_search", "arguments": {"query": "history of rock music"}}  
+</tool_call>
+```
+
+```markdown
+**User:**  
+I want to search for some information about music, can you help me with that?
+
+**Assistant:**  
+Of course! What specific information about music are you looking for?
+
+**User:**  
+I'm looking for information about the history of rock music and also information about Justin Bieber.
+
+**Assistant:**  
+<tool_call>  
+{"name": "web_search", "arguments": {"query": "history of rock music"}}  
+</tool_call>
+<tool_call>  
+{"name": "web_search", "arguments": {"query": "information about Justin Bieber"}}  
+</tool_call>
+```
+
+## Nested functions
+
+### Success
+
+```markdown
+**User:**  
+Create a meeting with Mr.A and Mr.B to discuss Edgerunner on 27/9/2024 at 5:00 PM.
+
+**Assistant:**  
+<tool_call>  
+{"name": "get_email_address", "arguments": {"name": "Mr.A"}}  
+</tool_call>
+<tool_call>  
+{"name": "get_email_address", "arguments": {"name": "Mr.B"}}  
+</tool_call>
+<tool_call>  
+{"name": "create_calendar_event", "arguments": {"title": "Edgerunner Discussion", "start_date": "2024-09-27 17:00:00", "end_date": "2024-09-27 18:00:00", "location": "", "invitees": ["$1", "$2"], "notes": "", "calendar": null}}  
+</tool_call>
+```
+
+```markdown
+**User:**  
+Generate an image with the prompt 'A beautiful landscape with mountains and a lake' and send it to the Discord server with the message 'Here is a beautiful landscape'.
+
+**Assistant:**  
+<tool_call>  
+{"name": "generate_image", "arguments": {"prompt": "A beautiful landscape with mountains and a lake"}}  
+</tool_call>
+<tool_call>  
+{"name": "send_to_discord", "arguments": {"message": "Here is a beautiful landscape.", "attachments": ["$1"]}}  
+</tool_call>
+```
+
+
+```markdown
+**User:**  
+Send the directions to the Eiffel Tower from the Louvre Museum to Mr. C and Mr. D with an SMS message.
+
+**Assistant:**  
+<tool_call>  
+{"name": "get_phone_number", "arguments": {"name": "Mr.C"}}  
+</tool_call>
+<tool_call>  
+{"name": "get_phone_number", "arguments": {"name": "Mr.D"}}  
+</tool_call>
+<tool_call>  
+{"name": "maps_show_directions", "arguments": {"start_location": "Louvre Museum", "end_location": "Eiffel Tower", "transport": "d"}}  
+</tool_call>
+<tool_call>  
+{"name": "send_sms", "arguments": {"recipients": ["$1", "$2"], "message": "Here are the directions to get from Louvre Museum to Eiffel Tower: $3"}}  
+</tool_call>
+```
+
+```markdown
+**User:**  
+Send an email to Mr. A attached with the file 'report.pdf' and ask when he is available for a meeting.
+
+**Assistant:**  
+<tool_call>  
+{"name": "get_email_address", "arguments": {"name": "Mr.A"}}  
+</tool_call>
+<tool_call>  
+{"name": "open_and_get_file_path", "arguments": {"file_name": "report.pdf"}}  
+</tool_call>
+<tool_call>  
+{"name": "compose_new_email", "arguments": {"recipients": ["$1"], "cc": [], "subject": "Availability for Meeting", "context": "Please find the attached report. When are you available for a meeting?", "attachments": ["$2"]}}  
+</tool_call>
+```
+
+### Failed 
+
+```markdown
+**User:**  
+I want to generate an image and send it to Discord. Can you help me with that?
+
+**Assistant:**  
+Of course! Could you please provide me with a description or prompt for the image you would like to generate?
+
+**User:**  
+I want to generate an image of a cat with a gray background.
+
+**Assistant:**  
+<tool_call>  
+{"name": "generate_image", "arguments": {"prompt": "a cat with a gray background"}}  
+</tool_call>
+<tool_call>  
+{"name": "send_to_discord", "arguments": {"message": "", "attachments": ["$1"]}}  
+</tool_call>
+```
+
+# Benchmarks
+
+## Berkeley Function Calling Benchmark Results
+
+| Test Name                   | EdgeRunner-Command-Nested| Qwen/Qwen2.5-7B-Instruct |
+|-----------------------------|------------------------------------------------|--------------------------|
+| multiple_function           | 0.95                                           | 0.81                     |
+| parallel_multiple_function  | 0.84                                           | 0.655                    |
+| parallel_function           | 0.86                                           | 0.69                     |
+| simple                      | 0.96                                           | 0.9275                   |
+| irrelevance                 | 0.74                                           | 0.625                    |
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]