feat: Updated README

README.md

@@ -5,69 +5,122 @@ license_link: https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct/blob/main/LI
 language:
 - en
 base_model:
-- Qwen/Qwen2.5-Coder-3B
+- Qwen/Qwen2.5-Coder-3B-Instruct
 pipeline_tag: text-generation
 library_name: transformers
 tags:
 - code
-- codeqwen
 - chat
 - qwen
 - qwen-coder
+- agent
 ---
 
-
-# Qwen2.5-Coder-3B-Instruct
+# Dria-Agent-α-3B
 
 ## Introduction
 
-Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:
+***Dria-Agent-α*** are series of large language models trained on top of the [Qwen2.5-Coder](https://huggingface.co/collections/Qwen/qwen25-coder-66eaa22e6f99801bf65b0c2f) series, specifically on top of the [Qwen/Qwen2.5-Coder-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct) and [Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) models to be used in agentic applications. These models are the first instalment of agent-focused LLMs (hence the **α** in the naming) we hope to improve with better and more elaborate techniques in subsequent releases.
 
-- Significantly improvements in **code generation**, **code reasoning** and **code fixing**. Base on the strong Qwen2.5, we scale up the training tokens into 5.5 trillion including source code, text-code grounding, Synthetic data, etc. Qwen2.5-Coder-32B has become the current state-of-the-art open-source codeLLM, with its coding abilities matching those of GPT-4o.
-- A more comprehensive foundation for real-world applications such as **Code Agents**. Not only enhancing coding capabilities but also maintaining its strengths in mathematics and general competencies.
+Dria-Agent-α employs ***Pythonic function calling***, which is LLMs using blocks of Python code to interact with provided tools and output actions. This method was inspired by many previous work, including but not limited to [DynaSaur](https://arxiv.org/pdf/2411.01747), [RLEF](https://arxiv.org/pdf/2410.02089), [ADAS](https://arxiv.org/pdf/2408.08435) and [CAMEL](https://arxiv.org/pdf/2303.17760). This way of function calling has a few advantages over traditional JSON-based function calling methods:
 
-**This repo contains the instruction-tuned 3B Qwen2.5-Coder model**, which has the following features:
-- Type: Causal Language Models
-- Training Stage: Pretraining & Post-training
-- Architecture: transformers with RoPE, SwiGLU, RMSNorm, Attention QKV bias and tied word embeddings
-- Number of Parameters: 3.09B
-- Number of Paramaters (Non-Embedding): 2.77B
-- Number of Layers: 36
-- Number of Attention Heads (GQA): 16 for Q and 2 for KV
-- Context Length: Full 32,768 tokens
-  
-For more details, please refer to our [blog](https://qwenlm.github.io/blog/qwen2.5-coder-family/), [GitHub](https://github.com/QwenLM/Qwen2.5-Coder), [Documentation](https://qwen.readthedocs.io/en/latest/), [Arxiv](https://arxiv.org/abs/2409.12186).
+1. **One-shot Parallel Multiple Function Calls:** The model can can utilise many synchronous processes in one chat turn to arrive to a solution, which would require other function calling models multiple turns of conversation.
+2. **Free-form Reasoning and Actions:** The model provides reasoning traces freely in natural language and the actions in between \`\`\`python \`\`\` blocks, as it already tends to do without special prompting or tuning. This tries to mitigate the possible performance loss caused by imposing specific formats on LLM outputs discussed in [Let Me Speak Freely?](https://arxiv.org/pdf/2408.02442)
+3. **On-the-fly Complex Solution Generation:** The solution provided by the model is essentially a Python program with the exclusion of some "risky" builtins like `exec`, `eval` and `compile` (see full list in **Quickstart** below). This enables the model to implement custom complex logic with conditionals and synchronous pipelines (using the output of one function in the next function's arguments) which would not be possible with the current JSON-based function calling methods (as far as we know).
 
-## Requirements
+## Quickstart
 
-The code of Qwen2.5-Coder has been in the latest Hugging face `transformers` and we advise you to use the latest version of `transformers`.
+````python
+import json
+from typing import Any, Dict, List
+from transformers import AutoModelForCausalLM, AutoTokenizer
 
-With `transformers<4.37.0`, you will encounter the following error:
-```
-KeyError: 'qwen2'
-```
+model_name = "driaforall/function-calling-3B-30k-2e"
+model = AutoModelForCausalLM.from_pretrained(
+    model_name, device_map="auto", torch_dtype="auto", trust_remote_code=True
+)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
 
-## Quickstart
+# Please use our provided prompt for best performance
+SYSTEM_PROMPT = """
+You are an expert AI assistant that specializes in providing Python code to solve the task/problem at hand provided by the user.
+
+You can use Python code freely, including the following available functions:
 
-Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.
+<|functions_schema|>
+{{functions_schema}}
+<|end_functions_schema|>
+
+The following dangerous builtins are restricted for security:
+- exec
+- eval
+- execfile
+- compile
+- importlib
+- __import__
+- input
+- exit
+
+Think step by step and provide your reasoning, outside of the function calls.
+You can write Python code and use the available functions. Provide all your python code in a SINGLE markdown code block like the following:
 
 ```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
+result = example_function(arg1, "string")
+result2 = example_function2(result, arg2)
+```
 
-model_name = "Qwen/Qwen2.5-Coder-3B-Instruct"
+DO NOT use print() statements AT ALL. Avoid mutating variables whenever possible.
+""".strip()
 
-model = AutoModelForCausalLM.from_pretrained(
-    model_name,
-    torch_dtype="auto",
-    device_map="auto"
-)
-tokenizer = AutoTokenizer.from_pretrained(model_name)
 
-prompt = "write a quick sort algorithm."
+get_sample_data = """
+def get_current_date():
+    \"\"\" Get the current date.
+
+    Returns:
+    - Current date in YYYY-MM-DD format
+    \"\"\"
+    pass
+
+def check_availability(day: str, start_time: str, end_time: str):
+    \"\"\" Check if a time slot is available on a given day.
+
+    Args:
+    - day: The day to check in YYYY-MM-DD format
+    - start_time: Start time in HH:MM format
+    - end_time: End time in HH:MM format
+
+    Returns:
+    - True if slot is available, False otherwise
+    \"\"\"
+    pass
+
+def make_appointment(day: str, start_time: str, end_time: str):
+    \"\"\" Make an appointment for a given time slot.
+
+    Args:
+    - day: The day to make appointment in YYYY-MM-DD format
+    - start_time: Start time in HH:MM format
+    - end_time: End time in HH:MM format
+    - title: The title of the appointment
+
+    Returns:
+    - True if appointment was made successfully, False otherwise
+    \"\"\"
+    pass
+"""
+
+# Helper function to create the system prompt for our model
+def format_prompt(tools: str):
+    return SYSTEM_PROMPT.format(functions_schema=tools)
+
+system_prompt = SYSTEM_PROMPT.replace("{{functions_schema}}", get_sample_data)
+
 messages = [
-    {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
-    {"role": "user", "content": prompt}
+    {"role": "system", "content": system_prompt},
+    {"role": "user", "content": "Can you check if I have tomorrow 10:00-12:00 available and make an appointment for a meeting with my thesis supervisor if so?"},
 ]
+
 text = tokenizer.apply_chat_template(
     messages,
     tokenize=False,
@@ -84,30 +137,58 @@ generated_ids = [
 ]
 
 response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+print(response)
+````
+
+The output should be something like:
+
+````
+To accomplish this task, we need to follow these steps:
+
+1. Determine the current date and add one day to it to find tomorrow's date.
+2. Check the availability of the time slot from 10:00 to 12:00 on that day.
+3. If the slot is available, make an appointment for the meeting.
+
+Here's the Python code to achieve this:
+
+```python
+from datetime import datetime, timedelta
+
+# Step 1: Determine the current date and add one day to get tomorrow's date
+current_date = get_current_date()
+tomorrow_date = (datetime.strptime(current_date, '%Y-%m-%d') + timedelta(days=1)).strftime('%Y-%m-%d')
+
+# Step 2: Check the availability of the time slot from 10:00 to 12:00 on that day
+slot_start_time = '10:00'
+slot_end_time = '12:00'
+is_slot_available = check_availability(tomorrow_date, slot_start_time, slot_end_time)
+
+# Step 3: If the slot is available, make an appointment for the meeting
+appointment_title = "Meeting with thesis supervisor"
+appointment_made = False
+
+if is_slot_available:
+    appointment_made = make_appointment(tomorrow_date, slot_start_time, slot_end_time, appointment_title)
 ```
 
+This code first calculates tomorrow's date, then checks if the time slot from 10:00 to 12:00 is available, and finally makes an appointment if the slot is available.
+````
 
 ## Evaluation & Performance
 
-Detailed evaluation results are reported in this [📑 blog](https://qwenlm.github.io/blog/qwen2.5-coder-family/).
+We evaluate the model on the following benchmarks:
 
-For requirements on GPU memory and the respective throughput, see results [here](https://qwen.readthedocs.io/en/latest/benchmark/speed_benchmark.html).
+1. Benchmark 1
+2. Benchmark 2
+3. ...
+4. **Dria-Pythonic-Agent-Benchmark (DPAB):** The benchmark we curated with a synthetic data generation +model-based validation + filtering and manual selection to evaluate LLMs on their Pythonic function calling ability, spanning multiple scenarios and tasks. More detailed information about the benchmark can be found on the [Github repo](https://github.com/firstbatchxyz/function-calling-eval) and in our [blog post](blog-link)
 
-## Citation
+Below are the evaluation results for Qwen2.5-Coder-3B-Instruct and Dria-Agent-α-3B
 
-If you find our work helpful, feel free to give us a cite.
+| Benchmark Name | Qwen2.5-Coder-3B-Instruct | Dria-Agent-α-3B |
+|----------------|---------------------------|-----------------|
+| BFCL | TBD | TBD |
+| MMLU-Pro | 35.2 ([Self Reported](https://arxiv.org/pdf/2409.12186)) | 29.8* |
+| DPAB | TBD | TBD |
 
-```
-@article{hui2024qwen2,
-      title={Qwen2. 5-Coder Technical Report},
-      author={Hui, Binyuan and Yang, Jian and Cui, Zeyu and Yang, Jiaxi and Liu, Dayiheng and Zhang, Lei and Liu, Tianyu and Zhang, Jiajun and Yu, Bowen and Dang, Kai and others},
-      journal={arXiv preprint arXiv:2409.12186},
-      year={2024}
-}
-@article{qwen2,
-      title={Qwen2 Technical Report}, 
-      author={An Yang and Baosong Yang and Binyuan Hui and Bo Zheng and Bowen Yu and Chang Zhou and Chengpeng Li and Chengyuan Li and Dayiheng Liu and Fei Huang and Guanting Dong and Haoran Wei and Huan Lin and Jialong Tang and Jialin Wang and Jian Yang and Jianhong Tu and Jianwei Zhang and Jianxin Ma and Jin Xu and Jingren Zhou and Jinze Bai and Jinzheng He and Junyang Lin and Kai Dang and Keming Lu and Keqin Chen and Kexin Yang and Mei Li and Mingfeng Xue and Na Ni and Pei Zhang and Peng Wang and Ru Peng and Rui Men and Ruize Gao and Runji Lin and Shijie Wang and Shuai Bai and Sinan Tan and Tianhang Zhu and Tianhao Li and Tianyu Liu and Wenbin Ge and Xiaodong Deng and Xiaohuan Zhou and Xingzhang Ren and Xinyu Zhang and Xipin Wei and Xuancheng Ren and Yang Fan and Yang Yao and Yichang Zhang and Yu Wan and Yunfei Chu and Yuqiong Liu and Zeyu Cui and Zhenru Zhang and Zhihao Fan},
-      journal={arXiv preprint arXiv:2407.10671},
-      year={2024}
-}
-```
+**\*Note:** The model tends to use Pythonic function calling for a lot of the test cases in STEM-related fields (math, physics, chemistry, etc.) in the MMLU-Pro benchmark, which isn't captured by the evaluation framework and scripts provided in their [Github repository](https://github.com/TIGER-AI-Lab/MMLU-Pro/tree/main). We haven't modified the script for evaluation, and leave it for the future iterations of this model. However, by performing qualitative analysis on the model responses, we suspect that the model's score will increase instead of suffering a ~6% decrease.