Commit
·
cee4ec0
1
Parent(s):
ee5afd4
Add evaluation and improve wording
Browse files- README.md +9 -3
- assets/sc2-instruct-evalplus.png +0 -0
- assets/sc2-instruct-lcb-ds.png +0 -0
README.md
CHANGED
@@ -104,7 +104,7 @@ We introduce StarCoder2-15B-Instruct-v0.1, the very first entirely self-aligned
|
|
104 |
|
105 |
### Intended use
|
106 |
|
107 |
-
The model is
|
108 |
|
109 |
Here is an example to get started with the model using the [transformers](https://huggingface.co/docs/transformers/index) library:
|
110 |
|
@@ -169,9 +169,15 @@ def sum_list_of_integers(numbers):
|
|
169 |
|
170 |
### Bias, Risks, and Limitations
|
171 |
|
172 |
-
StarCoder2-15B-Instruct-v0.1 is primarily finetuned for Python code generation tasks that can be verified through execution, which may lead to biases and limitations. For example, the model
|
173 |
|
174 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
175 |
|
176 |
## Training Details
|
177 |
|
|
|
104 |
|
105 |
### Intended use
|
106 |
|
107 |
+
The model is designed to respond to **coding-related instructions** in a single turn. Instructions in other styles may result in less accurate responses
|
108 |
|
109 |
Here is an example to get started with the model using the [transformers](https://huggingface.co/docs/transformers/index) library:
|
110 |
|
|
|
169 |
|
170 |
### Bias, Risks, and Limitations
|
171 |
|
172 |
+
StarCoder2-15B-Instruct-v0.1 is primarily finetuned for Python code generation tasks that can be verified through execution, which may lead to certain biases and limitations. For example, the model might not adhere strictly to instructions that dictate the output format. In these situations, it's beneficial to provide a **response prefix** or a **one-shot example** to steer the model’s output. Additionally, the model may have limitations with other programming languages and out-of-domain coding tasks.
|
173 |
|
174 |
+
The model also inherits the bias, risks, and limitations from its base StarCoder2-15B model. For more information, please refer to the [StarCoder2-15B model card](https://huggingface.co/bigcode/starcoder2-15b).
|
175 |
+
|
176 |
+
## Evaluation on EvalPlus, LiveCodeBench, and DS-1000
|
177 |
+
|
178 |
+

|
179 |
+
|
180 |
+

|
181 |
|
182 |
## Training Details
|
183 |
|
assets/sc2-instruct-evalplus.png
ADDED
![]() |
assets/sc2-instruct-lcb-ds.png
ADDED
![]() |