How to write a good issue

When you encounter something that doesn’t seem right with one of the Hugging Face libraries, you should definitely let us know so we can fix it (the same goes for any open source library, for that matter). If you are not completely certain whether the bug lies in your own code or one of our libraries, the first place to check is the forums. The community will help you figure this out, and the Hugging Face team also closely watches the discussions there.

When you are sure you have a bug in your hand, the first step is to build a minimal reproducible example.

Creating a minimal reproducible example

It’s very important to isolate the piece of code that produces the bug, as no one in the Hugging Face team is a magician (yet), and they can’t fix what they can’t see. A minimal reproducible example should, as the name indicates, be reproducible. This means that it should not rely on any external files or data you may have. Try to replace the data you are using with some dummy values that look like your real ones and still produce the same error.

Once you have something that is self-contained, you can try to reduce it into even less lines of code, building what we call a minimal reproducible example. While this requires a bit more work on your side, you will almost be guaranteed to get help and a fix if you provide a nice, short bug reproducer.

If you feel comfortable enough, go inspect the source code where your bug happens. You might find a solution to your problem (in which case you can even suggest a pull request to fix it), but more generally, this can help the maintainers better understand the source when they read your report.

Filling out the issue template

When you file your issue, you will notice there is a template to fill out. We will follow the one for 🤗 Transformers issues here, but the same kind of information will be required if you report an issue in another repository. Don’t leave the template blank: taking the time to fill it in will maximize your chances of getting an answer and solving your problem.

In general, when filing an issue, always stay courteous. This is an open source project, so you are using free software, and no one has any obligation to help you. You may include what you feel is justified criticism in your issue, but then the maintainers may very well take it badly and not be in a rush help you. Make sure you read the code of conduct of the project.

Including your environment information

🤗 Transformers provides a utility to get all the information we need about your environment. Just type the following in your terminal:

transformers-cli env

and you should get something like this:

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.12.0.dev0
- Platform: Linux-5.10.61-1-MANJARO-x86_64-with-arch-Manjaro-Linux
- Python version: 3.7.9
- PyTorch version (GPU?): 1.8.1+cu111 (True)
- Tensorflow version (GPU?): 2.5.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.3.4 (cpu)
- Jax version: 0.2.13
- JaxLib version: 0.1.65
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

You can also add a ! at the beginning of the transformers-cli env command to execute it from a notebook cell, and then copy and paste the result at the beginning of your issue.

Tagging people

Tagging people by typing an @ followed by their GitHub handle will send them a notification so they will see your issue and might reply quicker. Use this with moderation, because the people you tag might not appreciate being notified if it’s something they have no direct link to. If you have looked at the source files related to your bug, you should tag the last person that made changes at the line you think is responsible for your problem (you can find this information by looking at said line on GitHub, selecting it, then clicking “View git blame”).

Otherwise, the template offers suggestions of people to tag. In general, never tag more than three people!

Including a reproducible example

If you have managed to create a self-contained example that produces the bug, now is the time to include it! Type a line with three backticks followed by python, like this:

```python

then paste in your minimal reproducible example and type a new line with three backticks. This will ensure your code is properly formatted.

If you didn’t manage to create a reproducible example, explain in clear steps how you got to your issue. Include a link to a Google Colab notebook where you got the error if you can. The more information you share, the better able the maintainers will be to reply to you.

In all cases, you should copy and paste the whole error message you are getting. If you’re working in Colab, remember that some of the frames may be automatically collapsed in the stack trace, so make sure you expand them before copying. Like with the code sample, put that error message between two lines with three backticks, so it’s properly formatted.

Describing the expected behavior

Explain in a few lines what you expected to get, so that the maintainers get a full grasp of the problem. This part is generally pretty obvious, so it should fit in one sentence, but in some cases you may have a lot to say.

And then what?

Once your issue is filed, make sure to quickly check everything looks okay. You can edit the issue if you made a mistake, or even change its title if you realize the problem is different from what you initially thought.

There is no point pinging people if you don’t get an answer. If no one helps you in a few days, it’s likely that no one could make sense of your problem. Don’t hesitate to go back to the reproducible example. Can you make it shorter and more to the point? If you don’t get an answer in a week, you can leave a message gently asking for help, especially if you’ve edited your issue to include more information on the problem.