|
--- |
|
language: "Python" |
|
tags: |
|
- Code |
|
- GPyT |
|
- code generator |
|
license: "MIT" |
|
--- |
|
|
|
|
|
GPyT is a GPT2 model trained from scratch (not fine tuned) on Python code from Github. Overall, it was ~80GB of pure Python code, the current GPyT model is a mere 2 epochs through this data, so it may benefit greatly from continued training and/or fine-tuning. |
|
|
|
Newlines are replaced by `<N>` |
|
|
|
|
|
Input to the model is code, up to the context length of 1024, with newlines replaced by `<N>` |
|
|
|
Here's an example of a quick converter to take your multi-line code and replace the newlines: |
|
|
|
```py |
|
inp = """def do_something(): |
|
print("Hello") |
|
""" |
|
|
|
newlinechar = "<N>" |
|
converted = inp.replace("\ |
|
", newlinechar) |
|
print("length:", len(converted)) |
|
print(converted) |
|
``` |
|
|
|
This should give you something like: |
|
|
|
`def do_something():<N> print("Hello")<N>` |
|
|
|
...which is what the model is expecting as input. |
|
|
|
Considerations: |
|
|
|
1. This model is intended for educational and research use only. Do not trust model outputs. |
|
2. Model is highly likely to regurgitate code almost exactly as it saw it. It's up to you to determine licensing if you intend to actually use the generated code. |
|
3. All Python code was blindly pulled from github. This means included code is both Python 2 and 3, among other more subtle differences, such as tabs being 2 spaces in some cases and 4 in others...and more non-homologous things. |
|
4. Along with the above, this means the code generated could wind up doing or suggesting just about anything. Run the generated code at own risk...it could be *anything* |
|
|
|
|
|
|
|
|
