Training in progress, step 239, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "embed_tokens",
+    "lm_head"
+  ],
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "down_proj",
+    "up_proj",
+    "v_proj",
+    "o_proj",
+    "k_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96493447ef48c5ef5a374f6dbaa18f9bd1c19f8703b6ed28d7522c3fe2159c1d
+size 2503003904

last-checkpoint/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af7428ceaf9b1eeab0575cbaf2df17bd8d6c5b3f0cbf19e23237dd8c180df2a0
+size 5006244836

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a85bfbc10117a30a3cdf25b76454f2f5282ad7c86a91df07b8f117ba60fe5de
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8535ebad5f3dd28f5d6098f8b5c7f834756145ced2fe6bb016d9d45b73d0270a
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,20 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcfe42da0a4497e8b2b172c1f9f4ec423a46dc12907f4349c55025f670422ba9
+size 11418266

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,1706 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.1221881390593047,
+  "eval_steps": 500,
+  "global_step": 239,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0005112474437627812,
+      "grad_norm": 3.3048973083496094,
+      "learning_rate": 1.0000000000000002e-06,
+      "loss": 1.1371,
+      "step": 1
+    },
+    {
+      "epoch": 0.0010224948875255625,
+      "grad_norm": 3.735586404800415,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 1.1928,
+      "step": 2
+    },
+    {
+      "epoch": 0.0015337423312883436,
+      "grad_norm": 3.930663585662842,
+      "learning_rate": 3e-06,
+      "loss": 1.3484,
+      "step": 3
+    },
+    {
+      "epoch": 0.002044989775051125,
+      "grad_norm": 4.375387668609619,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 1.2683,
+      "step": 4
+    },
+    {
+      "epoch": 0.002556237218813906,
+      "grad_norm": 5.1499857902526855,
+      "learning_rate": 5e-06,
+      "loss": 1.1936,
+      "step": 5
+    },
+    {
+      "epoch": 0.003067484662576687,
+      "grad_norm": 5.033441066741943,
+      "learning_rate": 6e-06,
+      "loss": 1.3051,
+      "step": 6
+    },
+    {
+      "epoch": 0.0035787321063394683,
+      "grad_norm": 5.029082775115967,
+      "learning_rate": 7.000000000000001e-06,
+      "loss": 1.1804,
+      "step": 7
+    },
+    {
+      "epoch": 0.00408997955010225,
+      "grad_norm": 4.745914459228516,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 1.2379,
+      "step": 8
+    },
+    {
+      "epoch": 0.004601226993865031,
+      "grad_norm": 4.976576805114746,
+      "learning_rate": 9e-06,
+      "loss": 1.282,
+      "step": 9
+    },
+    {
+      "epoch": 0.005112474437627812,
+      "grad_norm": 4.350550174713135,
+      "learning_rate": 1e-05,
+      "loss": 1.1822,
+      "step": 10
+    },
+    {
+      "epoch": 0.005623721881390593,
+      "grad_norm": 4.9506940841674805,
+      "learning_rate": 1.1000000000000001e-05,
+      "loss": 1.2666,
+      "step": 11
+    },
+    {
+      "epoch": 0.006134969325153374,
+      "grad_norm": 5.068627834320068,
+      "learning_rate": 1.2e-05,
+      "loss": 1.2411,
+      "step": 12
+    },
+    {
+      "epoch": 0.0066462167689161555,
+      "grad_norm": 5.628175735473633,
+      "learning_rate": 1.3000000000000001e-05,
+      "loss": 1.2263,
+      "step": 13
+    },
+    {
+      "epoch": 0.007157464212678937,
+      "grad_norm": 5.16189432144165,
+      "learning_rate": 1.4000000000000001e-05,
+      "loss": 1.2809,
+      "step": 14
+    },
+    {
+      "epoch": 0.007668711656441718,
+      "grad_norm": 4.856331825256348,
+      "learning_rate": 1.5e-05,
+      "loss": 1.1944,
+      "step": 15
+    },
+    {
+      "epoch": 0.0081799591002045,
+      "grad_norm": 4.980719089508057,
+      "learning_rate": 1.6000000000000003e-05,
+      "loss": 1.2265,
+      "step": 16
+    },
+    {
+      "epoch": 0.008691206543967281,
+      "grad_norm": 5.2572832107543945,
+      "learning_rate": 1.7000000000000003e-05,
+      "loss": 1.2408,
+      "step": 17
+    },
+    {
+      "epoch": 0.009202453987730062,
+      "grad_norm": 5.128628730773926,
+      "learning_rate": 1.8e-05,
+      "loss": 1.2272,
+      "step": 18
+    },
+    {
+      "epoch": 0.009713701431492843,
+      "grad_norm": 5.2293829917907715,
+      "learning_rate": 1.9e-05,
+      "loss": 1.195,
+      "step": 19
+    },
+    {
+      "epoch": 0.010224948875255624,
+      "grad_norm": 4.794898986816406,
+      "learning_rate": 2e-05,
+      "loss": 1.2411,
+      "step": 20
+    },
+    {
+      "epoch": 0.010736196319018405,
+      "grad_norm": 5.007977485656738,
+      "learning_rate": 2.1e-05,
+      "loss": 1.1821,
+      "step": 21
+    },
+    {
+      "epoch": 0.011247443762781187,
+      "grad_norm": 4.727319240570068,
+      "learning_rate": 2.2000000000000003e-05,
+      "loss": 1.188,
+      "step": 22
+    },
+    {
+      "epoch": 0.011758691206543968,
+      "grad_norm": 4.606495380401611,
+      "learning_rate": 2.3000000000000003e-05,
+      "loss": 1.1372,
+      "step": 23
+    },
+    {
+      "epoch": 0.012269938650306749,
+      "grad_norm": 4.6991753578186035,
+      "learning_rate": 2.4e-05,
+      "loss": 1.1395,
+      "step": 24
+    },
+    {
+      "epoch": 0.01278118609406953,
+      "grad_norm": 4.635598659515381,
+      "learning_rate": 2.5e-05,
+      "loss": 1.1782,
+      "step": 25
+    },
+    {
+      "epoch": 0.013292433537832311,
+      "grad_norm": 4.882657051086426,
+      "learning_rate": 2.6000000000000002e-05,
+      "loss": 1.1939,
+      "step": 26
+    },
+    {
+      "epoch": 0.013803680981595092,
+      "grad_norm": 5.13659143447876,
+      "learning_rate": 2.7000000000000002e-05,
+      "loss": 1.1567,
+      "step": 27
+    },
+    {
+      "epoch": 0.014314928425357873,
+      "grad_norm": 5.102069854736328,
+      "learning_rate": 2.8000000000000003e-05,
+      "loss": 1.2254,
+      "step": 28
+    },
+    {
+      "epoch": 0.014826175869120654,
+      "grad_norm": 5.0762505531311035,
+      "learning_rate": 2.9e-05,
+      "loss": 1.1643,
+      "step": 29
+    },
+    {
+      "epoch": 0.015337423312883436,
+      "grad_norm": 4.819282054901123,
+      "learning_rate": 3e-05,
+      "loss": 1.0718,
+      "step": 30
+    },
+    {
+      "epoch": 0.015848670756646217,
+      "grad_norm": 4.949357032775879,
+      "learning_rate": 3.1e-05,
+      "loss": 1.091,
+      "step": 31
+    },
+    {
+      "epoch": 0.016359918200409,
+      "grad_norm": 5.103873252868652,
+      "learning_rate": 3.2000000000000005e-05,
+      "loss": 1.0602,
+      "step": 32
+    },
+    {
+      "epoch": 0.01687116564417178,
+      "grad_norm": 4.818650722503662,
+      "learning_rate": 3.3e-05,
+      "loss": 0.9318,
+      "step": 33
+    },
+    {
+      "epoch": 0.017382413087934562,
+      "grad_norm": 4.793953895568848,
+      "learning_rate": 3.4000000000000007e-05,
+      "loss": 1.0556,
+      "step": 34
+    },
+    {
+      "epoch": 0.01789366053169734,
+      "grad_norm": 5.540240287780762,
+      "learning_rate": 3.5e-05,
+      "loss": 1.1009,
+      "step": 35
+    },
+    {
+      "epoch": 0.018404907975460124,
+      "grad_norm": 5.293509483337402,
+      "learning_rate": 3.6e-05,
+      "loss": 1.1285,
+      "step": 36
+    },
+    {
+      "epoch": 0.018916155419222903,
+      "grad_norm": 5.887397289276123,
+      "learning_rate": 3.7e-05,
+      "loss": 1.0327,
+      "step": 37
+    },
+    {
+      "epoch": 0.019427402862985686,
+      "grad_norm": 4.989732265472412,
+      "learning_rate": 3.8e-05,
+      "loss": 0.9327,
+      "step": 38
+    },
+    {
+      "epoch": 0.019938650306748466,
+      "grad_norm": 5.251559257507324,
+      "learning_rate": 3.9000000000000006e-05,
+      "loss": 1.114,
+      "step": 39
+    },
+    {
+      "epoch": 0.02044989775051125,
+      "grad_norm": 6.304392337799072,
+      "learning_rate": 4e-05,
+      "loss": 1.0943,
+      "step": 40
+    },
+    {
+      "epoch": 0.020961145194274028,
+      "grad_norm": 5.590821266174316,
+      "learning_rate": 4.1e-05,
+      "loss": 1.0598,
+      "step": 41
+    },
+    {
+      "epoch": 0.02147239263803681,
+      "grad_norm": 5.784496784210205,
+      "learning_rate": 4.2e-05,
+      "loss": 1.021,
+      "step": 42
+    },
+    {
+      "epoch": 0.02198364008179959,
+      "grad_norm": 5.714466094970703,
+      "learning_rate": 4.3e-05,
+      "loss": 0.929,
+      "step": 43
+    },
+    {
+      "epoch": 0.022494887525562373,
+      "grad_norm": 6.554811000823975,
+      "learning_rate": 4.4000000000000006e-05,
+      "loss": 1.1903,
+      "step": 44
+    },
+    {
+      "epoch": 0.023006134969325152,
+      "grad_norm": 6.44047212600708,
+      "learning_rate": 4.5e-05,
+      "loss": 0.8508,
+      "step": 45
+    },
+    {
+      "epoch": 0.023517382413087935,
+      "grad_norm": 6.655950546264648,
+      "learning_rate": 4.600000000000001e-05,
+      "loss": 0.8196,
+      "step": 46
+    },
+    {
+      "epoch": 0.024028629856850715,
+      "grad_norm": 6.728236198425293,
+      "learning_rate": 4.7e-05,
+      "loss": 0.9124,
+      "step": 47
+    },
+    {
+      "epoch": 0.024539877300613498,
+      "grad_norm": 7.260775566101074,
+      "learning_rate": 4.8e-05,
+      "loss": 0.7433,
+      "step": 48
+    },
+    {
+      "epoch": 0.025051124744376277,
+      "grad_norm": 8.426919937133789,
+      "learning_rate": 4.9e-05,
+      "loss": 0.902,
+      "step": 49
+    },
+    {
+      "epoch": 0.02556237218813906,
+      "grad_norm": 14.32536506652832,
+      "learning_rate": 5e-05,
+      "loss": 0.8194,
+      "step": 50
+    },
+    {
+      "epoch": 0.02607361963190184,
+      "grad_norm": 4.535055637359619,
+      "learning_rate": 5.1000000000000006e-05,
+      "loss": 1.0298,
+      "step": 51
+    },
+    {
+      "epoch": 0.026584867075664622,
+      "grad_norm": 5.038204193115234,
+      "learning_rate": 5.2000000000000004e-05,
+      "loss": 1.0745,
+      "step": 52
+    },
+    {
+      "epoch": 0.0270961145194274,
+      "grad_norm": 6.9380927085876465,
+      "learning_rate": 5.300000000000001e-05,
+      "loss": 1.2208,
+      "step": 53
+    },
+    {
+      "epoch": 0.027607361963190184,
+      "grad_norm": 4.157510280609131,
+      "learning_rate": 5.4000000000000005e-05,
+      "loss": 1.135,
+      "step": 54
+    },
+    {
+      "epoch": 0.028118609406952964,
+      "grad_norm": 3.202308416366577,
+      "learning_rate": 5.500000000000001e-05,
+      "loss": 1.0679,
+      "step": 55
+    },
+    {
+      "epoch": 0.028629856850715747,
+      "grad_norm": 3.3872716426849365,
+      "learning_rate": 5.6000000000000006e-05,
+      "loss": 1.0871,
+      "step": 56
+    },
+    {
+      "epoch": 0.029141104294478526,
+      "grad_norm": 3.4161555767059326,
+      "learning_rate": 5.6999999999999996e-05,
+      "loss": 1.149,
+      "step": 57
+    },
+    {
+      "epoch": 0.02965235173824131,
+      "grad_norm": 4.0091962814331055,
+      "learning_rate": 5.8e-05,
+      "loss": 1.096,
+      "step": 58
+    },
+    {
+      "epoch": 0.03016359918200409,
+      "grad_norm": 3.4014410972595215,
+      "learning_rate": 5.9e-05,
+      "loss": 0.9183,
+      "step": 59
+    },
+    {
+      "epoch": 0.03067484662576687,
+      "grad_norm": 3.9149327278137207,
+      "learning_rate": 6e-05,
+      "loss": 1.0704,
+      "step": 60
+    },
+    {
+      "epoch": 0.031186094069529654,
+      "grad_norm": 4.084487438201904,
+      "learning_rate": 6.1e-05,
+      "loss": 1.0907,
+      "step": 61
+    },
+    {
+      "epoch": 0.03169734151329243,
+      "grad_norm": 3.6023764610290527,
+      "learning_rate": 6.2e-05,
+      "loss": 0.9582,
+      "step": 62
+    },
+    {
+      "epoch": 0.032208588957055216,
+      "grad_norm": 3.806703805923462,
+      "learning_rate": 6.3e-05,
+      "loss": 1.0072,
+      "step": 63
+    },
+    {
+      "epoch": 0.032719836400818,
+      "grad_norm": 3.710463523864746,
+      "learning_rate": 6.400000000000001e-05,
+      "loss": 0.9751,
+      "step": 64
+    },
+    {
+      "epoch": 0.033231083844580775,
+      "grad_norm": 3.8242218494415283,
+      "learning_rate": 6.500000000000001e-05,
+      "loss": 1.0477,
+      "step": 65
+    },
+    {
+      "epoch": 0.03374233128834356,
+      "grad_norm": 4.025355815887451,
+      "learning_rate": 6.6e-05,
+      "loss": 1.0147,
+      "step": 66
+    },
+    {
+      "epoch": 0.03425357873210634,
+      "grad_norm": 3.9239964485168457,
+      "learning_rate": 6.7e-05,
+      "loss": 1.1412,
+      "step": 67
+    },
+    {
+      "epoch": 0.034764826175869123,
+      "grad_norm": 4.182487964630127,
+      "learning_rate": 6.800000000000001e-05,
+      "loss": 1.012,
+      "step": 68
+    },
+    {
+      "epoch": 0.0352760736196319,
+      "grad_norm": 4.380066871643066,
+      "learning_rate": 6.9e-05,
+      "loss": 1.1207,
+      "step": 69
+    },
+    {
+      "epoch": 0.03578732106339468,
+      "grad_norm": 4.1866230964660645,
+      "learning_rate": 7e-05,
+      "loss": 1.0963,
+      "step": 70
+    },
+    {
+      "epoch": 0.036298568507157465,
+      "grad_norm": 4.3079833984375,
+      "learning_rate": 7.1e-05,
+      "loss": 0.9799,
+      "step": 71
+    },
+    {
+      "epoch": 0.03680981595092025,
+      "grad_norm": 4.587730407714844,
+      "learning_rate": 7.2e-05,
+      "loss": 1.0129,
+      "step": 72
+    },
+    {
+      "epoch": 0.037321063394683024,
+      "grad_norm": 4.465561389923096,
+      "learning_rate": 7.3e-05,
+      "loss": 1.0579,
+      "step": 73
+    },
+    {
+      "epoch": 0.03783231083844581,
+      "grad_norm": 4.654197692871094,
+      "learning_rate": 7.4e-05,
+      "loss": 1.0536,
+      "step": 74
+    },
+    {
+      "epoch": 0.03834355828220859,
+      "grad_norm": 4.685794353485107,
+      "learning_rate": 7.500000000000001e-05,
+      "loss": 1.0567,
+      "step": 75
+    },
+    {
+      "epoch": 0.03885480572597137,
+      "grad_norm": 4.281529426574707,
+      "learning_rate": 7.6e-05,
+      "loss": 0.9689,
+      "step": 76
+    },
+    {
+      "epoch": 0.03936605316973415,
+      "grad_norm": 4.096267223358154,
+      "learning_rate": 7.7e-05,
+      "loss": 1.0279,
+      "step": 77
+    },
+    {
+      "epoch": 0.03987730061349693,
+      "grad_norm": 4.406935214996338,
+      "learning_rate": 7.800000000000001e-05,
+      "loss": 1.0197,
+      "step": 78
+    },
+    {
+      "epoch": 0.040388548057259714,
+      "grad_norm": 4.344164848327637,
+      "learning_rate": 7.900000000000001e-05,
+      "loss": 0.9634,
+      "step": 79
+    },
+    {
+      "epoch": 0.0408997955010225,
+      "grad_norm": 4.969428539276123,
+      "learning_rate": 8e-05,
+      "loss": 1.0437,
+      "step": 80
+    },
+    {
+      "epoch": 0.04141104294478527,
+      "grad_norm": 4.627592086791992,
+      "learning_rate": 8.1e-05,
+      "loss": 1.0735,
+      "step": 81
+    },
+    {
+      "epoch": 0.041922290388548056,
+      "grad_norm": 4.50595760345459,
+      "learning_rate": 8.2e-05,
+      "loss": 0.9614,
+      "step": 82
+    },
+    {
+      "epoch": 0.04243353783231084,
+      "grad_norm": 4.57030725479126,
+      "learning_rate": 8.3e-05,
+      "loss": 0.9109,
+      "step": 83
+    },
+    {
+      "epoch": 0.04294478527607362,
+      "grad_norm": 5.381258487701416,
+      "learning_rate": 8.4e-05,
+      "loss": 1.1194,
+      "step": 84
+    },
+    {
+      "epoch": 0.0434560327198364,
+      "grad_norm": 5.224101543426514,
+      "learning_rate": 8.5e-05,
+      "loss": 1.0022,
+      "step": 85
+    },
+    {
+      "epoch": 0.04396728016359918,
+      "grad_norm": 4.670103073120117,
+      "learning_rate": 8.6e-05,
+      "loss": 0.9712,
+      "step": 86
+    },
+    {
+      "epoch": 0.04447852760736196,
+      "grad_norm": 5.264283657073975,
+      "learning_rate": 8.7e-05,
+      "loss": 0.9767,
+      "step": 87
+    },
+    {
+      "epoch": 0.044989775051124746,
+      "grad_norm": 4.800637245178223,
+      "learning_rate": 8.800000000000001e-05,
+      "loss": 0.8713,
+      "step": 88
+    },
+    {
+      "epoch": 0.04550102249488753,
+      "grad_norm": 5.133697986602783,
+      "learning_rate": 8.900000000000001e-05,
+      "loss": 0.9959,
+      "step": 89
+    },
+    {
+      "epoch": 0.046012269938650305,
+      "grad_norm": 5.831608295440674,
+      "learning_rate": 9e-05,
+      "loss": 0.9995,
+      "step": 90
+    },
+    {
+      "epoch": 0.04652351738241309,
+      "grad_norm": 5.324717998504639,
+      "learning_rate": 9.1e-05,
+      "loss": 0.9143,
+      "step": 91
+    },
+    {
+      "epoch": 0.04703476482617587,
+      "grad_norm": 6.4028401374816895,
+      "learning_rate": 9.200000000000001e-05,
+      "loss": 1.1914,
+      "step": 92
+    },
+    {
+      "epoch": 0.04754601226993865,
+      "grad_norm": 6.07197904586792,
+      "learning_rate": 9.300000000000001e-05,
+      "loss": 0.9809,
+      "step": 93
+    },
+    {
+      "epoch": 0.04805725971370143,
+      "grad_norm": 5.957538604736328,
+      "learning_rate": 9.4e-05,
+      "loss": 1.0473,
+      "step": 94
+    },
+    {
+      "epoch": 0.04856850715746421,
+      "grad_norm": 5.8357343673706055,
+      "learning_rate": 9.5e-05,
+      "loss": 0.9465,
+      "step": 95
+    },
+    {
+      "epoch": 0.049079754601226995,
+      "grad_norm": 5.610650062561035,
+      "learning_rate": 9.6e-05,
+      "loss": 0.7326,
+      "step": 96
+    },
+    {
+      "epoch": 0.04959100204498978,
+      "grad_norm": 6.265199661254883,
+      "learning_rate": 9.7e-05,
+      "loss": 0.9174,
+      "step": 97
+    },
+    {
+      "epoch": 0.050102249488752554,
+      "grad_norm": 7.403947353363037,
+      "learning_rate": 9.8e-05,
+      "loss": 0.8325,
+      "step": 98
+    },
+    {
+      "epoch": 0.05061349693251534,
+      "grad_norm": 7.57144021987915,
+      "learning_rate": 9.900000000000001e-05,
+      "loss": 0.6444,
+      "step": 99
+    },
+    {
+      "epoch": 0.05112474437627812,
+      "grad_norm": 10.456438064575195,
+      "learning_rate": 0.0001,
+      "loss": 0.6351,
+      "step": 100
+    },
+    {
+      "epoch": 0.0516359918200409,
+      "grad_norm": 3.450286865234375,
+      "learning_rate": 9.999992837185459e-05,
+      "loss": 1.1081,
+      "step": 101
+    },
+    {
+      "epoch": 0.05214723926380368,
+      "grad_norm": 3.6296398639678955,
+      "learning_rate": 9.999971348762359e-05,
+      "loss": 1.1465,
+      "step": 102
+    },
+    {
+      "epoch": 0.05265848670756646,
+      "grad_norm": 3.699775457382202,
+      "learning_rate": 9.999935534792264e-05,
+      "loss": 1.19,
+      "step": 103
+    },
+    {
+      "epoch": 0.053169734151329244,
+      "grad_norm": 3.4316041469573975,
+      "learning_rate": 9.999885395377788e-05,
+      "loss": 1.1055,
+      "step": 104
+    },
+    {
+      "epoch": 0.05368098159509203,
+      "grad_norm": 3.6994450092315674,
+      "learning_rate": 9.999820930662585e-05,
+      "loss": 1.1867,
+      "step": 105
+    },
+    {
+      "epoch": 0.0541922290388548,
+      "grad_norm": 3.6903419494628906,
+      "learning_rate": 9.999742140831357e-05,
+      "loss": 1.077,
+      "step": 106
+    },
+    {
+      "epoch": 0.054703476482617586,
+      "grad_norm": 4.079493522644043,
+      "learning_rate": 9.999649026109845e-05,
+      "loss": 1.1008,
+      "step": 107
+    },
+    {
+      "epoch": 0.05521472392638037,
+      "grad_norm": 3.9423539638519287,
+      "learning_rate": 9.999541586764836e-05,
+      "loss": 1.2268,
+      "step": 108
+    },
+    {
+      "epoch": 0.05572597137014315,
+      "grad_norm": 4.003759860992432,
+      "learning_rate": 9.999419823104155e-05,
+      "loss": 1.0735,
+      "step": 109
+    },
+    {
+      "epoch": 0.05623721881390593,
+      "grad_norm": 3.989661693572998,
+      "learning_rate": 9.999283735476672e-05,
+      "loss": 1.0735,
+      "step": 110
+    },
+    {
+      "epoch": 0.05674846625766871,
+      "grad_norm": 3.9586427211761475,
+      "learning_rate": 9.999133324272294e-05,
+      "loss": 1.0708,
+      "step": 111
+    },
+    {
+      "epoch": 0.05725971370143149,
+      "grad_norm": 4.020519733428955,
+      "learning_rate": 9.998968589921969e-05,
+      "loss": 1.0245,
+      "step": 112
+    },
+    {
+      "epoch": 0.057770961145194276,
+      "grad_norm": 4.426997184753418,
+      "learning_rate": 9.998789532897681e-05,
+      "loss": 1.0745,
+      "step": 113
+    },
+    {
+      "epoch": 0.05828220858895705,
+      "grad_norm": 3.993692636489868,
+      "learning_rate": 9.998596153712451e-05,
+      "loss": 1.0767,
+      "step": 114
+    },
+    {
+      "epoch": 0.058793456032719835,
+      "grad_norm": 4.429073333740234,
+      "learning_rate": 9.998388452920334e-05,
+      "loss": 1.017,
+      "step": 115
+    },
+    {
+      "epoch": 0.05930470347648262,
+      "grad_norm": 3.983215093612671,
+      "learning_rate": 9.99816643111642e-05,
+      "loss": 1.021,
+      "step": 116
+    },
+    {
+      "epoch": 0.0598159509202454,
+      "grad_norm": 4.212836742401123,
+      "learning_rate": 9.997930088936828e-05,
+      "loss": 0.982,
+      "step": 117
+    },
+    {
+      "epoch": 0.06032719836400818,
+      "grad_norm": 4.086669921875,
+      "learning_rate": 9.997679427058713e-05,
+      "loss": 1.0085,
+      "step": 118
+    },
+    {
+      "epoch": 0.06083844580777096,
+      "grad_norm": 3.8549580574035645,
+      "learning_rate": 9.997414446200246e-05,
+      "loss": 0.9703,
+      "step": 119
+    },
+    {
+      "epoch": 0.06134969325153374,
+      "grad_norm": 4.432587623596191,
+      "learning_rate": 9.997135147120633e-05,
+      "loss": 1.0989,
+      "step": 120
+    },
+    {
+      "epoch": 0.061860940695296525,
+      "grad_norm": 4.559906959533691,
+      "learning_rate": 9.996841530620103e-05,
+      "loss": 0.9276,
+      "step": 121
+    },
+    {
+      "epoch": 0.06237218813905931,
+      "grad_norm": 4.4931721687316895,
+      "learning_rate": 9.996533597539901e-05,
+      "loss": 0.9899,
+      "step": 122
+    },
+    {
+      "epoch": 0.06288343558282208,
+      "grad_norm": 4.057201862335205,
+      "learning_rate": 9.996211348762296e-05,
+      "loss": 1.0011,
+      "step": 123
+    },
+    {
+      "epoch": 0.06339468302658487,
+      "grad_norm": 4.463088512420654,
+      "learning_rate": 9.995874785210573e-05,
+      "loss": 0.9554,
+      "step": 124
+    },
+    {
+      "epoch": 0.06390593047034765,
+      "grad_norm": 4.215217590332031,
+      "learning_rate": 9.995523907849024e-05,
+      "loss": 0.9568,
+      "step": 125
+    },
+    {
+      "epoch": 0.06441717791411043,
+      "grad_norm": 4.900539875030518,
+      "learning_rate": 9.995158717682963e-05,
+      "loss": 0.9864,
+      "step": 126
+    },
+    {
+      "epoch": 0.06492842535787322,
+      "grad_norm": 4.836815357208252,
+      "learning_rate": 9.9947792157587e-05,
+      "loss": 0.9749,
+      "step": 127
+    },
+    {
+      "epoch": 0.065439672801636,
+      "grad_norm": 4.402955055236816,
+      "learning_rate": 9.99438540316356e-05,
+      "loss": 0.9626,
+      "step": 128
+    },
+    {
+      "epoch": 0.06595092024539877,
+      "grad_norm": 4.344229698181152,
+      "learning_rate": 9.993977281025862e-05,
+      "loss": 0.9272,
+      "step": 129
+    },
+    {
+      "epoch": 0.06646216768916155,
+      "grad_norm": 4.652369976043701,
+      "learning_rate": 9.99355485051493e-05,
+      "loss": 1.0139,
+      "step": 130
+    },
+    {
+      "epoch": 0.06697341513292433,
+      "grad_norm": 4.622213840484619,
+      "learning_rate": 9.99311811284108e-05,
+      "loss": 0.9265,
+      "step": 131
+    },
+    {
+      "epoch": 0.06748466257668712,
+      "grad_norm": 5.247522830963135,
+      "learning_rate": 9.992667069255619e-05,
+      "loss": 1.0134,
+      "step": 132
+    },
+    {
+      "epoch": 0.0679959100204499,
+      "grad_norm": 4.564031600952148,
+      "learning_rate": 9.992201721050847e-05,
+      "loss": 0.9342,
+      "step": 133
+    },
+    {
+      "epoch": 0.06850715746421268,
+      "grad_norm": 4.3959245681762695,
+      "learning_rate": 9.991722069560041e-05,
+      "loss": 0.9452,
+      "step": 134
+    },
+    {
+      "epoch": 0.06901840490797546,
+      "grad_norm": 4.695987701416016,
+      "learning_rate": 9.991228116157466e-05,
+      "loss": 0.9721,
+      "step": 135
+    },
+    {
+      "epoch": 0.06952965235173825,
+      "grad_norm": 4.595483303070068,
+      "learning_rate": 9.990719862258358e-05,
+      "loss": 0.9288,
+      "step": 136
+    },
+    {
+      "epoch": 0.07004089979550102,
+      "grad_norm": 4.6917009353637695,
+      "learning_rate": 9.99019730931893e-05,
+      "loss": 0.8785,
+      "step": 137
+    },
+    {
+      "epoch": 0.0705521472392638,
+      "grad_norm": 6.002026081085205,
+      "learning_rate": 9.98966045883636e-05,
+      "loss": 0.9841,
+      "step": 138
+    },
+    {
+      "epoch": 0.07106339468302658,
+      "grad_norm": 5.35076379776001,
+      "learning_rate": 9.989109312348796e-05,
+      "loss": 1.0745,
+      "step": 139
+    },
+    {
+      "epoch": 0.07157464212678936,
+      "grad_norm": 5.220756530761719,
+      "learning_rate": 9.98854387143534e-05,
+      "loss": 0.9308,
+      "step": 140
+    },
+    {
+      "epoch": 0.07208588957055215,
+      "grad_norm": 5.255858421325684,
+      "learning_rate": 9.98796413771605e-05,
+      "loss": 0.9596,
+      "step": 141
+    },
+    {
+      "epoch": 0.07259713701431493,
+      "grad_norm": 5.472203254699707,
+      "learning_rate": 9.987370112851939e-05,
+      "loss": 0.9245,
+      "step": 142
+    },
+    {
+      "epoch": 0.07310838445807771,
+      "grad_norm": 5.4891252517700195,
+      "learning_rate": 9.98676179854496e-05,
+      "loss": 0.9549,
+      "step": 143
+    },
+    {
+      "epoch": 0.0736196319018405,
+      "grad_norm": 5.602303504943848,
+      "learning_rate": 9.986139196538011e-05,
+      "loss": 0.9336,
+      "step": 144
+    },
+    {
+      "epoch": 0.07413087934560328,
+      "grad_norm": 6.35874080657959,
+      "learning_rate": 9.985502308614927e-05,
+      "loss": 1.0135,
+      "step": 145
+    },
+    {
+      "epoch": 0.07464212678936605,
+      "grad_norm": 6.352203369140625,
+      "learning_rate": 9.984851136600469e-05,
+      "loss": 0.8946,
+      "step": 146
+    },
+    {
+      "epoch": 0.07515337423312883,
+      "grad_norm": 6.650132179260254,
+      "learning_rate": 9.984185682360327e-05,
+      "loss": 0.943,
+      "step": 147
+    },
+    {
+      "epoch": 0.07566462167689161,
+      "grad_norm": 7.255720138549805,
+      "learning_rate": 9.983505947801115e-05,
+      "loss": 0.8747,
+      "step": 148
+    },
+    {
+      "epoch": 0.0761758691206544,
+      "grad_norm": 7.385144233703613,
+      "learning_rate": 9.982811934870353e-05,
+      "loss": 0.7153,
+      "step": 149
+    },
+    {
+      "epoch": 0.07668711656441718,
+      "grad_norm": 11.432644844055176,
+      "learning_rate": 9.982103645556478e-05,
+      "loss": 1.1386,
+      "step": 150
+    },
+    {
+      "epoch": 0.07719836400817996,
+      "grad_norm": 3.1650919914245605,
+      "learning_rate": 9.981381081888828e-05,
+      "loss": 1.0661,
+      "step": 151
+    },
+    {
+      "epoch": 0.07770961145194274,
+      "grad_norm": 3.7646584510803223,
+      "learning_rate": 9.980644245937639e-05,
+      "loss": 1.1944,
+      "step": 152
+    },
+    {
+      "epoch": 0.07822085889570553,
+      "grad_norm": 3.43169903755188,
+      "learning_rate": 9.979893139814039e-05,
+      "loss": 1.1582,
+      "step": 153
+    },
+    {
+      "epoch": 0.0787321063394683,
+      "grad_norm": 3.5647740364074707,
+      "learning_rate": 9.979127765670039e-05,
+      "loss": 1.1398,
+      "step": 154
+    },
+    {
+      "epoch": 0.07924335378323108,
+      "grad_norm": 3.8447012901306152,
+      "learning_rate": 9.978348125698534e-05,
+      "loss": 1.1126,
+      "step": 155
+    },
+    {
+      "epoch": 0.07975460122699386,
+      "grad_norm": 3.9558143615722656,
+      "learning_rate": 9.977554222133292e-05,
+      "loss": 1.1155,
+      "step": 156
+    },
+    {
+      "epoch": 0.08026584867075665,
+      "grad_norm": 3.363036870956421,
+      "learning_rate": 9.976746057248943e-05,
+      "loss": 0.9753,
+      "step": 157
+    },
+    {
+      "epoch": 0.08077709611451943,
+      "grad_norm": 4.080865383148193,
+      "learning_rate": 9.975923633360985e-05,
+      "loss": 1.1136,
+      "step": 158
+    },
+    {
+      "epoch": 0.08128834355828221,
+      "grad_norm": 3.7571935653686523,
+      "learning_rate": 9.975086952825762e-05,
+      "loss": 1.1015,
+      "step": 159
+    },
+    {
+      "epoch": 0.081799591002045,
+      "grad_norm": 3.6946327686309814,
+      "learning_rate": 9.974236018040474e-05,
+      "loss": 1.0684,
+      "step": 160
+    },
+    {
+      "epoch": 0.08231083844580778,
+      "grad_norm": 4.085092544555664,
+      "learning_rate": 9.973370831443152e-05,
+      "loss": 1.082,
+      "step": 161
+    },
+    {
+      "epoch": 0.08282208588957055,
+      "grad_norm": 4.023382186889648,
+      "learning_rate": 9.972491395512666e-05,
+      "loss": 1.059,
+      "step": 162
+    },
+    {
+      "epoch": 0.08333333333333333,
+      "grad_norm": 4.119484901428223,
+      "learning_rate": 9.97159771276871e-05,
+      "loss": 0.996,
+      "step": 163
+    },
+    {
+      "epoch": 0.08384458077709611,
+      "grad_norm": 3.6977996826171875,
+      "learning_rate": 9.970689785771798e-05,
+      "loss": 1.0283,
+      "step": 164
+    },
+    {
+      "epoch": 0.0843558282208589,
+      "grad_norm": 3.9231314659118652,
+      "learning_rate": 9.969767617123256e-05,
+      "loss": 1.0912,
+      "step": 165
+    },
+    {
+      "epoch": 0.08486707566462168,
+      "grad_norm": 3.965254306793213,
+      "learning_rate": 9.968831209465211e-05,
+      "loss": 1.0856,
+      "step": 166
+    },
+    {
+      "epoch": 0.08537832310838446,
+      "grad_norm": 3.6064834594726562,
+      "learning_rate": 9.967880565480589e-05,
+      "loss": 0.9545,
+      "step": 167
+    },
+    {
+      "epoch": 0.08588957055214724,
+      "grad_norm": 4.335506439208984,
+      "learning_rate": 9.966915687893108e-05,
+      "loss": 1.0064,
+      "step": 168
+    },
+    {
+      "epoch": 0.08640081799591003,
+      "grad_norm": 4.090417385101318,
+      "learning_rate": 9.96593657946726e-05,
+      "loss": 0.9032,
+      "step": 169
+    },
+    {
+      "epoch": 0.0869120654396728,
+      "grad_norm": 4.244670391082764,
+      "learning_rate": 9.964943243008315e-05,
+      "loss": 0.9544,
+      "step": 170
+    },
+    {
+      "epoch": 0.08742331288343558,
+      "grad_norm": 4.027634143829346,
+      "learning_rate": 9.963935681362308e-05,
+      "loss": 1.0019,
+      "step": 171
+    },
+    {
+      "epoch": 0.08793456032719836,
+      "grad_norm": 4.118989944458008,
+      "learning_rate": 9.962913897416028e-05,
+      "loss": 0.9132,
+      "step": 172
+    },
+    {
+      "epoch": 0.08844580777096114,
+      "grad_norm": 4.275842666625977,
+      "learning_rate": 9.961877894097016e-05,
+      "loss": 0.9875,
+      "step": 173
+    },
+    {
+      "epoch": 0.08895705521472393,
+      "grad_norm": 4.40177059173584,
+      "learning_rate": 9.96082767437355e-05,
+      "loss": 1.0284,
+      "step": 174
+    },
+    {
+      "epoch": 0.08946830265848671,
+      "grad_norm": 4.55718994140625,
+      "learning_rate": 9.959763241254645e-05,
+      "loss": 1.0491,
+      "step": 175
+    },
+    {
+      "epoch": 0.08997955010224949,
+      "grad_norm": 3.950263261795044,
+      "learning_rate": 9.958684597790031e-05,
+      "loss": 0.8668,
+      "step": 176
+    },
+    {
+      "epoch": 0.09049079754601227,
+      "grad_norm": 4.344690322875977,
+      "learning_rate": 9.957591747070163e-05,
+      "loss": 0.9561,
+      "step": 177
+    },
+    {
+      "epoch": 0.09100204498977506,
+      "grad_norm": 4.7496232986450195,
+      "learning_rate": 9.95648469222619e-05,
+      "loss": 1.2058,
+      "step": 178
+    },
+    {
+      "epoch": 0.09151329243353783,
+      "grad_norm": 4.409940242767334,
+      "learning_rate": 9.955363436429968e-05,
+      "loss": 1.0548,
+      "step": 179
+    },
+    {
+      "epoch": 0.09202453987730061,
+      "grad_norm": 4.629715919494629,
+      "learning_rate": 9.954227982894034e-05,
+      "loss": 0.9482,
+      "step": 180
+    },
+    {
+      "epoch": 0.09253578732106339,
+      "grad_norm": 4.308309078216553,
+      "learning_rate": 9.953078334871606e-05,
+      "loss": 0.9435,
+      "step": 181
+    },
+    {
+      "epoch": 0.09304703476482618,
+      "grad_norm": 4.249781131744385,
+      "learning_rate": 9.951914495656569e-05,
+      "loss": 0.9578,
+      "step": 182
+    },
+    {
+      "epoch": 0.09355828220858896,
+      "grad_norm": 4.906545162200928,
+      "learning_rate": 9.950736468583468e-05,
+      "loss": 1.0188,
+      "step": 183
+    },
+    {
+      "epoch": 0.09406952965235174,
+      "grad_norm": 5.413058280944824,
+      "learning_rate": 9.949544257027502e-05,
+      "loss": 1.1337,
+      "step": 184
+    },
+    {
+      "epoch": 0.09458077709611452,
+      "grad_norm": 4.498751640319824,
+      "learning_rate": 9.948337864404504e-05,
+      "loss": 0.9442,
+      "step": 185
+    },
+    {
+      "epoch": 0.0950920245398773,
+      "grad_norm": 4.727104187011719,
+      "learning_rate": 9.947117294170944e-05,
+      "loss": 1.0067,
+      "step": 186
+    },
+    {
+      "epoch": 0.09560327198364008,
+      "grad_norm": 4.974936008453369,
+      "learning_rate": 9.945882549823906e-05,
+      "loss": 1.0294,
+      "step": 187
+    },
+    {
+      "epoch": 0.09611451942740286,
+      "grad_norm": 5.312014102935791,
+      "learning_rate": 9.944633634901088e-05,
+      "loss": 0.9887,
+      "step": 188
+    },
+    {
+      "epoch": 0.09662576687116564,
+      "grad_norm": 5.3448405265808105,
+      "learning_rate": 9.943370552980791e-05,
+      "loss": 0.9342,
+      "step": 189
+    },
+    {
+      "epoch": 0.09713701431492842,
+      "grad_norm": 5.098217964172363,
+      "learning_rate": 9.942093307681901e-05,
+      "loss": 0.9,
+      "step": 190
+    },
+    {
+      "epoch": 0.09764826175869121,
+      "grad_norm": 5.733004570007324,
+      "learning_rate": 9.940801902663889e-05,
+      "loss": 1.0274,
+      "step": 191
+    },
+    {
+      "epoch": 0.09815950920245399,
+      "grad_norm": 5.353964328765869,
+      "learning_rate": 9.939496341626791e-05,
+      "loss": 1.0018,
+      "step": 192
+    },
+    {
+      "epoch": 0.09867075664621677,
+      "grad_norm": 5.414928913116455,
+      "learning_rate": 9.938176628311204e-05,
+      "loss": 0.8237,
+      "step": 193
+    },
+    {
+      "epoch": 0.09918200408997956,
+      "grad_norm": 5.456855297088623,
+      "learning_rate": 9.936842766498273e-05,
+      "loss": 0.9474,
+      "step": 194
+    },
+    {
+      "epoch": 0.09969325153374232,
+      "grad_norm": 6.7383856773376465,
+      "learning_rate": 9.935494760009678e-05,
+      "loss": 1.0278,
+      "step": 195
+    },
+    {
+      "epoch": 0.10020449897750511,
+      "grad_norm": 6.588730812072754,
+      "learning_rate": 9.934132612707632e-05,
+      "loss": 1.0217,
+      "step": 196
+    },
+    {
+      "epoch": 0.10071574642126789,
+      "grad_norm": 7.4416093826293945,
+      "learning_rate": 9.932756328494852e-05,
+      "loss": 0.9317,
+      "step": 197
+    },
+    {
+      "epoch": 0.10122699386503067,
+      "grad_norm": 7.225059986114502,
+      "learning_rate": 9.931365911314572e-05,
+      "loss": 0.8769,
+      "step": 198
+    },
+    {
+      "epoch": 0.10173824130879346,
+      "grad_norm": 7.279494285583496,
+      "learning_rate": 9.929961365150506e-05,
+      "loss": 0.6817,
+      "step": 199
+    },
+    {
+      "epoch": 0.10224948875255624,
+      "grad_norm": 10.894880294799805,
+      "learning_rate": 9.928542694026862e-05,
+      "loss": 0.9749,
+      "step": 200
+    },
+    {
+      "epoch": 0.10276073619631902,
+      "grad_norm": 3.4081413745880127,
+      "learning_rate": 9.927109902008303e-05,
+      "loss": 0.9225,
+      "step": 201
+    },
+    {
+      "epoch": 0.1032719836400818,
+      "grad_norm": 4.234488010406494,
+      "learning_rate": 9.925662993199967e-05,
+      "loss": 1.1298,
+      "step": 202
+    },
+    {
+      "epoch": 0.10378323108384459,
+      "grad_norm": 3.798875093460083,
+      "learning_rate": 9.924201971747423e-05,
+      "loss": 1.0496,
+      "step": 203
+    },
+    {
+      "epoch": 0.10429447852760736,
+      "grad_norm": 3.6677727699279785,
+      "learning_rate": 9.922726841836684e-05,
+      "loss": 1.1259,
+      "step": 204
+    },
+    {
+      "epoch": 0.10480572597137014,
+      "grad_norm": 3.7715201377868652,
+      "learning_rate": 9.921237607694184e-05,
+      "loss": 1.105,
+      "step": 205
+    },
+    {
+      "epoch": 0.10531697341513292,
+      "grad_norm": 3.534221649169922,
+      "learning_rate": 9.919734273586767e-05,
+      "loss": 1.0223,
+      "step": 206
+    },
+    {
+      "epoch": 0.1058282208588957,
+      "grad_norm": 3.9099631309509277,
+      "learning_rate": 9.91821684382167e-05,
+      "loss": 1.0874,
+      "step": 207
+    },
+    {
+      "epoch": 0.10633946830265849,
+      "grad_norm": 3.318946361541748,
+      "learning_rate": 9.916685322746524e-05,
+      "loss": 0.9967,
+      "step": 208
+    },
+    {
+      "epoch": 0.10685071574642127,
+      "grad_norm": 4.089531421661377,
+      "learning_rate": 9.915139714749328e-05,
+      "loss": 0.973,
+      "step": 209
+    },
+    {
+      "epoch": 0.10736196319018405,
+      "grad_norm": 3.753154993057251,
+      "learning_rate": 9.913580024258442e-05,
+      "loss": 1.0184,
+      "step": 210
+    },
+    {
+      "epoch": 0.10787321063394684,
+      "grad_norm": 4.053317070007324,
+      "learning_rate": 9.912006255742579e-05,
+      "loss": 1.1166,
+      "step": 211
+    },
+    {
+      "epoch": 0.1083844580777096,
+      "grad_norm": 3.554300308227539,
+      "learning_rate": 9.91041841371078e-05,
+      "loss": 0.9794,
+      "step": 212
+    },
+    {
+      "epoch": 0.10889570552147239,
+      "grad_norm": 3.9923880100250244,
+      "learning_rate": 9.908816502712415e-05,
+      "loss": 1.1504,
+      "step": 213
+    },
+    {
+      "epoch": 0.10940695296523517,
+      "grad_norm": 3.900874137878418,
+      "learning_rate": 9.90720052733716e-05,
+      "loss": 1.0554,
+      "step": 214
+    },
+    {
+      "epoch": 0.10991820040899795,
+      "grad_norm": 3.914747714996338,
+      "learning_rate": 9.905570492214987e-05,
+      "loss": 1.0801,
+      "step": 215
+    },
+    {
+      "epoch": 0.11042944785276074,
+      "grad_norm": 4.028392791748047,
+      "learning_rate": 9.903926402016153e-05,
+      "loss": 1.0332,
+      "step": 216
+    },
+    {
+      "epoch": 0.11094069529652352,
+      "grad_norm": 4.589059829711914,
+      "learning_rate": 9.902268261451182e-05,
+      "loss": 1.0179,
+      "step": 217
+    },
+    {
+      "epoch": 0.1114519427402863,
+      "grad_norm": 4.278644561767578,
+      "learning_rate": 9.900596075270856e-05,
+      "loss": 1.0743,
+      "step": 218
+    },
+    {
+      "epoch": 0.11196319018404909,
+      "grad_norm": 4.3040008544921875,
+      "learning_rate": 9.898909848266196e-05,
+      "loss": 0.9975,
+      "step": 219
+    },
+    {
+      "epoch": 0.11247443762781185,
+      "grad_norm": 3.8854451179504395,
+      "learning_rate": 9.897209585268458e-05,
+      "loss": 1.003,
+      "step": 220
+    },
+    {
+      "epoch": 0.11298568507157464,
+      "grad_norm": 4.615149021148682,
+      "learning_rate": 9.89549529114911e-05,
+      "loss": 0.9241,
+      "step": 221
+    },
+    {
+      "epoch": 0.11349693251533742,
+      "grad_norm": 4.247766494750977,
+      "learning_rate": 9.893766970819819e-05,
+      "loss": 0.9801,
+      "step": 222
+    },
+    {
+      "epoch": 0.1140081799591002,
+      "grad_norm": 4.650023460388184,
+      "learning_rate": 9.89202462923244e-05,
+      "loss": 1.0031,
+      "step": 223
+    },
+    {
+      "epoch": 0.11451942740286299,
+      "grad_norm": 4.242456912994385,
+      "learning_rate": 9.890268271379e-05,
+      "loss": 1.0527,
+      "step": 224
+    },
+    {
+      "epoch": 0.11503067484662577,
+      "grad_norm": 4.012096881866455,
+      "learning_rate": 9.888497902291685e-05,
+      "loss": 0.9761,
+      "step": 225
+    },
+    {
+      "epoch": 0.11554192229038855,
+      "grad_norm": 4.403133392333984,
+      "learning_rate": 9.886713527042828e-05,
+      "loss": 0.9944,
+      "step": 226
+    },
+    {
+      "epoch": 0.11605316973415133,
+      "grad_norm": 4.339402675628662,
+      "learning_rate": 9.884915150744885e-05,
+      "loss": 1.0336,
+      "step": 227
+    },
+    {
+      "epoch": 0.1165644171779141,
+      "grad_norm": 5.434019088745117,
+      "learning_rate": 9.883102778550434e-05,
+      "loss": 1.0191,
+      "step": 228
+    },
+    {
+      "epoch": 0.11707566462167689,
+      "grad_norm": 4.682615280151367,
+      "learning_rate": 9.881276415652148e-05,
+      "loss": 0.9405,
+      "step": 229
+    },
+    {
+      "epoch": 0.11758691206543967,
+      "grad_norm": 4.088139057159424,
+      "learning_rate": 9.879436067282783e-05,
+      "loss": 0.8859,
+      "step": 230
+    },
+    {
+      "epoch": 0.11809815950920245,
+      "grad_norm": 4.25062370300293,
+      "learning_rate": 9.877581738715174e-05,
+      "loss": 0.9826,
+      "step": 231
+    },
+    {
+      "epoch": 0.11860940695296524,
+      "grad_norm": 4.560945510864258,
+      "learning_rate": 9.875713435262203e-05,
+      "loss": 0.8834,
+      "step": 232
+    },
+    {
+      "epoch": 0.11912065439672802,
+      "grad_norm": 4.465799331665039,
+      "learning_rate": 9.873831162276796e-05,
+      "loss": 0.9343,
+      "step": 233
+    },
+    {
+      "epoch": 0.1196319018404908,
+      "grad_norm": 4.841387748718262,
+      "learning_rate": 9.871934925151898e-05,
+      "loss": 0.9551,
+      "step": 234
+    },
+    {
+      "epoch": 0.12014314928425358,
+      "grad_norm": 4.5967326164245605,
+      "learning_rate": 9.870024729320471e-05,
+      "loss": 0.9566,
+      "step": 235
+    },
+    {
+      "epoch": 0.12065439672801637,
+      "grad_norm": 5.284646987915039,
+      "learning_rate": 9.868100580255466e-05,
+      "loss": 0.9958,
+      "step": 236
+    },
+    {
+      "epoch": 0.12116564417177914,
+      "grad_norm": 5.193093299865723,
+      "learning_rate": 9.86616248346981e-05,
+      "loss": 1.0898,
+      "step": 237
+    },
+    {
+      "epoch": 0.12167689161554192,
+      "grad_norm": 5.222631931304932,
+      "learning_rate": 9.864210444516395e-05,
+      "loss": 0.9452,
+      "step": 238
+    },
+    {
+      "epoch": 0.1221881390593047,
+      "grad_norm": 5.9728569984436035,
+      "learning_rate": 9.86224446898806e-05,
+      "loss": 1.0229,
+      "step": 239
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 1956,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 239,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 9.135735681790771e+16,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc5251bf95070f231470f9c38db9e7d00dc699a13c521ea498c351ab37f51204
+size 6840