Training in progress, epoch 0, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: katuni4ka/tiny-random-olmo-hf
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "katuni4ka/tiny-random-olmo-hf",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "up_proj",
+    "k_proj",
+    "down_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d28ac96260ae2a6cf65ad8641e6d48243b65401e32c33eab1742e86adbc5c92
+size 315888

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a7c03caa6ec27a6f73ce237b5cf271d795b44285cb514bb52d23d14a4b8a1ca
+size 648522

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:586828fa3e2cb15a58da7e28bb8a0982c981adafa1537bd938535b45f69a0278
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:321880767d0a20ffe7202eaf538e97cbcbb29b9663e8d5355a6dbafc2eeacd61
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,16 @@
+{
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,247 @@
+{
+  "add_bos_token": false,
+  "add_eos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "|||IP_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "1": {
+      "content": "<|padding|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50254": {
+      "content": "                        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50255": {
+      "content": "                       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50256": {
+      "content": "                      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50257": {
+      "content": "                     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50258": {
+      "content": "                    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50259": {
+      "content": "                   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50260": {
+      "content": "                  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50261": {
+      "content": "                 ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50262": {
+      "content": "                ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50263": {
+      "content": "               ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50264": {
+      "content": "              ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50265": {
+      "content": "             ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50266": {
+      "content": "            ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50267": {
+      "content": "           ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50268": {
+      "content": "          ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50269": {
+      "content": "         ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50270": {
+      "content": "        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50271": {
+      "content": "       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50272": {
+      "content": "      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50273": {
+      "content": "     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50274": {
+      "content": "    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50275": {
+      "content": "   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50276": {
+      "content": "  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50277": {
+      "content": "|||EMAIL_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50278": {
+      "content": "|||PHONE_NUMBER|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50279": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50280": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": null,
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "tokenizer_class": "GPTNeoXTokenizer",
+  "unk_token": null
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,699 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.9947089947089947,
+  "eval_steps": 500,
+  "global_step": 94,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.010582010582010581,
+      "grad_norm": 0.15184591710567474,
+      "learning_rate": 1e-05,
+      "loss": 10.8451,
+      "step": 1
+    },
+    {
+      "epoch": 0.021164021164021163,
+      "grad_norm": 0.1444021612405777,
+      "learning_rate": 2e-05,
+      "loss": 10.8495,
+      "step": 2
+    },
+    {
+      "epoch": 0.031746031746031744,
+      "grad_norm": 0.12686049938201904,
+      "learning_rate": 3e-05,
+      "loss": 10.8472,
+      "step": 3
+    },
+    {
+      "epoch": 0.042328042328042326,
+      "grad_norm": 0.11982937902212143,
+      "learning_rate": 4e-05,
+      "loss": 10.8484,
+      "step": 4
+    },
+    {
+      "epoch": 0.05291005291005291,
+      "grad_norm": 0.1377434879541397,
+      "learning_rate": 5e-05,
+      "loss": 10.8523,
+      "step": 5
+    },
+    {
+      "epoch": 0.06349206349206349,
+      "grad_norm": 0.13242502510547638,
+      "learning_rate": 6e-05,
+      "loss": 10.843,
+      "step": 6
+    },
+    {
+      "epoch": 0.07407407407407407,
+      "grad_norm": 0.12401815503835678,
+      "learning_rate": 7e-05,
+      "loss": 10.8488,
+      "step": 7
+    },
+    {
+      "epoch": 0.08465608465608465,
+      "grad_norm": 0.13792751729488373,
+      "learning_rate": 8e-05,
+      "loss": 10.8416,
+      "step": 8
+    },
+    {
+      "epoch": 0.09523809523809523,
+      "grad_norm": 0.13162340223789215,
+      "learning_rate": 9e-05,
+      "loss": 10.842,
+      "step": 9
+    },
+    {
+      "epoch": 0.10582010582010581,
+      "grad_norm": 0.13254305720329285,
+      "learning_rate": 0.0001,
+      "loss": 10.8433,
+      "step": 10
+    },
+    {
+      "epoch": 0.1164021164021164,
+      "grad_norm": 0.12940731644630432,
+      "learning_rate": 9.999666499017257e-05,
+      "loss": 10.8411,
+      "step": 11
+    },
+    {
+      "epoch": 0.12698412698412698,
+      "grad_norm": 0.12482935190200806,
+      "learning_rate": 9.998666040558187e-05,
+      "loss": 10.8387,
+      "step": 12
+    },
+    {
+      "epoch": 0.13756613756613756,
+      "grad_norm": 0.1289866715669632,
+      "learning_rate": 9.996998758084344e-05,
+      "loss": 10.8364,
+      "step": 13
+    },
+    {
+      "epoch": 0.14814814814814814,
+      "grad_norm": 0.13676129281520844,
+      "learning_rate": 9.994664874011863e-05,
+      "loss": 10.844,
+      "step": 14
+    },
+    {
+      "epoch": 0.15873015873015872,
+      "grad_norm": 0.12447760254144669,
+      "learning_rate": 9.991664699681799e-05,
+      "loss": 10.8294,
+      "step": 15
+    },
+    {
+      "epoch": 0.1693121693121693,
+      "grad_norm": 0.13445915281772614,
+      "learning_rate": 9.987998635318585e-05,
+      "loss": 10.8285,
+      "step": 16
+    },
+    {
+      "epoch": 0.17989417989417988,
+      "grad_norm": 0.13642500340938568,
+      "learning_rate": 9.98366716997665e-05,
+      "loss": 10.8364,
+      "step": 17
+    },
+    {
+      "epoch": 0.19047619047619047,
+      "grad_norm": 0.16439010202884674,
+      "learning_rate": 9.978670881475172e-05,
+      "loss": 10.8395,
+      "step": 18
+    },
+    {
+      "epoch": 0.20105820105820105,
+      "grad_norm": 0.13272134959697723,
+      "learning_rate": 9.973010436321005e-05,
+      "loss": 10.8292,
+      "step": 19
+    },
+    {
+      "epoch": 0.21164021164021163,
+      "grad_norm": 0.13817480206489563,
+      "learning_rate": 9.966686589619751e-05,
+      "loss": 10.8342,
+      "step": 20
+    },
+    {
+      "epoch": 0.2222222222222222,
+      "grad_norm": 0.14028826355934143,
+      "learning_rate": 9.95970018497505e-05,
+      "loss": 10.8291,
+      "step": 21
+    },
+    {
+      "epoch": 0.2328042328042328,
+      "grad_norm": 0.15570807456970215,
+      "learning_rate": 9.952052154376026e-05,
+      "loss": 10.8301,
+      "step": 22
+    },
+    {
+      "epoch": 0.24338624338624337,
+      "grad_norm": 0.19065511226654053,
+      "learning_rate": 9.94374351807297e-05,
+      "loss": 10.8231,
+      "step": 23
+    },
+    {
+      "epoch": 0.25396825396825395,
+      "grad_norm": 0.1478724181652069,
+      "learning_rate": 9.934775384441228e-05,
+      "loss": 10.8329,
+      "step": 24
+    },
+    {
+      "epoch": 0.26455026455026454,
+      "grad_norm": 0.14294123649597168,
+      "learning_rate": 9.925148949833355e-05,
+      "loss": 10.8294,
+      "step": 25
+    },
+    {
+      "epoch": 0.2751322751322751,
+      "grad_norm": 0.13709235191345215,
+      "learning_rate": 9.91486549841951e-05,
+      "loss": 10.8186,
+      "step": 26
+    },
+    {
+      "epoch": 0.2857142857142857,
+      "grad_norm": 0.1486971527338028,
+      "learning_rate": 9.903926402016153e-05,
+      "loss": 10.8111,
+      "step": 27
+    },
+    {
+      "epoch": 0.2962962962962963,
+      "grad_norm": 0.1510615050792694,
+      "learning_rate": 9.892333119903046e-05,
+      "loss": 10.8215,
+      "step": 28
+    },
+    {
+      "epoch": 0.30687830687830686,
+      "grad_norm": 0.1415850967168808,
+      "learning_rate": 9.880087198628578e-05,
+      "loss": 10.8154,
+      "step": 29
+    },
+    {
+      "epoch": 0.31746031746031744,
+      "grad_norm": 0.1371394395828247,
+      "learning_rate": 9.867190271803465e-05,
+      "loss": 10.8263,
+      "step": 30
+    },
+    {
+      "epoch": 0.328042328042328,
+      "grad_norm": 0.14669376611709595,
+      "learning_rate": 9.853644059882811e-05,
+      "loss": 10.8273,
+      "step": 31
+    },
+    {
+      "epoch": 0.3386243386243386,
+      "grad_norm": 0.151437446475029,
+      "learning_rate": 9.839450369936613e-05,
+      "loss": 10.8185,
+      "step": 32
+    },
+    {
+      "epoch": 0.3492063492063492,
+      "grad_norm": 0.15013644099235535,
+      "learning_rate": 9.82461109540869e-05,
+      "loss": 10.816,
+      "step": 33
+    },
+    {
+      "epoch": 0.35978835978835977,
+      "grad_norm": 0.13472935557365417,
+      "learning_rate": 9.809128215864097e-05,
+      "loss": 10.8251,
+      "step": 34
+    },
+    {
+      "epoch": 0.37037037037037035,
+      "grad_norm": 0.14929385483264923,
+      "learning_rate": 9.793003796725048e-05,
+      "loss": 10.816,
+      "step": 35
+    },
+    {
+      "epoch": 0.38095238095238093,
+      "grad_norm": 0.13741157948970795,
+      "learning_rate": 9.7762399889954e-05,
+      "loss": 10.8155,
+      "step": 36
+    },
+    {
+      "epoch": 0.3915343915343915,
+      "grad_norm": 0.14787937700748444,
+      "learning_rate": 9.758839028973692e-05,
+      "loss": 10.8046,
+      "step": 37
+    },
+    {
+      "epoch": 0.4021164021164021,
+      "grad_norm": 0.15740682184696198,
+      "learning_rate": 9.74080323795483e-05,
+      "loss": 10.8063,
+      "step": 38
+    },
+    {
+      "epoch": 0.4126984126984127,
+      "grad_norm": 0.14199665188789368,
+      "learning_rate": 9.722135021920426e-05,
+      "loss": 10.8159,
+      "step": 39
+    },
+    {
+      "epoch": 0.42328042328042326,
+      "grad_norm": 0.1560543179512024,
+      "learning_rate": 9.702836871217839e-05,
+      "loss": 10.8004,
+      "step": 40
+    },
+    {
+      "epoch": 0.43386243386243384,
+      "grad_norm": 0.15109054744243622,
+      "learning_rate": 9.682911360227958e-05,
+      "loss": 10.8092,
+      "step": 41
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 0.14842171967029572,
+      "learning_rate": 9.662361147021779e-05,
+      "loss": 10.7969,
+      "step": 42
+    },
+    {
+      "epoch": 0.455026455026455,
+      "grad_norm": 0.15142671763896942,
+      "learning_rate": 9.641188973005826e-05,
+      "loss": 10.8044,
+      "step": 43
+    },
+    {
+      "epoch": 0.4656084656084656,
+      "grad_norm": 0.1637977957725525,
+      "learning_rate": 9.619397662556435e-05,
+      "loss": 10.8053,
+      "step": 44
+    },
+    {
+      "epoch": 0.47619047619047616,
+      "grad_norm": 0.1760113537311554,
+      "learning_rate": 9.596990122642983e-05,
+      "loss": 10.7961,
+      "step": 45
+    },
+    {
+      "epoch": 0.48677248677248675,
+      "grad_norm": 0.19414639472961426,
+      "learning_rate": 9.573969342440106e-05,
+      "loss": 10.792,
+      "step": 46
+    },
+    {
+      "epoch": 0.4973544973544973,
+      "grad_norm": 0.17807187139987946,
+      "learning_rate": 9.55033839292893e-05,
+      "loss": 10.7988,
+      "step": 47
+    },
+    {
+      "epoch": 0.5079365079365079,
+      "grad_norm": 0.15330158174037933,
+      "learning_rate": 9.526100426487409e-05,
+      "loss": 10.8007,
+      "step": 48
+    },
+    {
+      "epoch": 0.5185185185185185,
+      "grad_norm": 0.14986762404441833,
+      "learning_rate": 9.501258676469799e-05,
+      "loss": 10.8059,
+      "step": 49
+    },
+    {
+      "epoch": 0.5291005291005291,
+      "grad_norm": 0.15032872557640076,
+      "learning_rate": 9.475816456775313e-05,
+      "loss": 10.7963,
+      "step": 50
+    },
+    {
+      "epoch": 0.5396825396825397,
+      "grad_norm": 0.15320739150047302,
+      "learning_rate": 9.44977716140606e-05,
+      "loss": 10.7961,
+      "step": 51
+    },
+    {
+      "epoch": 0.5502645502645502,
+      "grad_norm": 0.1596822887659073,
+      "learning_rate": 9.423144264014277e-05,
+      "loss": 10.7898,
+      "step": 52
+    },
+    {
+      "epoch": 0.5608465608465608,
+      "grad_norm": 0.15990599989891052,
+      "learning_rate": 9.395921317438949e-05,
+      "loss": 10.7878,
+      "step": 53
+    },
+    {
+      "epoch": 0.5714285714285714,
+      "grad_norm": 0.1632925122976303,
+      "learning_rate": 9.368111953231848e-05,
+      "loss": 10.7937,
+      "step": 54
+    },
+    {
+      "epoch": 0.582010582010582,
+      "grad_norm": 0.15481893718242645,
+      "learning_rate": 9.339719881173093e-05,
+      "loss": 10.7851,
+      "step": 55
+    },
+    {
+      "epoch": 0.5925925925925926,
+      "grad_norm": 0.13583026826381683,
+      "learning_rate": 9.310748888776254e-05,
+      "loss": 10.7921,
+      "step": 56
+    },
+    {
+      "epoch": 0.6031746031746031,
+      "grad_norm": 0.1494726985692978,
+      "learning_rate": 9.281202840783108e-05,
+      "loss": 10.7896,
+      "step": 57
+    },
+    {
+      "epoch": 0.6137566137566137,
+      "grad_norm": 0.1633342057466507,
+      "learning_rate": 9.251085678648072e-05,
+      "loss": 10.7761,
+      "step": 58
+    },
+    {
+      "epoch": 0.6243386243386243,
+      "grad_norm": 0.1558162122964859,
+      "learning_rate": 9.220401420012411e-05,
+      "loss": 10.7774,
+      "step": 59
+    },
+    {
+      "epoch": 0.6349206349206349,
+      "grad_norm": 0.15188248455524445,
+      "learning_rate": 9.189154158168292e-05,
+      "loss": 10.7863,
+      "step": 60
+    },
+    {
+      "epoch": 0.6455026455026455,
+      "grad_norm": 0.1540272831916809,
+      "learning_rate": 9.157348061512727e-05,
+      "loss": 10.7816,
+      "step": 61
+    },
+    {
+      "epoch": 0.656084656084656,
+      "grad_norm": 0.14700107276439667,
+      "learning_rate": 9.124987372991511e-05,
+      "loss": 10.7696,
+      "step": 62
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "grad_norm": 0.16768917441368103,
+      "learning_rate": 9.092076409533218e-05,
+      "loss": 10.7657,
+      "step": 63
+    },
+    {
+      "epoch": 0.6772486772486772,
+      "grad_norm": 0.1637629121541977,
+      "learning_rate": 9.058619561473307e-05,
+      "loss": 10.7605,
+      "step": 64
+    },
+    {
+      "epoch": 0.6878306878306878,
+      "grad_norm": 0.1513640135526657,
+      "learning_rate": 9.02462129196846e-05,
+      "loss": 10.7699,
+      "step": 65
+    },
+    {
+      "epoch": 0.6984126984126984,
+      "grad_norm": 0.1465969681739807,
+      "learning_rate": 8.9900861364012e-05,
+      "loss": 10.771,
+      "step": 66
+    },
+    {
+      "epoch": 0.708994708994709,
+      "grad_norm": 0.1486431509256363,
+      "learning_rate": 8.955018701774846e-05,
+      "loss": 10.7742,
+      "step": 67
+    },
+    {
+      "epoch": 0.7195767195767195,
+      "grad_norm": 0.17778757214546204,
+      "learning_rate": 8.919423666098969e-05,
+      "loss": 10.7613,
+      "step": 68
+    },
+    {
+      "epoch": 0.7301587301587301,
+      "grad_norm": 0.172640860080719,
+      "learning_rate": 8.883305777765318e-05,
+      "loss": 10.7576,
+      "step": 69
+    },
+    {
+      "epoch": 0.7407407407407407,
+      "grad_norm": 0.14984062314033508,
+      "learning_rate": 8.846669854914396e-05,
+      "loss": 10.7717,
+      "step": 70
+    },
+    {
+      "epoch": 0.7513227513227513,
+      "grad_norm": 0.15748809278011322,
+      "learning_rate": 8.809520784792709e-05,
+      "loss": 10.7638,
+      "step": 71
+    },
+    {
+      "epoch": 0.7619047619047619,
+      "grad_norm": 0.12908557057380676,
+      "learning_rate": 8.77186352310082e-05,
+      "loss": 10.7669,
+      "step": 72
+    },
+    {
+      "epoch": 0.7724867724867724,
+      "grad_norm": 0.14902107417583466,
+      "learning_rate": 8.733703093332237e-05,
+      "loss": 10.7616,
+      "step": 73
+    },
+    {
+      "epoch": 0.783068783068783,
+      "grad_norm": 0.13321872055530548,
+      "learning_rate": 8.695044586103296e-05,
+      "loss": 10.7746,
+      "step": 74
+    },
+    {
+      "epoch": 0.7936507936507936,
+      "grad_norm": 0.12217468023300171,
+      "learning_rate": 8.655893158474055e-05,
+      "loss": 10.7665,
+      "step": 75
+    },
+    {
+      "epoch": 0.8042328042328042,
+      "grad_norm": 0.138421893119812,
+      "learning_rate": 8.616254033260351e-05,
+      "loss": 10.7685,
+      "step": 76
+    },
+    {
+      "epoch": 0.8148148148148148,
+      "grad_norm": 0.15046954154968262,
+      "learning_rate": 8.576132498337068e-05,
+      "loss": 10.7556,
+      "step": 77
+    },
+    {
+      "epoch": 0.8253968253968254,
+      "grad_norm": 0.13696914911270142,
+      "learning_rate": 8.535533905932738e-05,
+      "loss": 10.7609,
+      "step": 78
+    },
+    {
+      "epoch": 0.8359788359788359,
+      "grad_norm": 0.1344854235649109,
+      "learning_rate": 8.494463671915547e-05,
+      "loss": 10.7543,
+      "step": 79
+    },
+    {
+      "epoch": 0.8465608465608465,
+      "grad_norm": 0.13561226427555084,
+      "learning_rate": 8.452927275070858e-05,
+      "loss": 10.7548,
+      "step": 80
+    },
+    {
+      "epoch": 0.8571428571428571,
+      "grad_norm": 0.13678033649921417,
+      "learning_rate": 8.410930256370336e-05,
+      "loss": 10.7564,
+      "step": 81
+    },
+    {
+      "epoch": 0.8677248677248677,
+      "grad_norm": 0.12919801473617554,
+      "learning_rate": 8.368478218232787e-05,
+      "loss": 10.7493,
+      "step": 82
+    },
+    {
+      "epoch": 0.8783068783068783,
+      "grad_norm": 0.108885258436203,
+      "learning_rate": 8.325576823776784e-05,
+      "loss": 10.7635,
+      "step": 83
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 0.1373911052942276,
+      "learning_rate": 8.282231796065214e-05,
+      "loss": 10.7532,
+      "step": 84
+    },
+    {
+      "epoch": 0.8994708994708994,
+      "grad_norm": 0.1275387555360794,
+      "learning_rate": 8.23844891734181e-05,
+      "loss": 10.7435,
+      "step": 85
+    },
+    {
+      "epoch": 0.91005291005291,
+      "grad_norm": 0.13758745789527893,
+      "learning_rate": 8.194234028259806e-05,
+      "loss": 10.7407,
+      "step": 86
+    },
+    {
+      "epoch": 0.9206349206349206,
+      "grad_norm": 0.12855514883995056,
+      "learning_rate": 8.149593027102789e-05,
+      "loss": 10.7406,
+      "step": 87
+    },
+    {
+      "epoch": 0.9312169312169312,
+      "grad_norm": 0.1377517431974411,
+      "learning_rate": 8.104531868997858e-05,
+      "loss": 10.7447,
+      "step": 88
+    },
+    {
+      "epoch": 0.9417989417989417,
+      "grad_norm": 0.1041102483868599,
+      "learning_rate": 8.059056565121216e-05,
+      "loss": 10.7487,
+      "step": 89
+    },
+    {
+      "epoch": 0.9523809523809523,
+      "grad_norm": 0.12848980724811554,
+      "learning_rate": 8.013173181896283e-05,
+      "loss": 10.7392,
+      "step": 90
+    },
+    {
+      "epoch": 0.9629629629629629,
+      "grad_norm": 0.15677225589752197,
+      "learning_rate": 7.966887840184411e-05,
+      "loss": 10.7318,
+      "step": 91
+    },
+    {
+      "epoch": 0.9735449735449735,
+      "grad_norm": 0.15663453936576843,
+      "learning_rate": 7.920206714468383e-05,
+      "loss": 10.7357,
+      "step": 92
+    },
+    {
+      "epoch": 0.9841269841269841,
+      "grad_norm": 0.1426771879196167,
+      "learning_rate": 7.87313603202872e-05,
+      "loss": 10.7456,
+      "step": 93
+    },
+    {
+      "epoch": 0.9947089947089947,
+      "grad_norm": 0.11514159291982651,
+      "learning_rate": 7.82568207211296e-05,
+      "loss": 10.7402,
+      "step": 94
+    },
+    {
+      "epoch": 0.9947089947089947,
+      "eval_loss": 10.736422538757324,
+      "eval_runtime": 0.4445,
+      "eval_samples_per_second": 359.96,
+      "eval_steps_per_second": 89.99,
+      "step": 94
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 282,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 63331911598080.0,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b04aaa5d7de554b6ab56a819e77b932045a3e86ebcb996e3e05eafc230b283b0
+size 6776