mohityadav/mental-health-advisor-gpt

README.md

@@ -0,0 +1,72 @@
+---
+license: apache-2.0
+library_name: peft
+tags:
+- generated_from_trainer
+base_model: TheBloke/Mistral-7B-Instruct-v0.2-GPTQ
+model-index:
+- name: working
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# working
+
+This model is a fine-tuned version of [TheBloke/Mistral-7B-Instruct-v0.2-GPTQ](https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GPTQ) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.2984
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0002
+- train_batch_size: 4
+- eval_batch_size: 4
+- seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 16
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 2
+- num_epochs: 10
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 3.1139        | 0.9412 | 12   | 2.3084          |
+| 1.9191        | 1.9608 | 25   | 1.7055          |
+| 1.6083        | 2.9804 | 38   | 1.5741          |
+| 1.496         | 4.0    | 51   | 1.4695          |
+| 1.5309        | 4.9412 | 63   | 1.4038          |
+| 1.3627        | 5.9608 | 76   | 1.3573          |
+| 1.3111        | 6.9804 | 89   | 1.3281          |
+| 1.2995        | 8.0    | 102  | 1.3092          |
+| 1.3866        | 8.9412 | 114  | 1.2998          |
+| 1.18          | 9.4118 | 120  | 1.2984          |
+
+
+### Framework versions
+
+- PEFT 0.11.1
+- Transformers 4.41.2
+- Pytorch 2.1.2
+- Datasets 2.19.2
+- Tokenizers 0.19.1

adapter_config.json

@@ -0,0 +1,28 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": null,
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5cf94b36f046daba8d3cb473ba0348657a9ab8c56dbc2fb787002a64c357e7fa
+size 8401408

runs/Jun26_06-58-21_adc95cf38b20/events.out.tfevents.1719385345.adc95cf38b20.34.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bca8edc01aa31a9e70f9815dbca96d3ac87c4247cc92a4349a8cb745bdc575e3
+size 5481

runs/Jun26_06-58-21_adc95cf38b20/events.out.tfevents.1719385423.adc95cf38b20.34.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b0060337b7a08396a73032f8dc408d21d457d6d92945573baaee52a0267a943
+size 5481

runs/Jun26_07-07-26_adc95cf38b20/events.out.tfevents.1719385648.adc95cf38b20.34.2

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dcba8cc9427be248c5ef089831420e28a59559d37e4d7b1fc100ffbe95dd1d22
+size 16267

runs/Jun26_07-16-15_adc95cf38b20/events.out.tfevents.1719386177.adc95cf38b20.34.3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:71bffc05745c6f8799b41a6de611947dd2d1daf0eef79e856f120996c9f55035
+size 5481

runs/Jun26_07-19-05_adc95cf38b20/events.out.tfevents.1719386347.adc95cf38b20.34.4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e7083d5109e346d61a1145b6a1fdbd73bcf0549de48bd847c6788e0072883c2
+size 10559

runs/Jun26_07-19-05_adc95cf38b20/events.out.tfevents.1719391099.adc95cf38b20.34.5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1664a1d1de74046bcd26b750d0bb5b1b704fb72256c794a139ae46f340dc4c85
+size 354

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be11ac523aaa8727f2185f783bf03400ad8ca8ff6208855f6a032bd7b8a919b3
+size 5112

wandb/debug.log

@@ -0,0 +1,246 @@
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Current SDK version is 0.17.0
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Configure stats pid to 34
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Loading settings from /root/.config/wandb/settings
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Loading settings from /kaggle/working/wandb/settings
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Loading settings from environment variables: {}
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Applying setup settings: {'_disable_service': False}
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Inferring run settings from compute environment: {'program': '<python with no main file>'}
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Applying login settings: {}
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Applying login settings: {}
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Applying login settings: {}
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Applying login settings: {}
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Applying login settings: {'api_key': '***REDACTED***'}
+2024-06-26 07:16:28,967 ERROR   MainThread:34 [wandb_setup.py:_flush():78] error in wandb.init()
+Traceback (most recent call last):
+  File "/opt/conda/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3553, in run_code
+    exec(code_obj, self.user_global_ns, self.user_ns)
+  File "/tmp/ipykernel_34/2014566126.py", line 10, in <module>
+    trainer.train()
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 1885, in train
+    return inner_training_loop(
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 2147, in _inner_training_loop
+    self.control = self.callback_handler.on_train_begin(args, self.state, self.control)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer_callback.py", line 454, in on_train_begin
+    return self.call_event("on_train_begin", args, state, control)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer_callback.py", line 498, in call_event
+    result = getattr(callback, event)(
+  File "/opt/conda/lib/python3.10/site-packages/transformers/integrations/integration_utils.py", line 773, in on_train_begin
+    self.setup(args, state, model, **kwargs)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/integrations/integration_utils.py", line 746, in setup
+    self._wandb.init(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1178, in init
+    wandb._sentry.reraise(e)
+  File "/opt/conda/lib/python3.10/site-packages/wandb/analytics/sentry.py", line 155, in reraise
+    raise exc.with_traceback(sys.exc_info()[2])
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1163, in init
+    wi.setup(kwargs)
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 300, in setup
+    wandb_login._login(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 334, in _login
+    wlogin.prompt_api_key()
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 256, in prompt_api_key
+    key, status = self._prompt_api_key()
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 236, in _prompt_api_key
+    key = apikey.prompt_api_key(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/lib/apikey.py", line 151, in prompt_api_key
+    key = input_callback(api_ask).strip()
+  File "/opt/conda/lib/python3.10/site-packages/click/termui.py", line 164, in prompt
+    value = prompt_func(prompt)
+  File "/opt/conda/lib/python3.10/site-packages/click/termui.py", line 147, in prompt_func
+    raise Abort() from None
+click.exceptions.Abort
+2024-06-26 07:16:28,968 ERROR   MainThread:34 [wandb_setup.py:_flush():78] error in wandb.init()
+Traceback (most recent call last):
+  File "/opt/conda/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3553, in run_code
+    exec(code_obj, self.user_global_ns, self.user_ns)
+  File "/tmp/ipykernel_34/4032920361.py", line 1, in <module>
+    trainer.train()
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 1885, in train
+    return inner_training_loop(
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 2147, in _inner_training_loop
+    self.control = self.callback_handler.on_train_begin(args, self.state, self.control)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer_callback.py", line 454, in on_train_begin
+    return self.call_event("on_train_begin", args, state, control)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer_callback.py", line 498, in call_event
+    result = getattr(callback, event)(
+  File "/opt/conda/lib/python3.10/site-packages/transformers/integrations/integration_utils.py", line 773, in on_train_begin
+    self.setup(args, state, model, **kwargs)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/integrations/integration_utils.py", line 746, in setup
+    self._wandb.init(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1178, in init
+    wandb._sentry.reraise(e)
+  File "/opt/conda/lib/python3.10/site-packages/wandb/analytics/sentry.py", line 155, in reraise
+    raise exc.with_traceback(sys.exc_info()[2])
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1163, in init
+    wi.setup(kwargs)
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 300, in setup
+    wandb_login._login(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 334, in _login
+    wlogin.prompt_api_key()
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 256, in prompt_api_key
+    key, status = self._prompt_api_key()
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 236, in _prompt_api_key
+    key = apikey.prompt_api_key(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/lib/apikey.py", line 151, in prompt_api_key
+    key = input_callback(api_ask).strip()
+  File "/opt/conda/lib/python3.10/site-packages/click/termui.py", line 164, in prompt
+    value = prompt_func(prompt)
+  File "/opt/conda/lib/python3.10/site-packages/click/termui.py", line 147, in prompt_func
+    raise Abort() from None
+click.exceptions.Abort
+2024-06-26 07:16:28,968 ERROR   MainThread:34 [wandb_setup.py:_flush():78] error in wandb.init()
+Traceback (most recent call last):
+  File "/opt/conda/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3553, in run_code
+    exec(code_obj, self.user_global_ns, self.user_ns)
+  File "/tmp/ipykernel_34/4032920361.py", line 1, in <module>
+    trainer.train()
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 1885, in train
+    return inner_training_loop(
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 2147, in _inner_training_loop
+    self.control = self.callback_handler.on_train_begin(args, self.state, self.control)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer_callback.py", line 454, in on_train_begin
+    return self.call_event("on_train_begin", args, state, control)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer_callback.py", line 498, in call_event
+    result = getattr(callback, event)(
+  File "/opt/conda/lib/python3.10/site-packages/transformers/integrations/integration_utils.py", line 773, in on_train_begin
+    self.setup(args, state, model, **kwargs)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/integrations/integration_utils.py", line 746, in setup
+    self._wandb.init(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1178, in init
+    wandb._sentry.reraise(e)
+  File "/opt/conda/lib/python3.10/site-packages/wandb/analytics/sentry.py", line 155, in reraise
+    raise exc.with_traceback(sys.exc_info()[2])
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1163, in init
+    wi.setup(kwargs)
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 300, in setup
+    wandb_login._login(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 334, in _login
+    wlogin.prompt_api_key()
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 256, in prompt_api_key
+    key, status = self._prompt_api_key()
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 236, in _prompt_api_key
+    key = apikey.prompt_api_key(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/lib/apikey.py", line 151, in prompt_api_key
+    key = input_callback(api_ask).strip()
+  File "/opt/conda/lib/python3.10/site-packages/click/termui.py", line 164, in prompt
+    value = prompt_func(prompt)
+  File "/opt/conda/lib/python3.10/site-packages/click/termui.py", line 147, in prompt_func
+    raise Abort() from None
+click.exceptions.Abort
+2024-06-26 07:16:28,969 INFO    MainThread:34 [wandb_init.py:_log_setup():520] Logging user logs to /kaggle/working/wandb/run-20240626_071628-smnm2aje/logs/debug.log
+2024-06-26 07:16:28,969 INFO    MainThread:34 [wandb_init.py:_log_setup():521] Logging internal logs to /kaggle/working/wandb/run-20240626_071628-smnm2aje/logs/debug-internal.log
+2024-06-26 07:16:28,969 INFO    MainThread:34 [wandb_init.py:_jupyter_setup():466] configuring jupyter hooks <wandb.sdk.wandb_init._WandbInit object at 0x78720aaa5b40>
+2024-06-26 07:16:28,969 INFO    MainThread:34 [wandb_init.py:init():560] calling init triggers
+2024-06-26 07:16:28,969 INFO    MainThread:34 [wandb_init.py:init():567] wandb.init called with sweep_config: {}
+config: {}
+2024-06-26 07:16:28,969 INFO    MainThread:34 [wandb_init.py:init():610] starting backend
+2024-06-26 07:16:28,969 INFO    MainThread:34 [wandb_init.py:init():614] setting up manager
+2024-06-26 07:16:28,971 INFO    MainThread:34 [backend.py:_multiprocessing_setup():105] multiprocessing start_methods=fork,spawn,forkserver, using: spawn
+2024-06-26 07:16:28,977 INFO    MainThread:34 [wandb_init.py:init():622] backend started and connected
+2024-06-26 07:16:28,989 INFO    MainThread:34 [wandb_run.py:_label_probe_notebook():1328] probe notebook
+2024-06-26 07:16:29,303 INFO    MainThread:34 [wandb_init.py:init():711] updated telemetry
+2024-06-26 07:16:29,306 INFO    MainThread:34 [wandb_init.py:init():744] communicating run to backend with 90.0 second timeout
+2024-06-26 07:16:29,429 INFO    MainThread:34 [wandb_run.py:_on_init():2396] communicating current version
+2024-06-26 07:16:29,507 INFO    MainThread:34 [wandb_run.py:_on_init():2405] got version response upgrade_message: "wandb version 0.17.3 is available!  To upgrade, please run:\n $ pip install wandb --upgrade"
+
+2024-06-26 07:16:29,507 INFO    MainThread:34 [wandb_init.py:init():795] starting run threads in backend
+2024-06-26 07:16:45,662 INFO    MainThread:34 [wandb_run.py:_console_start():2374] atexit reg
+2024-06-26 07:16:45,662 INFO    MainThread:34 [wandb_run.py:_redirect():2229] redirect: wrap_raw
+2024-06-26 07:16:45,662 INFO    MainThread:34 [wandb_run.py:_redirect():2294] Wrapping output streams.
+2024-06-26 07:16:45,663 INFO    MainThread:34 [wandb_run.py:_redirect():2319] Redirects installed.
+2024-06-26 07:16:45,672 INFO    MainThread:34 [wandb_init.py:init():838] run started, returning control to user process
+2024-06-26 07:16:45,678 INFO    MainThread:34 [wandb_run.py:_config_callback():1376] config_cb None None {'vocab_size': 32000, 'max_position_embeddings': 32768, 'hidden_size': 4096, 'intermediate_size': 14336, 'num_hidden_layers': 32, 'num_attention_heads': 32, 'sliding_window': None, 'num_key_value_heads': 8, 'hidden_act': 'silu', 'initializer_range': 0.02, 'rms_norm_eps': 1e-05, 'use_cache': False, 'rope_theta': 1000000.0, 'attention_dropout': 0.0, 'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': 'bfloat16', 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': False, 'chunk_size_feed_forward': 0, 'is_encoder_decoder': False, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 20, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 1, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': None, 'num_return_sequences': 1, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': None, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': ['MistralForCausalLM'], 'finetuning_task': None, 'id2label': {0: 'LABEL_0', 1: 'LABEL_1'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1}, 'tokenizer_class': None, 'prefix': None, 'bos_token_id': 1, 'pad_token_id': 0, 'eos_token_id': 2, 'sep_token_id': None, 'decoder_start_token_id': None, 'task_specific_params': None, 'problem_type': None, '_name_or_path': 'TheBloke/Mistral-7B-Instruct-v0.2-GPTQ', 'transformers_version': '4.41.2', 'model_type': 'mistral', 'pretraining_tp': 1, 'quantization_config': {'quant_method': 'QuantizationMethod.GPTQ', 'bits': 4, 'tokenizer': None, 'dataset': None, 'group_size': 128, 'damp_percent': 0.1, 'desc_act': True, 'sym': True, 'true_sequential': True, 'use_cuda_fp16': False, 'model_seqlen': None, 'block_name_to_quantize': None, 'module_name_preceding_first_block': None, 'batch_size': 1, 'pad_token_id': None, 'use_exllama': True, 'max_input_length': None, 'exllama_config': {'version': 'ExllamaVersion.ONE'}, 'cache_block_outputs': True, 'modules_in_block_to_quantize': None}, 'output_dir': '/kaggle/working/', 'overwrite_output_dir': False, 'do_train': False, 'do_eval': True, 'do_predict': False, 'eval_strategy': 'epoch', 'prediction_loss_only': False, 'per_device_train_batch_size': 4, 'per_device_eval_batch_size': 4, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 4, 'eval_accumulation_steps': None, 'eval_delay': 0, 'learning_rate': 0.0002, 'weight_decay': 0.01, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam_epsilon': 1e-08, 'max_grad_norm': 1.0, 'num_train_epochs': 10, 'max_steps': -1, 'lr_scheduler_type': 'linear', 'lr_scheduler_kwargs': {}, 'warmup_ratio': 0.0, 'warmup_steps': 2, 'log_level': 'passive', 'log_level_replica': 'warning', 'log_on_each_node': True, 'logging_dir': '/kaggle/working/runs/Jun26_07-16-15_adc95cf38b20', 'logging_strategy': 'epoch', 'logging_first_step': False, 'logging_steps': 500, 'logging_nan_inf_filter': True, 'save_strategy': 'epoch', 'save_steps': 500, 'save_total_limit': None, 'save_safetensors': True, 'save_on_each_node': False, 'save_only_model': False, 'restore_callback_states_from_checkpoint': False, 'no_cuda': False, 'use_cpu': False, 'use_mps_device': False, 'seed': 42, 'data_seed': None, 'jit_mode_eval': False, 'use_ipex': False, 'bf16': False, 'fp16': True, 'fp16_opt_level': 'O1', 'half_precision_backend': 'auto', 'bf16_full_eval': False, 'fp16_full_eval': False, 'tf32': None, 'local_rank': 0, 'ddp_backend': None, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': None, 'dataloader_num_workers': 0, 'dataloader_prefetch_factor': None, 'past_index': -1, 'run_name': '/kaggle/working/', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': True, 'metric_for_best_model': 'loss', 'greater_is_better': False, 'ignore_data_skip': False, 'fsdp': [], 'fsdp_min_num_params': 0, 'fsdp_config': {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, 'fsdp_transformer_layer_cls_to_wrap': None, 'accelerator_config': {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}, 'deepspeed': None, 'label_smoothing_factor': 0.0, 'optim': 'paged_adamw_8bit', 'optim_args': None, 'adafactor': False, 'group_by_length': False, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'ddp_bucket_cap_mb': None, 'ddp_broadcast_buffers': None, 'dataloader_pin_memory': True, 'dataloader_persistent_workers': False, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': False, 'resume_from_checkpoint': None, 'hub_model_id': None, 'hub_strategy': 'every_save', 'hub_token': '<HUB_TOKEN>', 'hub_private_repo': False, 'hub_always_push': False, 'gradient_checkpointing': False, 'gradient_checkpointing_kwargs': None, 'include_inputs_for_metrics': False, 'eval_do_concat_batches': True, 'fp16_backend': 'auto', 'evaluation_strategy': None, 'push_to_hub_model_id': None, 'push_to_hub_organization': None, 'push_to_hub_token': '<PUSH_TO_HUB_TOKEN>', 'mp_parameters': '', 'auto_find_batch_size': False, 'full_determinism': False, 'torchdynamo': None, 'ray_scope': 'last', 'ddp_timeout': 1800, 'torch_compile': False, 'torch_compile_backend': None, 'torch_compile_mode': None, 'dispatch_batches': None, 'split_batches': None, 'include_tokens_per_second': False, 'include_num_input_tokens_seen': False, 'neftune_noise_alpha': None, 'optim_target_modules': None, 'batch_eval_metrics': False}
+2024-06-26 07:18:26,851 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:26,851 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:50,892 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:50,915 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:50,915 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:51,909 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:51,913 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:51,913 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:52,853 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:52,855 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:52,855 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:53,667 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:53,670 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:53,670 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:54,691 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:54,756 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:54,756 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:55,617 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:55,682 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:55,682 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:56,512 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:56,513 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:56,513 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:57,999 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:58,001 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:58,001 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:58,997 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:59,009 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:59,009 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:19:00,090 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:19:00,092 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:19:00,092 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:19:01,722 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:19:01,723 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:19:01,723 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:19:03,098 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:19:03,117 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:19:03,117 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:19:04,117 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:19:04,212 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:19:04,212 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:19:05,065 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:19:05,097 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:19:05,097 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:19:06,209 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:19:06,220 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:19:06,220 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:19:07,300 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:19:07,961 INFO    MainThread:34 [wandb_run.py:_config_callback():1376] config_cb None None {'vocab_size': 32000, 'max_position_embeddings': 32768, 'hidden_size': 4096, 'intermediate_size': 14336, 'num_hidden_layers': 32, 'num_attention_heads': 32, 'sliding_window': None, 'num_key_value_heads': 8, 'hidden_act': 'silu', 'initializer_range': 0.02, 'rms_norm_eps': 1e-05, 'use_cache': False, 'rope_theta': 1000000.0, 'attention_dropout': 0.0, 'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': 'bfloat16', 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': False, 'chunk_size_feed_forward': 0, 'is_encoder_decoder': False, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 20, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 1, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': None, 'num_return_sequences': 1, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': None, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': ['MistralForCausalLM'], 'finetuning_task': None, 'id2label': {0: 'LABEL_0', 1: 'LABEL_1'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1}, 'tokenizer_class': None, 'prefix': None, 'bos_token_id': 1, 'pad_token_id': 0, 'eos_token_id': 2, 'sep_token_id': None, 'decoder_start_token_id': None, 'task_specific_params': None, 'problem_type': None, '_name_or_path': 'TheBloke/Mistral-7B-Instruct-v0.2-GPTQ', 'transformers_version': '4.41.2', 'model_type': 'mistral', 'pretraining_tp': 1, 'quantization_config': {'quant_method': 'QuantizationMethod.GPTQ', 'bits': 4, 'tokenizer': None, 'dataset': None, 'group_size': 128, 'damp_percent': 0.1, 'desc_act': True, 'sym': True, 'true_sequential': True, 'use_cuda_fp16': False, 'model_seqlen': None, 'block_name_to_quantize': None, 'module_name_preceding_first_block': None, 'batch_size': 1, 'pad_token_id': None, 'use_exllama': True, 'max_input_length': None, 'exllama_config': {'version': 'ExllamaVersion.ONE'}, 'cache_block_outputs': True, 'modules_in_block_to_quantize': None}, 'output_dir': '/kaggle/working/', 'overwrite_output_dir': False, 'do_train': False, 'do_eval': True, 'do_predict': False, 'eval_strategy': 'epoch', 'prediction_loss_only': False, 'per_device_train_batch_size': 4, 'per_device_eval_batch_size': 4, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 4, 'eval_accumulation_steps': None, 'eval_delay': 0, 'learning_rate': 0.0002, 'weight_decay': 0.01, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam_epsilon': 1e-08, 'max_grad_norm': 1.0, 'num_train_epochs': 10, 'max_steps': -1, 'lr_scheduler_type': 'linear', 'lr_scheduler_kwargs': {}, 'warmup_ratio': 0.0, 'warmup_steps': 2, 'log_level': 'passive', 'log_level_replica': 'warning', 'log_on_each_node': True, 'logging_dir': '/kaggle/working/runs/Jun26_07-19-05_adc95cf38b20', 'logging_strategy': 'epoch', 'logging_first_step': False, 'logging_steps': 500, 'logging_nan_inf_filter': True, 'save_strategy': 'epoch', 'save_steps': 500, 'save_total_limit': None, 'save_safetensors': True, 'save_on_each_node': False, 'save_only_model': False, 'restore_callback_states_from_checkpoint': False, 'no_cuda': False, 'use_cpu': False, 'use_mps_device': False, 'seed': 42, 'data_seed': None, 'jit_mode_eval': False, 'use_ipex': False, 'bf16': False, 'fp16': True, 'fp16_opt_level': 'O1', 'half_precision_backend': 'auto', 'bf16_full_eval': False, 'fp16_full_eval': False, 'tf32': None, 'local_rank': 0, 'ddp_backend': None, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': None, 'dataloader_num_workers': 0, 'dataloader_prefetch_factor': None, 'past_index': -1, 'run_name': '/kaggle/working/', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': True, 'metric_for_best_model': 'loss', 'greater_is_better': False, 'ignore_data_skip': False, 'fsdp': [], 'fsdp_min_num_params': 0, 'fsdp_config': {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, 'fsdp_transformer_layer_cls_to_wrap': None, 'accelerator_config': {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}, 'deepspeed': None, 'label_smoothing_factor': 0.0, 'optim': 'paged_adamw_8bit', 'optim_args': None, 'adafactor': False, 'group_by_length': False, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'ddp_bucket_cap_mb': None, 'ddp_broadcast_buffers': None, 'dataloader_pin_memory': True, 'dataloader_persistent_workers': False, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': False, 'resume_from_checkpoint': None, 'hub_model_id': None, 'hub_strategy': 'every_save', 'hub_token': '<HUB_TOKEN>', 'hub_private_repo': False, 'hub_always_push': False, 'gradient_checkpointing': False, 'gradient_checkpointing_kwargs': None, 'include_inputs_for_metrics': False, 'eval_do_concat_batches': True, 'fp16_backend': 'auto', 'evaluation_strategy': None, 'push_to_hub_model_id': None, 'push_to_hub_organization': None, 'push_to_hub_token': '<PUSH_TO_HUB_TOKEN>', 'mp_parameters': '', 'auto_find_batch_size': False, 'full_determinism': False, 'torchdynamo': None, 'ray_scope': 'last', 'ddp_timeout': 1800, 'torch_compile': False, 'torch_compile_backend': None, 'torch_compile_mode': None, 'dispatch_batches': None, 'split_batches': None, 'include_tokens_per_second': False, 'include_num_input_tokens_seen': False, 'neftune_noise_alpha': None, 'optim_target_modules': None, 'batch_eval_metrics': False}
+2024-06-26 08:35:48,386 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:35:48,387 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:36:11,677 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:38:19,575 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:38:19,575 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:39:55,100 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:39:55,103 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:39:55,103 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:40:32,017 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:43:02,593 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:43:02,594 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:45:10,959 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:45:10,963 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:45:10,963 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:45:13,384 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:47:43,994 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:47:43,994 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:49:58,760 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:53:33,771 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:53:33,771 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:54:32,055 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:54:32,059 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:54:32,059 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:54:32,754 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:58:07,756 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:58:07,756 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 09:12:46,632 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 09:12:46,669 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 09:12:46,669 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 09:16:22,037 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 09:16:22,038 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 09:16:22,038 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 09:16:35,228 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 09:16:36,438 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 09:16:36,438 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 09:18:25,016 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 09:18:26,213 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 09:18:26,214 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 09:21:21,819 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 09:21:21,842 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 09:21:21,842 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 09:21:36,632 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 09:21:36,633 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 09:21:36,634 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 09:21:37,142 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend

wandb/run-20240626_071628-smnm2aje/files/config.yaml

@@ -0,0 +1,724 @@
+wandb_version: 1
+
+_wandb:
+  desc: null
+  value:
+    python_version: 3.10.13
+    cli_version: 0.17.0
+    framework: huggingface
+    huggingface_version: 4.41.2
+    is_jupyter_run: true
+    is_kaggle_kernel: true
+    start_time: 1719386188
+    t:
+      1:
+      - 1
+      - 2
+      - 3
+      - 5
+      - 11
+      - 12
+      - 49
+      - 51
+      - 53
+      - 55
+      - 71
+      - 98
+      - 99
+      - 105
+      2:
+      - 1
+      - 2
+      - 3
+      - 5
+      - 11
+      - 12
+      - 49
+      - 51
+      - 53
+      - 55
+      - 71
+      - 98
+      - 99
+      - 105
+      3:
+      - 7
+      - 13
+      - 23
+      - 62
+      - 66
+      4: 3.10.13
+      5: 0.17.0
+      6: 4.41.2
+      8:
+      - 1
+      - 2
+      - 5
+      9:
+        1: transformers_trainer
+      13: linux-x86_64
+    m:
+    - 1: train/global_step
+      6:
+      - 3
+    - 1: train/loss
+      5: 1
+      6:
+      - 1
+    - 1: train/grad_norm
+      5: 1
+      6:
+      - 1
+    - 1: train/learning_rate
+      5: 1
+      6:
+      - 1
+    - 1: train/epoch
+      5: 1
+      6:
+      - 1
+    - 1: eval/loss
+      5: 1
+      6:
+      - 1
+    - 1: eval/runtime
+      5: 1
+      6:
+      - 1
+    - 1: eval/samples_per_second
+      5: 1
+      6:
+      - 1
+    - 1: eval/steps_per_second
+      5: 1
+      6:
+      - 1
+vocab_size:
+  desc: null
+  value: 32000
+max_position_embeddings:
+  desc: null
+  value: 32768
+hidden_size:
+  desc: null
+  value: 4096
+intermediate_size:
+  desc: null
+  value: 14336
+num_hidden_layers:
+  desc: null
+  value: 32
+num_attention_heads:
+  desc: null
+  value: 32
+sliding_window:
+  desc: null
+  value: null
+num_key_value_heads:
+  desc: null
+  value: 8
+hidden_act:
+  desc: null
+  value: silu
+initializer_range:
+  desc: null
+  value: 0.02
+rms_norm_eps:
+  desc: null
+  value: 1.0e-05
+use_cache:
+  desc: null
+  value: false
+rope_theta:
+  desc: null
+  value: 1000000.0
+attention_dropout:
+  desc: null
+  value: 0.0
+return_dict:
+  desc: null
+  value: true
+output_hidden_states:
+  desc: null
+  value: false
+output_attentions:
+  desc: null
+  value: false
+torchscript:
+  desc: null
+  value: false
+torch_dtype:
+  desc: null
+  value: bfloat16
+use_bfloat16:
+  desc: null
+  value: false
+tf_legacy_loss:
+  desc: null
+  value: false
+pruned_heads:
+  desc: null
+  value: {}
+tie_word_embeddings:
+  desc: null
+  value: false
+chunk_size_feed_forward:
+  desc: null
+  value: 0
+is_encoder_decoder:
+  desc: null
+  value: false
+is_decoder:
+  desc: null
+  value: false
+cross_attention_hidden_size:
+  desc: null
+  value: null
+add_cross_attention:
+  desc: null
+  value: false
+tie_encoder_decoder:
+  desc: null
+  value: false
+max_length:
+  desc: null
+  value: 20
+min_length:
+  desc: null
+  value: 0
+do_sample:
+  desc: null
+  value: false
+early_stopping:
+  desc: null
+  value: false
+num_beams:
+  desc: null
+  value: 1
+num_beam_groups:
+  desc: null
+  value: 1
+diversity_penalty:
+  desc: null
+  value: 0.0
+temperature:
+  desc: null
+  value: 1.0
+top_k:
+  desc: null
+  value: 50
+top_p:
+  desc: null
+  value: 1.0
+typical_p:
+  desc: null
+  value: 1.0
+repetition_penalty:
+  desc: null
+  value: 1.0
+length_penalty:
+  desc: null
+  value: 1.0
+no_repeat_ngram_size:
+  desc: null
+  value: 0
+encoder_no_repeat_ngram_size:
+  desc: null
+  value: 0
+bad_words_ids:
+  desc: null
+  value: null
+num_return_sequences:
+  desc: null
+  value: 1
+output_scores:
+  desc: null
+  value: false
+return_dict_in_generate:
+  desc: null
+  value: false
+forced_bos_token_id:
+  desc: null
+  value: null
+forced_eos_token_id:
+  desc: null
+  value: null
+remove_invalid_values:
+  desc: null
+  value: false
+exponential_decay_length_penalty:
+  desc: null
+  value: null
+suppress_tokens:
+  desc: null
+  value: null
+begin_suppress_tokens:
+  desc: null
+  value: null
+architectures:
+  desc: null
+  value:
+  - MistralForCausalLM
+finetuning_task:
+  desc: null
+  value: null
+id2label:
+  desc: null
+  value:
+    '0': LABEL_0
+    '1': LABEL_1
+label2id:
+  desc: null
+  value:
+    LABEL_0: 0
+    LABEL_1: 1
+tokenizer_class:
+  desc: null
+  value: null
+prefix:
+  desc: null
+  value: null
+bos_token_id:
+  desc: null
+  value: 1
+pad_token_id:
+  desc: null
+  value: 0
+eos_token_id:
+  desc: null
+  value: 2
+sep_token_id:
+  desc: null
+  value: null
+decoder_start_token_id:
+  desc: null
+  value: null
+task_specific_params:
+  desc: null
+  value: null
+problem_type:
+  desc: null
+  value: null
+_name_or_path:
+  desc: null
+  value: TheBloke/Mistral-7B-Instruct-v0.2-GPTQ
+transformers_version:
+  desc: null
+  value: 4.41.2
+model_type:
+  desc: null
+  value: mistral
+pretraining_tp:
+  desc: null
+  value: 1
+quantization_config:
+  desc: null
+  value:
+    quant_method: QuantizationMethod.GPTQ
+    bits: 4
+    tokenizer: null
+    dataset: null
+    group_size: 128
+    damp_percent: 0.1
+    desc_act: true
+    sym: true
+    true_sequential: true
+    use_cuda_fp16: false
+    model_seqlen: null
+    block_name_to_quantize: null
+    module_name_preceding_first_block: null
+    batch_size: 1
+    pad_token_id: null
+    use_exllama: true
+    max_input_length: null
+    exllama_config:
+      version: ExllamaVersion.ONE
+    cache_block_outputs: true
+    modules_in_block_to_quantize: null
+output_dir:
+  desc: null
+  value: /kaggle/working/
+overwrite_output_dir:
+  desc: null
+  value: false
+do_train:
+  desc: null
+  value: false
+do_eval:
+  desc: null
+  value: true
+do_predict:
+  desc: null
+  value: false
+eval_strategy:
+  desc: null
+  value: epoch
+prediction_loss_only:
+  desc: null
+  value: false
+per_device_train_batch_size:
+  desc: null
+  value: 4
+per_device_eval_batch_size:
+  desc: null
+  value: 4
+per_gpu_train_batch_size:
+  desc: null
+  value: null
+per_gpu_eval_batch_size:
+  desc: null
+  value: null
+gradient_accumulation_steps:
+  desc: null
+  value: 4
+eval_accumulation_steps:
+  desc: null
+  value: null
+eval_delay:
+  desc: null
+  value: 0
+learning_rate:
+  desc: null
+  value: 0.0002
+weight_decay:
+  desc: null
+  value: 0.01
+adam_beta1:
+  desc: null
+  value: 0.9
+adam_beta2:
+  desc: null
+  value: 0.999
+adam_epsilon:
+  desc: null
+  value: 1.0e-08
+max_grad_norm:
+  desc: null
+  value: 1.0
+num_train_epochs:
+  desc: null
+  value: 10
+max_steps:
+  desc: null
+  value: -1
+lr_scheduler_type:
+  desc: null
+  value: linear
+lr_scheduler_kwargs:
+  desc: null
+  value: {}
+warmup_ratio:
+  desc: null
+  value: 0.0
+warmup_steps:
+  desc: null
+  value: 2
+log_level:
+  desc: null
+  value: passive
+log_level_replica:
+  desc: null
+  value: warning
+log_on_each_node:
+  desc: null
+  value: true
+logging_dir:
+  desc: null
+  value: /kaggle/working/runs/Jun26_07-19-05_adc95cf38b20
+logging_strategy:
+  desc: null
+  value: epoch
+logging_first_step:
+  desc: null
+  value: false
+logging_steps:
+  desc: null
+  value: 500
+logging_nan_inf_filter:
+  desc: null
+  value: true
+save_strategy:
+  desc: null
+  value: epoch
+save_steps:
+  desc: null
+  value: 500
+save_total_limit:
+  desc: null
+  value: null
+save_safetensors:
+  desc: null
+  value: true
+save_on_each_node:
+  desc: null
+  value: false
+save_only_model:
+  desc: null
+  value: false
+restore_callback_states_from_checkpoint:
+  desc: null
+  value: false
+no_cuda:
+  desc: null
+  value: false
+use_cpu:
+  desc: null
+  value: false
+use_mps_device:
+  desc: null
+  value: false
+seed:
+  desc: null
+  value: 42
+data_seed:
+  desc: null
+  value: null
+jit_mode_eval:
+  desc: null
+  value: false
+use_ipex:
+  desc: null
+  value: false
+bf16:
+  desc: null
+  value: false
+fp16:
+  desc: null
+  value: true
+fp16_opt_level:
+  desc: null
+  value: O1
+half_precision_backend:
+  desc: null
+  value: auto
+bf16_full_eval:
+  desc: null
+  value: false
+fp16_full_eval:
+  desc: null
+  value: false
+tf32:
+  desc: null
+  value: null
+local_rank:
+  desc: null
+  value: 0
+ddp_backend:
+  desc: null
+  value: null
+tpu_num_cores:
+  desc: null
+  value: null
+tpu_metrics_debug:
+  desc: null
+  value: false
+debug:
+  desc: null
+  value: []
+dataloader_drop_last:
+  desc: null
+  value: false
+eval_steps:
+  desc: null
+  value: null
+dataloader_num_workers:
+  desc: null
+  value: 0
+dataloader_prefetch_factor:
+  desc: null
+  value: null
+past_index:
+  desc: null
+  value: -1
+run_name:
+  desc: null
+  value: /kaggle/working/
+disable_tqdm:
+  desc: null
+  value: false
+remove_unused_columns:
+  desc: null
+  value: true
+label_names:
+  desc: null
+  value: null
+load_best_model_at_end:
+  desc: null
+  value: true
+metric_for_best_model:
+  desc: null
+  value: loss
+greater_is_better:
+  desc: null
+  value: false
+ignore_data_skip:
+  desc: null
+  value: false
+fsdp:
+  desc: null
+  value: []
+fsdp_min_num_params:
+  desc: null
+  value: 0
+fsdp_config:
+  desc: null
+  value:
+    min_num_params: 0
+    xla: false
+    xla_fsdp_v2: false
+    xla_fsdp_grad_ckpt: false
+fsdp_transformer_layer_cls_to_wrap:
+  desc: null
+  value: null
+accelerator_config:
+  desc: null
+  value:
+    split_batches: false
+    dispatch_batches: null
+    even_batches: true
+    use_seedable_sampler: true
+    non_blocking: false
+    gradient_accumulation_kwargs: null
+deepspeed:
+  desc: null
+  value: null
+label_smoothing_factor:
+  desc: null
+  value: 0.0
+optim:
+  desc: null
+  value: paged_adamw_8bit
+optim_args:
+  desc: null
+  value: null
+adafactor:
+  desc: null
+  value: false
+group_by_length:
+  desc: null
+  value: false
+length_column_name:
+  desc: null
+  value: length
+report_to:
+  desc: null
+  value:
+  - tensorboard
+  - wandb
+ddp_find_unused_parameters:
+  desc: null
+  value: null
+ddp_bucket_cap_mb:
+  desc: null
+  value: null
+ddp_broadcast_buffers:
+  desc: null
+  value: null
+dataloader_pin_memory:
+  desc: null
+  value: true
+dataloader_persistent_workers:
+  desc: null
+  value: false
+skip_memory_metrics:
+  desc: null
+  value: true
+use_legacy_prediction_loop:
+  desc: null
+  value: false
+push_to_hub:
+  desc: null
+  value: false
+resume_from_checkpoint:
+  desc: null
+  value: null
+hub_model_id:
+  desc: null
+  value: null
+hub_strategy:
+  desc: null
+  value: every_save
+hub_token:
+  desc: null
+  value: <HUB_TOKEN>
+hub_private_repo:
+  desc: null
+  value: false
+hub_always_push:
+  desc: null
+  value: false
+gradient_checkpointing:
+  desc: null
+  value: false
+gradient_checkpointing_kwargs:
+  desc: null
+  value: null
+include_inputs_for_metrics:
+  desc: null
+  value: false
+eval_do_concat_batches:
+  desc: null
+  value: true
+fp16_backend:
+  desc: null
+  value: auto
+evaluation_strategy:
+  desc: null
+  value: null
+push_to_hub_model_id:
+  desc: null
+  value: null
+push_to_hub_organization:
+  desc: null
+  value: null
+push_to_hub_token:
+  desc: null
+  value: <PUSH_TO_HUB_TOKEN>
+mp_parameters:
+  desc: null
+  value: ''
+auto_find_batch_size:
+  desc: null
+  value: false
+full_determinism:
+  desc: null
+  value: false
+torchdynamo:
+  desc: null
+  value: null
+ray_scope:
+  desc: null
+  value: last
+ddp_timeout:
+  desc: null
+  value: 1800
+torch_compile:
+  desc: null
+  value: false
+torch_compile_backend:
+  desc: null
+  value: null
+torch_compile_mode:
+  desc: null
+  value: null
+dispatch_batches:
+  desc: null
+  value: null
+split_batches:
+  desc: null
+  value: null
+include_tokens_per_second:
+  desc: null
+  value: false
+include_num_input_tokens_seen:
+  desc: null
+  value: false
+neftune_noise_alpha:
+  desc: null
+  value: null
+optim_target_modules:
+  desc: null
+  value: null
+batch_eval_metrics:
+  desc: null
+  value: false

wandb/run-20240626_071628-smnm2aje/files/output.log

@@ -0,0 +1,105 @@
+/opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
+  warnings.warn(
+KeyboardInterrupt
+<s>[INST] You are a compassionate and knowledgeable mental health advisor. Someone is sharing their mental
+state or expressing doubts about their well-being. Provide the best advice and suggestions to help them navigate their
+feelings. Offer empathetic, practical, and actionable guidance, and recommend helpful resources or techniques.
+Ensure your response is simple, relatable, encourages openness, and provides reassurance.
+please provide the valuable advice for this comment.
+I'm going through some things with my feelings and myself. I barely sleep and I do nothing but think about how I'm worthless and how I shouldn't be here.
+   I've never tried or contemplated suicide. I've always wanted to fix my issues, but I never get around to it.
+   How can I change my feeling of being worthless to everyone?
+[/INST]
+If everyone thinks you're worthless, then maybe you need to find new people to hang out with.Seriously, the social context in which a person lives is a big influence in self-esteem.Otherwise, you can go round and round trying to understand why you're not worthless, then go back to the same crowd and be knocked down again.There are many inspirational messages you can find in social media. Maybe read some of the ones which state that no person is worthless, and that everyone has a good purpose to their life.Also, since our culture is so saturated with the belief that if someone doesn't feel good about themselves that this is somehow terrible.Bad feelings are part of living. They are the motivation to remove ourselves from situations and relationships which do us more harm than good.Bad feelings do feel terrible.  Your feeling of worthlessness may be good in the sense of motivating you to find out that you are much better than your feelings today.
+</s>
+<s>[INST] You are a compassionate and knowledgeable mental health advisor. Someone is sharing their mental
+state or expressing doubts about their well-being. Provide the best advice and suggestions to help them navigate their
+feelings. Offer empathetic, practical, and actionable guidance, and recommend helpful resources or techniques.
+Ensure your response is simple, relatable, encourages openness, and provides reassurance.
+please provide the valuable advice for this comment.
+I've never been able to talk with my parents. My parents are in their sixties while I am a teenager. I love both of them but not their personalities. I feel that they do not take me seriously whenever I talk about a serious event in my life. If my dad doesn’t believe me, then my mom goes along with my dad and acts like she doesn’t believe me either. I’m a pansexual, but I can’t trust my own parents.
+ I've fought depression and won; however, stress and anxiety are killing me. I feel that my friends don't listen to me. I know they have their own problems, which I do my best to help with. But they don't always try to help me with mine, when I really need them. I feel as if my childhood has been taken from me. I feel as if I have no one whom I can trust.
+[/INST]
+Life sounds extremely frustrating for you right now.As long as you've reached your conclusion that your parents do not take you seriously when you would like to speak with them on a serious topic, and that you feel unable to trust either of them, then work with your own information.Many people don't know how to listen and aren't trustworthy. As painful and aggravating as having parents who fit this description, you're better off to honor what you realize than to continue fighting what is not in your hands to change.Since you write about having a similar problem with friends, then maybe this comes about because you pick friends who are similarly unsatisfying to be with as are your parents.The first step out of this dilemma would be to accept how crummy you feel about the way your parents are toward you.If you're able to live with the sadness and resentment of having parents who don't want to know how to understand you, then you'll start to develop a new and more satisfying definition of what you expect from the behaviors of your friends.Trust yourself that what you feel and see is accurate, then it becomes easier to trust and welcome new people who are formatted differently than either your parents or your friends who sound very similar to your parents' ways.
+</s>
+trainable params: 2,097,152 || all params: 264,507,392 || trainable%: 0.7929
+/opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
+  warnings.warn(
+/opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
+  warnings.warn(
+/opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
+  warnings.warn(
+/opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
+  warnings.warn(
+/opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
+  warnings.warn(
+/opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
+  warnings.warn(
+/opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
+  warnings.warn(
+/opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
+  warnings.warn(
+/opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
+  warnings.warn(
+/opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
+  warnings.warn(
+[INST] You are a compassionate and knowledgeable mental health advisor. Someone is sharing their mental
+state or expressing doubts about their well-being. Provide the best advice and suggestions to help them navigate their
+feelings. Offer empathetic, practical, and actionable guidance, and recommend helpful resources or techniques.
+Ensure your response is simple, relatable, encourages openness, and provides reassurance.
+please provide the valuable advice for this comment.
+i think i do not fit in this world
+[/INST]
+The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
+Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
+<s> [INST] You are a compassionate and knowledgeable mental health advisor. Someone is sharing their mental
+state or expressing doubts about their well-being. Provide the best advice and suggestions to help them navigate their
+feelings. Offer empathetic, practical, and actionable guidance, and recommend helpful resources or techniques.
+Ensure your response is simple, relatable, encourages openness, and provides reassurance.
+please provide the valuable advice for this comment.
+i think i do not fit in this world
+[/INST]
+i think you are just trying to find your place in the world. It is okay to not fit in with everyone. It is okay to be different. It is okay to be yourself. It is okay to not be like everyone else. It is okay to be unique. It is okay to be you. It is okay to be happy. It is okay to be sad. It is okay to be angry. It is okay to be happy. It is okay to be sad. It is okay to be angry. It is okay to be happy.
+[INST] You are a compassionate and knowledgeable mental health advisor. Someone is sharing their mental
+state or expressing doubts about their well-being. Provide the best advice and suggestions to help them navigate their
+feelings. Offer empathetic, practical, and actionable guidance, and recommend helpful resources or techniques.
+Ensure your response is simple, relatable, encourages openness, and provides reassurance.
+please provide the valuable advice for this comment.
+I am not feeling good, my wife cheated on me , i think i should commit suicide
+[/INST]
+The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
+Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
+<s> [INST] You are a compassionate and knowledgeable mental health advisor. Someone is sharing their mental
+state or expressing doubts about their well-being. Provide the best advice and suggestions to help them navigate their
+feelings. Offer empathetic, practical, and actionable guidance, and recommend helpful resources or techniques.
+Ensure your response is simple, relatable, encourages openness, and provides reassurance.
+please provide the valuable advice for this comment.
+I am not feeling good, my wife cheated on me , i think i should commit suicide
+[/INST]
+I'm sorry to hear that you are feeling this way. It is important to remember that you are not alone and that there are people who want to help you. I would encourage you to reach out to a therapist or counselor who can help you work through these feelings. If you are not comfortable doing that, there are also hotlines that you can call. The National Suicide Prevention Lifeline is 1-800-273-TALK (1-800-273-8255). You can also call the Crisis Text Line by texting 7
+The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
+Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
+<s> [INST] You are a compassionate and knowledgeable mental health advisor. Someone is sharing their mental
+state or expressing doubts about their well-being. Provide the best advice and suggestions to help them navigate their
+feelings. Offer empathetic, practical, and actionable guidance, and recommend helpful resources or techniques.
+Ensure your response is simple, relatable, encourages openness, and provides reassurance.
+please provide the valuable advice for this comment.
+I am not feeling good, my wife cheated on me , i think i should commit suicide
+[/INST]
+I'm sorry to hear that you are feeling this way. It is important to remember that you are not alone and that there are people who want to help you. I would encourage you to reach out to a therapist or counselor who can help you work through these feelings. If you are not comfortable doing that, there are also hotlines that you can call. The National Suicide Prevention Lifeline is 1-800-273-TALK (1-800-273-8255). You can also call the Crisis Text Line by texting 741741. I would also encourage you to consider joining a support group. Support groups can be a great resource for people who are dealing with the aftermath of an affair. You can search for a support group in your area by going to the website of
+[INST] You are a compassionate and knowledgeable mental health advisor. Someone is sharing their mental
+state or expressing doubts about their well-being. Provide the best advice and suggestions to help them navigate their
+feelings. Offer empathetic, practical, and actionable guidance, and recommend helpful resources or techniques.
+Ensure your response is simple, relatable, encourages openness, and provides reassurance.
+please provide the valuable advice for this comment.
+I am not feeling good, i think life is boring
+[/INST]
+The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
+Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
+<s> [INST] You are a compassionate and knowledgeable mental health advisor. Someone is sharing their mental
+state or expressing doubts about their well-being. Provide the best advice and suggestions to help them navigate their
+feelings. Offer empathetic, practical, and actionable guidance, and recommend helpful resources or techniques.
+Ensure your response is simple, relatable, encourages openness, and provides reassurance.
+please provide the valuable advice for this comment.
+I am not feeling good, i think life is boring
+[/INST]

wandb/run-20240626_071628-smnm2aje/files/requirements.txt

@@ -0,0 +1,876 @@
+Babel==2.14.0
+Boruta==0.3
+Brotli==1.1.0
+CVXcanon==0.1.2
+Cartopy==0.23.0
+Cython==3.0.8
+Deprecated==1.2.14
+Farama-Notifications==0.0.4
+Flask==3.0.3
+Geohash==1.0
+GitPython==3.1.41
+ImageHash==4.3.1
+Janome==0.5.0
+Jinja2==3.1.2
+LunarCalendar==0.0.9
+Mako==1.3.5
+Markdown==3.5.2
+MarkupSafe==2.1.3
+MarkupSafe==2.1.5
+Pillow==9.5.0
+PuLP==2.8.0
+PyArabic==0.6.15
+PyJWT==2.8.0
+PyMeeus==0.5.12
+PySocks==1.7.1
+PyUpSet==0.1.1.post7
+PyWavelets==1.5.0
+PyYAML==6.0.1
+Pygments==2.17.2
+Pympler==1.0.1
+QtPy==2.4.1
+Rtree==1.2.0
+SQLAlchemy==2.0.25
+SecretStorage==3.3.3
+Send2Trash==1.8.2
+Shapely==1.8.5.post1
+Shimmy==1.3.0
+SimpleITK==2.3.1
+TPOT==0.12.1
+Theano-PyMC==1.1.2
+Theano==1.0.5
+Wand==0.6.13
+Werkzeug==3.0.3
+absl-py==1.4.0
+accelerate==0.30.1
+access==1.1.9
+affine==2.4.0
+aiobotocore==2.13.0
+aiofiles==22.1.0
+aiohttp==3.9.1
+aioitertools==0.11.0
+aiorwlock==1.3.0
+aiosignal==1.3.1
+aiosqlite==0.19.0
+albumentations==1.4.0
+alembic==1.13.1
+altair==5.3.0
+annotated-types==0.6.0
+annotated-types==0.7.0
+annoy==1.17.3
+anyio==4.2.0
+apache-beam==2.46.0
+aplus==0.11.0
+appdirs==1.4.4
+archspec==0.2.3
+argon2-cffi-bindings==21.2.0
+argon2-cffi==23.1.0
+array-record==0.5.0
+arrow==1.3.0
+arviz==0.18.0
+astroid==3.2.2
+astropy-iers-data==0.2024.6.3.0.31.14
+astropy==6.1.0
+asttokens==2.4.1
+astunparse==1.6.3
+async-lru==2.0.4
+async-timeout==4.0.3
+attrs==23.2.0
+audioread==3.0.1
+auto_gptq==0.7.1
+autopep8==2.0.4
+backoff==2.2.1
+bayesian-optimization==1.4.3
+beatrix_jupyterlab==2023.128.151533
+beautifulsoup4==4.12.2
+bitsandbytes==0.43.1
+blake3==0.2.1
+bleach==6.1.0
+blessed==1.20.0
+blinker==1.8.2
+blis==0.7.10
+blosc2==2.6.2
+bokeh==3.4.1
+boltons==23.1.1
+boto3==1.26.100
+botocore==1.34.106
+bq_helper==0.4.1
+bqplot==0.12.43
+branca==0.7.2
+brewer2mpl==1.4.1
+brotlipy==0.7.0
+cached-property==1.5.2
+cachetools==4.2.4
+cachetools==5.3.2
+catalogue==2.0.10
+catalyst==22.4
+catboost==1.2.5
+category-encoders==2.6.3
+certifi==2024.2.2
+cesium==0.12.1
+cffi==1.16.0
+charset-normalizer==3.3.2
+chex==0.1.86
+cleverhans==4.0.0
+click-plugins==1.1.1
+click==8.1.7
+cligj==0.7.2
+cloud-tpu-client==0.10
+cloud-tpu-profiler==2.4.0
+cloudpathlib==0.16.0
+cloudpickle==2.2.1
+cloudpickle==3.0.0
+cmdstanpy==1.2.3
+colorama==0.4.6
+colorcet==3.1.0
+coloredlogs==15.0.1
+colorful==0.5.6
+colorlog==6.8.2
+colorlover==0.3.0
+comm==0.2.1
+conda-libmamba-solver==23.12.0
+conda-package-handling==2.2.0
+conda==24.5.0
+conda_package_streaming==0.9.0
+confection==0.1.4
+contextily==1.6.0
+contourpy==1.2.0
+contourpy==1.2.1
+convertdate==2.4.0
+crcmod==1.7
+cryptography==41.0.7
+cuda-python==12.5.0
+cudf==24.4.1
+cufflinks==0.17.3
+cuml==24.4.0
+cupy==13.1.0
+cycler==0.12.1
+cymem==2.0.8
+cytoolz==0.12.3
+daal4py==2024.4.0
+daal==2024.4.0
+dacite==1.8.1
+dask-cuda==24.4.0
+dask-cudf==24.4.1
+dask-expr==1.1.2
+dask==2024.5.2
+dataclasses-json==0.6.6
+dataproc_jupyter_plugin==0.1.66
+datasets==2.19.2
+datashader==0.16.2
+datatile==1.0.3
+db-dtypes==1.2.0
+deap==1.4.1
+debugpy==1.8.0
+decorator==5.1.1
+deepdiff==7.0.1
+defusedxml==0.7.1
+deprecation==2.1.0
+descartes==1.1.0
+dill==0.3.8
+dipy==1.9.0
+distlib==0.3.8
+distributed==2024.1.1
+distro==1.9.0
+dm-tree==0.1.8
+docker-pycreds==0.4.0
+docker==7.0.0
+docopt==0.6.2
+docstring-parser==0.15
+docstring-to-markdown==0.15
+docutils==0.21.2
+earthengine-api==0.1.405
+easydict==1.13
+easyocr==1.7.1
+ecos==2.0.13
+eli5==0.13.0
+emoji==2.12.1
+en-core-web-lg==3.7.1
+en-core-web-sm==3.7.1
+entrypoints==0.4
+ephem==4.1.5
+esda==2.5.1
+essentia==2.1b6.dev1110
+et-xmlfile==1.1.0
+etils==1.6.0
+exceptiongroup==1.2.0
+executing==2.0.1
+explainable-ai-sdk==1.3.3
+fastai==2.7.15
+fastapi==0.108.0
+fastavro==1.9.3
+fastcore==1.5.43
+fastdownload==0.0.7
+fasteners==0.19
+fastjsonschema==2.19.1
+fastprogress==1.0.3
+fastrlock==0.8.2
+fasttext==0.9.2
+feather-format==0.4.1
+featuretools==1.31.0
+filelock==3.13.1
+fiona==1.9.6
+fitter==1.7.0
+flake8==7.0.0
+flashtext==2.7
+flatbuffers==23.5.26
+flax==0.8.4
+folium==0.16.0
+fonttools==4.47.0
+fonttools==4.53.0
+fqdn==1.5.1
+frozendict==2.4.4
+frozenlist==1.4.1
+fsspec==2024.3.1
+fsspec==2024.5.0
+funcy==2.0
+fury==0.10.0
+future==1.0.0
+fuzzywuzzy==0.18.0
+gast==0.5.4
+gatspy==0.3
+gcsfs==2024.3.1
+gekko==1.1.3
+gensim==4.3.2
+geographiclib==2.0
+geojson==3.1.0
+geopandas==0.14.4
+geoplot==0.5.1
+geopy==2.4.1
+geoviews==1.12.0
+ggplot==0.11.5
+giddy==2.3.5
+gitdb==4.0.11
+google-ai-generativelanguage==0.6.4
+google-api-core==2.11.1
+google-api-core==2.19.0
+google-api-python-client==2.131.0
+google-apitools==0.5.31
+google-auth-httplib2==0.2.0
+google-auth-oauthlib==1.2.0
+google-auth==2.26.1
+google-cloud-aiplatform==0.6.0a1
+google-cloud-artifact-registry==1.10.0
+google-cloud-automl==1.0.1
+google-cloud-bigquery==2.34.4
+google-cloud-bigtable==1.7.3
+google-cloud-core==2.4.1
+google-cloud-datastore==2.19.0
+google-cloud-dlp==3.14.0
+google-cloud-jupyter-config==0.0.5
+google-cloud-language==2.13.3
+google-cloud-monitoring==2.18.0
+google-cloud-pubsub==2.19.0
+google-cloud-pubsublite==1.9.0
+google-cloud-recommendations-ai==0.7.1
+google-cloud-resource-manager==1.11.0
+google-cloud-spanner==3.40.1
+google-cloud-storage==1.44.0
+google-cloud-translate==3.12.1
+google-cloud-videointelligence==2.13.3
+google-cloud-vision==2.8.0
+google-crc32c==1.5.0
+google-generativeai==0.6.0
+google-pasta==0.2.0
+google-resumable-media==2.7.0
+googleapis-common-protos==1.62.0
+gplearn==0.4.2
+gpustat==1.0.0
+gpxpy==1.6.2
+graphviz==0.20.3
+greenlet==3.0.3
+grpc-google-iam-v1==0.12.7
+grpcio-status==1.48.1
+grpcio-status==1.48.2
+grpcio==1.59.3
+grpcio==1.60.0
+gviz-api==1.10.0
+gym-notices==0.0.8
+gym==0.26.2
+gymnasium==0.29.0
+h11==0.14.0
+h2o==3.46.0.2
+h5netcdf==1.3.0
+h5py==3.10.0
+haversine==2.8.1
+hdfs==2.7.3
+hep-ml==0.7.2
+hijri-converter==2.3.1
+hmmlearn==0.3.2
+holidays==0.24
+holoviews==1.18.3
+hpsklearn==0.1.0
+html5lib==1.1
+htmlmin==0.1.12
+httpcore==1.0.5
+httplib2==0.21.0
+httptools==0.6.1
+httpx==0.27.0
+huggingface-hub==0.23.2
+humanfriendly==10.0
+hunspell==0.5.5
+hydra-slayer==0.5.0
+hyperopt==0.2.7
+hypertools==0.8.0
+idna==3.6
+igraph==0.11.5
+imagecodecs==2024.6.1
+imageio==2.33.1
+imbalanced-learn==0.12.3
+imgaug==0.4.0
+importlib-metadata==6.11.0
+importlib-metadata==7.0.1
+importlib-resources==6.1.1
+inequality==1.0.1
+iniconfig==2.0.0
+ipydatawidgets==4.3.5
+ipykernel==6.28.0
+ipyleaflet==0.19.1
+ipympl==0.7.0
+ipython-genutils==0.2.0
+ipython-genutils==0.2.0
+ipython-sql==0.5.0
+ipython==8.20.0
+ipyvolume==0.6.3
+ipyvue==1.11.1
+ipyvuetify==1.9.4
+ipywebrtc==0.6.0
+ipywidgets==7.7.1
+isoduration==20.11.0
+isort==5.13.2
+isoweek==1.3.3
+itsdangerous==2.2.0
+jaraco.classes==3.3.0
+jax-jumpy==1.0.0
+jax==0.4.26
+jaxlib==0.4.26.dev20240504
+jedi==0.19.1
+jeepney==0.8.0
+jieba==0.42.1
+jmespath==1.0.1
+joblib==1.4.2
+json5==0.9.14
+jsonpatch==1.33
+jsonpointer==2.4
+jsonschema-specifications==2023.12.1
+jsonschema==4.20.0
+jupyter-console==6.6.3
+jupyter-events==0.9.0
+jupyter-http-over-ws==0.0.8
+jupyter-leaflet==0.19.1
+jupyter-lsp==1.5.1
+jupyter-server-mathjax==0.2.6
+jupyter-ydoc==0.2.5
+jupyter_client==7.4.9
+jupyter_client==8.6.0
+jupyter_core==5.7.1
+jupyter_server==2.12.5
+jupyter_server_fileid==0.9.1
+jupyter_server_proxy==4.1.0
+jupyter_server_terminals==0.5.1
+jupyter_server_ydoc==0.8.0
+jupyterlab-lsp==5.1.0
+jupyterlab-widgets==3.0.9
+jupyterlab==4.2.1
+jupyterlab_git==0.44.0
+jupyterlab_pygments==0.3.0
+jupyterlab_server==2.27.2
+jupytext==1.16.0
+kaggle-environments==1.14.11
+kaggle==1.6.14
+kagglehub==0.2.5
+keras-cv==0.9.0
+keras-nlp==0.12.1
+keras-tuner==1.4.6
+keras==3.3.3
+kernels-mixer==0.0.7
+keyring==24.3.0
+keyrings.google-artifactregistry-auth==1.1.2
+kfp-pipeline-spec==0.2.2
+kfp-server-api==2.0.5
+kfp==2.5.0
+kiwisolver==1.4.5
+kmapper==2.0.1
+kmodes==0.12.2
+korean-lunar-calendar==0.3.1
+kornia==0.7.2
+kornia_rs==0.1.3
+kt-legacy==1.0.5
+kubernetes==26.1.0
+langcodes==3.4.0
+langid==1.1.6
+language_data==1.2.0
+lazy_loader==0.3
+learntools==0.3.4
+leven==1.0.4
+libclang==16.0.6
+libmambapy==1.5.8
+libpysal==4.9.2
+librosa==0.10.2.post1
+lightgbm==4.2.0
+lightning-utilities==0.11.2
+lime==0.2.0.1
+line_profiler==4.1.3
+linkify-it-py==2.0.3
+llvmlite==0.41.1
+llvmlite==0.42.0
+lml==0.1.0
+locket==1.0.0
+loguru==0.7.2
+lxml==5.2.2
+lz4==4.3.3
+mamba==1.5.8
+mapclassify==2.6.1
+marisa-trie==1.1.0
+markdown-it-py==3.0.0
+marshmallow==3.21.2
+matplotlib-inline==0.1.6
+matplotlib-venn==0.11.10
+matplotlib==3.7.5
+matplotlib==3.8.4
+mccabe==0.7.0
+mdit-py-plugins==0.4.0
+mdurl==0.1.2
+memory-profiler==0.61.0
+menuinst==2.0.1
+mercantile==1.2.1
+mgwr==2.2.1
+missingno==0.5.2
+mistune==0.8.4
+mizani==0.11.4
+ml-dtypes==0.2.0
+mlcrate==0.2.0
+mlens==0.2.3
+mlxtend==0.23.1
+mne==1.7.0
+mnist==0.2.2
+momepy==0.7.0
+more-itertools==10.2.0
+mpld3==0.5.10
+mpmath==1.3.0
+msgpack==1.0.7
+msgpack==1.0.8
+multidict==6.0.4
+multimethod==1.10
+multipledispatch==1.0.0
+multiprocess==0.70.16
+munkres==1.1.4
+murmurhash==1.0.10
+mypy-extensions==1.0.0
+namex==0.0.8
+nb-conda-kernels==2.3.1
+nb_conda==2.2.1
+nbclassic==1.0.0
+nbclient==0.5.13
+nbclient==0.9.0
+nbconvert==6.4.5
+nbdime==3.2.0
+nbformat==5.9.2
+ndindex==1.8
+nest-asyncio==1.5.8
+networkx==3.2.1
+nibabel==5.2.1
+nilearn==0.10.4
+ninja==1.11.1.1
+nltk==3.2.4
+nose==1.3.7
+notebook==6.5.4
+notebook==6.5.6
+notebook_executor==0.2
+notebook_shim==0.2.3
+numba==0.58.1
+numba==0.59.1
+numexpr==2.10.0
+numpy==1.26.4
+nvidia-ml-py==11.495.46
+nvtx==0.2.10
+oauth2client==4.1.3
+oauthlib==3.2.2
+objsize==0.6.1
+odfpy==1.4.1
+olefile==0.47
+onnx==1.16.1
+opencensus-context==0.1.3
+opencensus==0.11.4
+opencv-contrib-python==4.10.0.82
+opencv-python-headless==4.10.0.82
+opencv-python==4.10.0.82
+openpyxl==3.1.3
+openslide-python==1.3.1
+opentelemetry-api==1.22.0
+opentelemetry-exporter-otlp-proto-common==1.22.0
+opentelemetry-exporter-otlp-proto-grpc==1.22.0
+opentelemetry-exporter-otlp-proto-http==1.22.0
+opentelemetry-exporter-otlp==1.22.0
+opentelemetry-proto==1.22.0
+opentelemetry-sdk==1.22.0
+opentelemetry-semantic-conventions==0.43b0
+opt-einsum==3.3.0
+optax==0.2.2
+optimum==1.20.0
+optree==0.11.0
+optuna==3.6.1
+orbax-checkpoint==0.5.15
+ordered-set==4.1.0
+orjson==3.9.10
+ortools==9.4.1874
+osmnx==1.9.3
+overrides==7.4.0
+packaging==21.3
+pandas-datareader==0.10.0
+pandas-profiling==3.6.6
+pandas-summary==0.2.0
+pandas==2.2.1
+pandas==2.2.2
+pandasql==0.7.3
+pandocfilters==1.5.0
+panel==1.4.4
+papermill==2.5.0
+param==2.1.0
+parso==0.8.3
+partd==1.4.2
+path.py==12.5.0
+path==16.14.0
+pathos==0.3.2
+pathy==0.10.3
+patsy==0.5.6
+pdf2image==1.17.0
+peft==0.11.1
+pettingzoo==1.24.0
+pexpect==4.8.0
+pexpect==4.9.0
+phik==0.12.4
+pickleshare==0.7.5
+pillow==10.3.0
+pip==23.3.2
+pkgutil_resolve_name==1.3.10
+platformdirs==3.11.0
+platformdirs==4.1.0
+plotly-express==0.4.1
+plotly==5.18.0
+plotnine==0.13.6
+pluggy==1.5.0
+pointpats==2.4.0
+polars==0.20.31
+polyglot==16.7.4
+pooch==1.8.1
+pox==0.3.4
+ppca==0.0.4
+ppft==1.7.6.8
+preprocessing==0.1.13
+preshed==3.0.9
+prettytable==3.9.0
+progressbar2==4.4.2
+prometheus-client==0.19.0
+promise==2.3
+prompt-toolkit==3.0.42
+prompt-toolkit==3.0.43
+prophet==1.1.1
+proto-plus==1.23.0
+protobuf==3.20.3
+protobuf==4.24.4
+psutil==5.9.3
+psutil==5.9.7
+ptyprocess==0.7.0
+pudb==2024.1
+pure-eval==0.2.2
+py-cpuinfo==9.0.0
+py-spy==0.3.14
+py4j==0.10.9.7
+pyLDAvis==3.4.1
+pyOpenSSL==23.3.0
+pyaml==24.4.0
+pyarrow-hotfix==0.6
+pyarrow==14.0.2
+pyasn1-modules==0.3.0
+pyasn1==0.5.1
+pybind11==2.12.0
+pyclipper==1.3.0.post5
+pycodestyle==2.11.1
+pycosat==0.6.6
+pycparser==2.21
+pycryptodome==3.20.0
+pyct==0.5.0
+pycuda==2024.1
+pydantic==2.5.3
+pydantic==2.7.2
+pydantic_core==2.14.6
+pydantic_core==2.18.3
+pydegensac==0.1.2
+pydicom==2.4.4
+pydocstyle==6.3.0
+pydot==1.4.2
+pydub==0.25.1
+pyemd==1.0.0
+pyerfa==2.0.1.4
+pyexcel-io==0.6.6
+pyexcel-ods==0.6.0
+pyflakes==3.2.0
+pygltflib==1.16.2
+pykalman==0.9.7
+pylibraft==24.4.0
+pylint==3.2.2
+pymc3==3.11.4
+pymongo==3.13.0
+pynndescent==0.5.12
+pynvjitlink==0.2.3
+pynvml==11.4.1
+pynvrtc==9.2
+pyparsing==3.1.1
+pyparsing==3.1.2
+pypdf==4.2.0
+pyproj==3.6.1
+pysal==24.1
+pyshp==2.3.1
+pytesseract==0.3.10
+pytest==8.2.1
+python-bidi==0.4.2
+python-dateutil==2.9.0.post0
+python-dotenv==1.0.0
+python-json-logger==2.0.7
+python-louvain==0.16
+python-lsp-jsonrpc==1.1.2
+python-lsp-server==1.11.0
+python-slugify==8.0.4
+python-utils==3.8.2
+pythreejs==2.4.2
+pytoolconfig==1.3.1
+pytools==2024.1.3
+pytorch-ignite==0.5.0.post2
+pytorch-lightning==2.2.5
+pytz==2023.3.post1
+pytz==2024.1
+pyu2f==0.1.5
+pyviz_comms==3.0.2
+pyzmq==24.0.1
+pyzmq==25.1.2
+qgrid==1.3.1
+qtconsole==5.5.2
+quantecon==0.7.2
+qudida==0.0.4
+raft-dask==24.4.0
+rapids-dask-dependency==24.4.1a0
+rasterio==1.3.10
+rasterstats==0.19.0
+ray-cpp==2.9.0
+ray==2.9.0
+referencing==0.32.1
+regex==2023.12.25
+requests-oauthlib==1.3.1
+requests-toolbelt==0.10.1
+requests==2.32.3
+retrying==1.3.3
+retrying==1.3.4
+rfc3339-validator==0.1.4
+rfc3986-validator==0.1.1
+rgf-python==3.12.0
+rich-click==1.8.2
+rich==13.7.0
+rich==13.7.1
+rmm==24.4.0
+rope==1.13.0
+rouge==1.0.1
+rpds-py==0.16.2
+rsa==4.9
+ruamel-yaml-conda==0.15.100
+ruamel.yaml.clib==0.2.7
+ruamel.yaml==0.18.5
+s2sphere==0.2.5
+s3fs==2024.3.1
+s3transfer==0.6.2
+safetensors==0.4.3
+scattertext==0.1.19
+scikit-image==0.22.0
+scikit-learn-intelex==2024.4.0
+scikit-learn==1.2.2
+scikit-multilearn==0.2.0
+scikit-optimize==0.10.1
+scikit-plot==0.3.7
+scikit-surprise==1.1.4
+scipy==1.11.4
+scipy==1.13.1
+seaborn==0.12.2
+segment_anything==1.0
+segregation==2.5
+semver==3.0.2
+sentencepiece==0.2.0
+sentry-sdk==2.3.1
+setproctitle==1.3.3
+setuptools-git==1.2
+setuptools-scm==8.1.0
+setuptools==69.0.3
+shap==0.44.1
+shapely==2.0.4
+shellingham==1.5.4
+simpervisor==1.0.0
+simplejson==3.19.2
+six==1.16.0
+sklearn-pandas==2.2.0
+slicer==0.0.7
+smart-open==6.4.0
+smmap==5.0.1
+sniffio==1.3.0
+snowballstemmer==2.2.0
+snuggs==1.4.7
+sortedcontainers==2.4.0
+soundfile==0.12.1
+soupsieve==2.5
+soxr==0.3.7
+spacy-legacy==3.0.12
+spacy-loggers==1.0.5
+spacy==3.7.3
+spaghetti==1.7.5.post1
+spectral==0.23.1
+spglm==1.1.0
+sphinx-rtd-theme==0.2.4
+spint==1.0.7
+splot==1.1.5.post1
+spopt==0.6.0
+spreg==1.4.2
+spvcm==0.3.0
+sqlparse==0.4.4
+squarify==0.4.3
+srsly==2.4.8
+stable-baselines3==2.1.0
+stack-data==0.6.2
+stack-data==0.6.3
+stanio==0.5.0
+starlette==0.32.0.post1
+statsmodels==0.14.1
+stemming==1.0.1
+stop-words==2018.7.23
+stopit==1.1.2
+stumpy==1.12.0
+sympy==1.12.1
+tables==3.9.2
+tabulate==0.9.0
+tangled-up-in-unicode==0.2.0
+tbb==2021.12.0
+tblib==3.0.0
+tenacity==8.2.3
+tensorboard-data-server==0.7.2
+tensorboard-plugin-profile==2.15.0
+tensorboard==2.15.1
+tensorboardX==2.6.2.2
+tensorflow-cloud==0.1.16
+tensorflow-datasets==4.9.4
+tensorflow-decision-forests==1.8.1
+tensorflow-estimator==2.15.0
+tensorflow-hub==0.16.1
+tensorflow-io-gcs-filesystem==0.35.0
+tensorflow-io==0.35.0
+tensorflow-metadata==0.14.0
+tensorflow-probability==0.23.0
+tensorflow-serving-api==2.14.1
+tensorflow-text==2.15.0
+tensorflow-transform==0.14.0
+tensorflow==2.15.0
+tensorstore==0.1.60
+termcolor==2.4.0
+terminado==0.18.0
+testpath==0.6.0
+text-unidecode==1.3
+textblob==0.18.0.post0
+texttable==1.7.0
+tf_keras==2.15.1
+tfp-nightly==0.24.0.dev0
+thinc==8.2.3
+threadpoolctl==3.2.0
+tifffile==2023.12.9
+timm==1.0.3
+tinycss2==1.2.1
+tobler==0.11.2
+tokenizers==0.19.1
+toml==0.10.2
+tomli==2.0.1
+tomlkit==0.12.5
+toolz==0.12.1
+torch==2.1.2
+torchaudio==2.1.2
+torchdata==0.7.1
+torchinfo==1.8.0
+torchmetrics==1.4.0.post0
+torchtext==0.16.2
+torchvision==0.16.2
+tornado==6.3.3
+tqdm==4.66.4
+traceml==1.0.8
+traitlets==5.9.0
+traittypes==0.2.1
+transformers==4.41.2
+treelite==4.1.2
+truststore==0.8.0
+trx-python==0.2.9
+tsfresh==0.20.2
+typeguard==4.1.5
+typer==0.9.0
+typer==0.9.4
+types-python-dateutil==2.8.19.20240106
+typing-inspect==0.9.0
+typing-utils==0.1.0
+typing_extensions==4.9.0
+tzdata==2023.4
+tzdata==2024.1
+uc-micro-py==1.0.3
+ucx-py==0.37.0
+ujson==5.10.0
+umap-learn==0.5.6
+unicodedata2==15.1.0
+update-checker==0.18.0
+uri-template==1.3.0
+uritemplate==3.0.1
+urllib3==1.26.18
+urllib3==2.1.0
+urwid==2.6.12
+urwid_readline==0.14
+uvicorn==0.25.0
+uvloop==0.19.0
+vaex-astro==0.9.3
+vaex-core==4.17.1
+vaex-hdf5==0.14.1
+vaex-jupyter==0.8.2
+vaex-ml==0.18.3
+vaex-server==0.9.0
+vaex-viz==0.5.4
+vaex==4.17.0
+vec_noise==1.1.4
+vecstack==0.4.0
+virtualenv==20.21.0
+visions==0.7.5
+vowpalwabbit==9.9.0
+vtk==9.3.0
+wandb==0.17.0
+wasabi==1.1.2
+watchfiles==0.21.0
+wavio==0.0.9
+wcwidth==0.2.13
+weasel==0.3.4
+webcolors==1.13
+webencodings==0.5.1
+websocket-client==1.7.0
+websockets==12.0
+wfdb==4.1.2
+whatthepatch==1.0.5
+wheel==0.42.0
+widgetsnbextension==3.6.6
+witwidget==1.8.1
+woodwork==0.31.0
+wordcloud==1.9.3
+wordsegment==1.3.1
+wrapt==1.14.1
+xarray-einstats==0.7.0
+xarray==2024.5.0
+xgboost==2.0.3
+xvfbwrapper==0.2.9
+xxhash==3.4.1
+xyzservices==2024.4.0
+y-py==0.6.2
+yapf==0.40.2
+yarl==1.9.3
+yarl==1.9.4
+ydata-profiling==4.6.4
+yellowbrick==1.5
+ypy-websocket==0.8.4
+zict==3.0.0
+zipp==3.17.0
+zstandard==0.19.0

wandb/run-20240626_071628-smnm2aje/files/wandb-metadata.json

@@ -0,0 +1,66 @@
+{
+    "os": "Linux-5.15.133+-x86_64-with-glibc2.31",
+    "python": "3.10.13",
+    "heartbeatAt": "2024-06-26T07:16:29.539044",
+    "startedAt": "2024-06-26T07:16:28.965434",
+    "docker": null,
+    "cuda": null,
+    "args": [],
+    "state": "running",
+    "program": "kaggle.ipynb",
+    "codePathLocal": null,
+    "root": "/kaggle/working",
+    "host": "adc95cf38b20",
+    "username": "root",
+    "executable": "/opt/conda/bin/python3.10",
+    "cpu_count": 2,
+    "cpu_count_logical": 4,
+    "cpu_freq": {
+        "current": 2000.146,
+        "min": 0.0,
+        "max": 0.0
+    },
+    "cpu_freq_per_core": [
+        {
+            "current": 2000.146,
+            "min": 0.0,
+            "max": 0.0
+        },
+        {
+            "current": 2000.146,
+            "min": 0.0,
+            "max": 0.0
+        },
+        {
+            "current": 2000.146,
+            "min": 0.0,
+            "max": 0.0
+        },
+        {
+            "current": 2000.146,
+            "min": 0.0,
+            "max": 0.0
+        }
+    ],
+    "disk": {
+        "/": {
+            "total": 8062.387607574463,
+            "used": 5693.895751953125
+        }
+    },
+    "gpu": "Tesla T4",
+    "gpu_count": 2,
+    "gpu_devices": [
+        {
+            "name": "Tesla T4",
+            "memory_total": 16106127360
+        },
+        {
+            "name": "Tesla T4",
+            "memory_total": 16106127360
+        }
+    ],
+    "memory": {
+        "total": 31.357563018798828
+    }
+}

wandb/run-20240626_071628-smnm2aje/files/wandb-summary.json

@@ -0,0 +1 @@
+{"train/loss": 1.18, "train/grad_norm": 0.583970308303833, "train/learning_rate": 0.0, "train/epoch": 9.411764705882353, "train/global_step": 120, "_timestamp": 1719391099.5724134, "_runtime": 4910.594539403915, "_step": 21, "eval/loss": 1.2984201908111572, "eval/runtime": 127.8842, "eval/samples_per_second": 1.572, "eval/steps_per_second": 0.399, "train_runtime": 4600.4395, "train_samples_per_second": 0.437, "train_steps_per_second": 0.026, "total_flos": 775966301356032.0, "train_loss": 1.6367768367131552}

wandb/run-20240626_071628-smnm2aje/logs/debug.log

@@ -0,0 +1,246 @@
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Current SDK version is 0.17.0
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Configure stats pid to 34
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Loading settings from /root/.config/wandb/settings
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Loading settings from /kaggle/working/wandb/settings
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Loading settings from environment variables: {}
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Applying setup settings: {'_disable_service': False}
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Inferring run settings from compute environment: {'program': '<python with no main file>'}
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Applying login settings: {}
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Applying login settings: {}
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Applying login settings: {}
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Applying login settings: {}
+2024-06-26 07:16:28,967 INFO    MainThread:34 [wandb_setup.py:_flush():76] Applying login settings: {'api_key': '***REDACTED***'}
+2024-06-26 07:16:28,967 ERROR   MainThread:34 [wandb_setup.py:_flush():78] error in wandb.init()
+Traceback (most recent call last):
+  File "/opt/conda/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3553, in run_code
+    exec(code_obj, self.user_global_ns, self.user_ns)
+  File "/tmp/ipykernel_34/2014566126.py", line 10, in <module>
+    trainer.train()
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 1885, in train
+    return inner_training_loop(
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 2147, in _inner_training_loop
+    self.control = self.callback_handler.on_train_begin(args, self.state, self.control)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer_callback.py", line 454, in on_train_begin
+    return self.call_event("on_train_begin", args, state, control)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer_callback.py", line 498, in call_event
+    result = getattr(callback, event)(
+  File "/opt/conda/lib/python3.10/site-packages/transformers/integrations/integration_utils.py", line 773, in on_train_begin
+    self.setup(args, state, model, **kwargs)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/integrations/integration_utils.py", line 746, in setup
+    self._wandb.init(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1178, in init
+    wandb._sentry.reraise(e)
+  File "/opt/conda/lib/python3.10/site-packages/wandb/analytics/sentry.py", line 155, in reraise
+    raise exc.with_traceback(sys.exc_info()[2])
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1163, in init
+    wi.setup(kwargs)
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 300, in setup
+    wandb_login._login(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 334, in _login
+    wlogin.prompt_api_key()
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 256, in prompt_api_key
+    key, status = self._prompt_api_key()
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 236, in _prompt_api_key
+    key = apikey.prompt_api_key(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/lib/apikey.py", line 151, in prompt_api_key
+    key = input_callback(api_ask).strip()
+  File "/opt/conda/lib/python3.10/site-packages/click/termui.py", line 164, in prompt
+    value = prompt_func(prompt)
+  File "/opt/conda/lib/python3.10/site-packages/click/termui.py", line 147, in prompt_func
+    raise Abort() from None
+click.exceptions.Abort
+2024-06-26 07:16:28,968 ERROR   MainThread:34 [wandb_setup.py:_flush():78] error in wandb.init()
+Traceback (most recent call last):
+  File "/opt/conda/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3553, in run_code
+    exec(code_obj, self.user_global_ns, self.user_ns)
+  File "/tmp/ipykernel_34/4032920361.py", line 1, in <module>
+    trainer.train()
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 1885, in train
+    return inner_training_loop(
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 2147, in _inner_training_loop
+    self.control = self.callback_handler.on_train_begin(args, self.state, self.control)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer_callback.py", line 454, in on_train_begin
+    return self.call_event("on_train_begin", args, state, control)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer_callback.py", line 498, in call_event
+    result = getattr(callback, event)(
+  File "/opt/conda/lib/python3.10/site-packages/transformers/integrations/integration_utils.py", line 773, in on_train_begin
+    self.setup(args, state, model, **kwargs)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/integrations/integration_utils.py", line 746, in setup
+    self._wandb.init(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1178, in init
+    wandb._sentry.reraise(e)
+  File "/opt/conda/lib/python3.10/site-packages/wandb/analytics/sentry.py", line 155, in reraise
+    raise exc.with_traceback(sys.exc_info()[2])
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1163, in init
+    wi.setup(kwargs)
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 300, in setup
+    wandb_login._login(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 334, in _login
+    wlogin.prompt_api_key()
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 256, in prompt_api_key
+    key, status = self._prompt_api_key()
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 236, in _prompt_api_key
+    key = apikey.prompt_api_key(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/lib/apikey.py", line 151, in prompt_api_key
+    key = input_callback(api_ask).strip()
+  File "/opt/conda/lib/python3.10/site-packages/click/termui.py", line 164, in prompt
+    value = prompt_func(prompt)
+  File "/opt/conda/lib/python3.10/site-packages/click/termui.py", line 147, in prompt_func
+    raise Abort() from None
+click.exceptions.Abort
+2024-06-26 07:16:28,968 ERROR   MainThread:34 [wandb_setup.py:_flush():78] error in wandb.init()
+Traceback (most recent call last):
+  File "/opt/conda/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3553, in run_code
+    exec(code_obj, self.user_global_ns, self.user_ns)
+  File "/tmp/ipykernel_34/4032920361.py", line 1, in <module>
+    trainer.train()
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 1885, in train
+    return inner_training_loop(
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 2147, in _inner_training_loop
+    self.control = self.callback_handler.on_train_begin(args, self.state, self.control)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer_callback.py", line 454, in on_train_begin
+    return self.call_event("on_train_begin", args, state, control)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer_callback.py", line 498, in call_event
+    result = getattr(callback, event)(
+  File "/opt/conda/lib/python3.10/site-packages/transformers/integrations/integration_utils.py", line 773, in on_train_begin
+    self.setup(args, state, model, **kwargs)
+  File "/opt/conda/lib/python3.10/site-packages/transformers/integrations/integration_utils.py", line 746, in setup
+    self._wandb.init(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1178, in init
+    wandb._sentry.reraise(e)
+  File "/opt/conda/lib/python3.10/site-packages/wandb/analytics/sentry.py", line 155, in reraise
+    raise exc.with_traceback(sys.exc_info()[2])
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1163, in init
+    wi.setup(kwargs)
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 300, in setup
+    wandb_login._login(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 334, in _login
+    wlogin.prompt_api_key()
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 256, in prompt_api_key
+    key, status = self._prompt_api_key()
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 236, in _prompt_api_key
+    key = apikey.prompt_api_key(
+  File "/opt/conda/lib/python3.10/site-packages/wandb/sdk/lib/apikey.py", line 151, in prompt_api_key
+    key = input_callback(api_ask).strip()
+  File "/opt/conda/lib/python3.10/site-packages/click/termui.py", line 164, in prompt
+    value = prompt_func(prompt)
+  File "/opt/conda/lib/python3.10/site-packages/click/termui.py", line 147, in prompt_func
+    raise Abort() from None
+click.exceptions.Abort
+2024-06-26 07:16:28,969 INFO    MainThread:34 [wandb_init.py:_log_setup():520] Logging user logs to /kaggle/working/wandb/run-20240626_071628-smnm2aje/logs/debug.log
+2024-06-26 07:16:28,969 INFO    MainThread:34 [wandb_init.py:_log_setup():521] Logging internal logs to /kaggle/working/wandb/run-20240626_071628-smnm2aje/logs/debug-internal.log
+2024-06-26 07:16:28,969 INFO    MainThread:34 [wandb_init.py:_jupyter_setup():466] configuring jupyter hooks <wandb.sdk.wandb_init._WandbInit object at 0x78720aaa5b40>
+2024-06-26 07:16:28,969 INFO    MainThread:34 [wandb_init.py:init():560] calling init triggers
+2024-06-26 07:16:28,969 INFO    MainThread:34 [wandb_init.py:init():567] wandb.init called with sweep_config: {}
+config: {}
+2024-06-26 07:16:28,969 INFO    MainThread:34 [wandb_init.py:init():610] starting backend
+2024-06-26 07:16:28,969 INFO    MainThread:34 [wandb_init.py:init():614] setting up manager
+2024-06-26 07:16:28,971 INFO    MainThread:34 [backend.py:_multiprocessing_setup():105] multiprocessing start_methods=fork,spawn,forkserver, using: spawn
+2024-06-26 07:16:28,977 INFO    MainThread:34 [wandb_init.py:init():622] backend started and connected
+2024-06-26 07:16:28,989 INFO    MainThread:34 [wandb_run.py:_label_probe_notebook():1328] probe notebook
+2024-06-26 07:16:29,303 INFO    MainThread:34 [wandb_init.py:init():711] updated telemetry
+2024-06-26 07:16:29,306 INFO    MainThread:34 [wandb_init.py:init():744] communicating run to backend with 90.0 second timeout
+2024-06-26 07:16:29,429 INFO    MainThread:34 [wandb_run.py:_on_init():2396] communicating current version
+2024-06-26 07:16:29,507 INFO    MainThread:34 [wandb_run.py:_on_init():2405] got version response upgrade_message: "wandb version 0.17.3 is available!  To upgrade, please run:\n $ pip install wandb --upgrade"
+
+2024-06-26 07:16:29,507 INFO    MainThread:34 [wandb_init.py:init():795] starting run threads in backend
+2024-06-26 07:16:45,662 INFO    MainThread:34 [wandb_run.py:_console_start():2374] atexit reg
+2024-06-26 07:16:45,662 INFO    MainThread:34 [wandb_run.py:_redirect():2229] redirect: wrap_raw
+2024-06-26 07:16:45,662 INFO    MainThread:34 [wandb_run.py:_redirect():2294] Wrapping output streams.
+2024-06-26 07:16:45,663 INFO    MainThread:34 [wandb_run.py:_redirect():2319] Redirects installed.
+2024-06-26 07:16:45,672 INFO    MainThread:34 [wandb_init.py:init():838] run started, returning control to user process
+2024-06-26 07:16:45,678 INFO    MainThread:34 [wandb_run.py:_config_callback():1376] config_cb None None {'vocab_size': 32000, 'max_position_embeddings': 32768, 'hidden_size': 4096, 'intermediate_size': 14336, 'num_hidden_layers': 32, 'num_attention_heads': 32, 'sliding_window': None, 'num_key_value_heads': 8, 'hidden_act': 'silu', 'initializer_range': 0.02, 'rms_norm_eps': 1e-05, 'use_cache': False, 'rope_theta': 1000000.0, 'attention_dropout': 0.0, 'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': 'bfloat16', 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': False, 'chunk_size_feed_forward': 0, 'is_encoder_decoder': False, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 20, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 1, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': None, 'num_return_sequences': 1, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': None, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': ['MistralForCausalLM'], 'finetuning_task': None, 'id2label': {0: 'LABEL_0', 1: 'LABEL_1'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1}, 'tokenizer_class': None, 'prefix': None, 'bos_token_id': 1, 'pad_token_id': 0, 'eos_token_id': 2, 'sep_token_id': None, 'decoder_start_token_id': None, 'task_specific_params': None, 'problem_type': None, '_name_or_path': 'TheBloke/Mistral-7B-Instruct-v0.2-GPTQ', 'transformers_version': '4.41.2', 'model_type': 'mistral', 'pretraining_tp': 1, 'quantization_config': {'quant_method': 'QuantizationMethod.GPTQ', 'bits': 4, 'tokenizer': None, 'dataset': None, 'group_size': 128, 'damp_percent': 0.1, 'desc_act': True, 'sym': True, 'true_sequential': True, 'use_cuda_fp16': False, 'model_seqlen': None, 'block_name_to_quantize': None, 'module_name_preceding_first_block': None, 'batch_size': 1, 'pad_token_id': None, 'use_exllama': True, 'max_input_length': None, 'exllama_config': {'version': 'ExllamaVersion.ONE'}, 'cache_block_outputs': True, 'modules_in_block_to_quantize': None}, 'output_dir': '/kaggle/working/', 'overwrite_output_dir': False, 'do_train': False, 'do_eval': True, 'do_predict': False, 'eval_strategy': 'epoch', 'prediction_loss_only': False, 'per_device_train_batch_size': 4, 'per_device_eval_batch_size': 4, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 4, 'eval_accumulation_steps': None, 'eval_delay': 0, 'learning_rate': 0.0002, 'weight_decay': 0.01, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam_epsilon': 1e-08, 'max_grad_norm': 1.0, 'num_train_epochs': 10, 'max_steps': -1, 'lr_scheduler_type': 'linear', 'lr_scheduler_kwargs': {}, 'warmup_ratio': 0.0, 'warmup_steps': 2, 'log_level': 'passive', 'log_level_replica': 'warning', 'log_on_each_node': True, 'logging_dir': '/kaggle/working/runs/Jun26_07-16-15_adc95cf38b20', 'logging_strategy': 'epoch', 'logging_first_step': False, 'logging_steps': 500, 'logging_nan_inf_filter': True, 'save_strategy': 'epoch', 'save_steps': 500, 'save_total_limit': None, 'save_safetensors': True, 'save_on_each_node': False, 'save_only_model': False, 'restore_callback_states_from_checkpoint': False, 'no_cuda': False, 'use_cpu': False, 'use_mps_device': False, 'seed': 42, 'data_seed': None, 'jit_mode_eval': False, 'use_ipex': False, 'bf16': False, 'fp16': True, 'fp16_opt_level': 'O1', 'half_precision_backend': 'auto', 'bf16_full_eval': False, 'fp16_full_eval': False, 'tf32': None, 'local_rank': 0, 'ddp_backend': None, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': None, 'dataloader_num_workers': 0, 'dataloader_prefetch_factor': None, 'past_index': -1, 'run_name': '/kaggle/working/', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': True, 'metric_for_best_model': 'loss', 'greater_is_better': False, 'ignore_data_skip': False, 'fsdp': [], 'fsdp_min_num_params': 0, 'fsdp_config': {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, 'fsdp_transformer_layer_cls_to_wrap': None, 'accelerator_config': {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}, 'deepspeed': None, 'label_smoothing_factor': 0.0, 'optim': 'paged_adamw_8bit', 'optim_args': None, 'adafactor': False, 'group_by_length': False, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'ddp_bucket_cap_mb': None, 'ddp_broadcast_buffers': None, 'dataloader_pin_memory': True, 'dataloader_persistent_workers': False, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': False, 'resume_from_checkpoint': None, 'hub_model_id': None, 'hub_strategy': 'every_save', 'hub_token': '<HUB_TOKEN>', 'hub_private_repo': False, 'hub_always_push': False, 'gradient_checkpointing': False, 'gradient_checkpointing_kwargs': None, 'include_inputs_for_metrics': False, 'eval_do_concat_batches': True, 'fp16_backend': 'auto', 'evaluation_strategy': None, 'push_to_hub_model_id': None, 'push_to_hub_organization': None, 'push_to_hub_token': '<PUSH_TO_HUB_TOKEN>', 'mp_parameters': '', 'auto_find_batch_size': False, 'full_determinism': False, 'torchdynamo': None, 'ray_scope': 'last', 'ddp_timeout': 1800, 'torch_compile': False, 'torch_compile_backend': None, 'torch_compile_mode': None, 'dispatch_batches': None, 'split_batches': None, 'include_tokens_per_second': False, 'include_num_input_tokens_seen': False, 'neftune_noise_alpha': None, 'optim_target_modules': None, 'batch_eval_metrics': False}
+2024-06-26 07:18:26,851 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:26,851 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:50,892 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:50,915 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:50,915 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:51,909 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:51,913 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:51,913 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:52,853 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:52,855 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:52,855 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:53,667 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:53,670 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:53,670 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:54,691 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:54,756 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:54,756 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:55,617 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:55,682 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:55,682 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:56,512 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:56,513 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:56,513 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:57,999 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:58,001 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:58,001 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:18:58,997 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:18:59,009 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:18:59,009 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:19:00,090 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:19:00,092 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:19:00,092 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:19:01,722 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:19:01,723 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:19:01,723 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:19:03,098 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:19:03,117 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:19:03,117 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:19:04,117 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:19:04,212 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:19:04,212 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:19:05,065 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:19:05,097 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:19:05,097 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:19:06,209 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:19:06,220 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 07:19:06,220 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 07:19:07,300 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 07:19:07,961 INFO    MainThread:34 [wandb_run.py:_config_callback():1376] config_cb None None {'vocab_size': 32000, 'max_position_embeddings': 32768, 'hidden_size': 4096, 'intermediate_size': 14336, 'num_hidden_layers': 32, 'num_attention_heads': 32, 'sliding_window': None, 'num_key_value_heads': 8, 'hidden_act': 'silu', 'initializer_range': 0.02, 'rms_norm_eps': 1e-05, 'use_cache': False, 'rope_theta': 1000000.0, 'attention_dropout': 0.0, 'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': 'bfloat16', 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': False, 'chunk_size_feed_forward': 0, 'is_encoder_decoder': False, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 20, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 1, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': None, 'num_return_sequences': 1, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': None, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': ['MistralForCausalLM'], 'finetuning_task': None, 'id2label': {0: 'LABEL_0', 1: 'LABEL_1'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1}, 'tokenizer_class': None, 'prefix': None, 'bos_token_id': 1, 'pad_token_id': 0, 'eos_token_id': 2, 'sep_token_id': None, 'decoder_start_token_id': None, 'task_specific_params': None, 'problem_type': None, '_name_or_path': 'TheBloke/Mistral-7B-Instruct-v0.2-GPTQ', 'transformers_version': '4.41.2', 'model_type': 'mistral', 'pretraining_tp': 1, 'quantization_config': {'quant_method': 'QuantizationMethod.GPTQ', 'bits': 4, 'tokenizer': None, 'dataset': None, 'group_size': 128, 'damp_percent': 0.1, 'desc_act': True, 'sym': True, 'true_sequential': True, 'use_cuda_fp16': False, 'model_seqlen': None, 'block_name_to_quantize': None, 'module_name_preceding_first_block': None, 'batch_size': 1, 'pad_token_id': None, 'use_exllama': True, 'max_input_length': None, 'exllama_config': {'version': 'ExllamaVersion.ONE'}, 'cache_block_outputs': True, 'modules_in_block_to_quantize': None}, 'output_dir': '/kaggle/working/', 'overwrite_output_dir': False, 'do_train': False, 'do_eval': True, 'do_predict': False, 'eval_strategy': 'epoch', 'prediction_loss_only': False, 'per_device_train_batch_size': 4, 'per_device_eval_batch_size': 4, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 4, 'eval_accumulation_steps': None, 'eval_delay': 0, 'learning_rate': 0.0002, 'weight_decay': 0.01, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam_epsilon': 1e-08, 'max_grad_norm': 1.0, 'num_train_epochs': 10, 'max_steps': -1, 'lr_scheduler_type': 'linear', 'lr_scheduler_kwargs': {}, 'warmup_ratio': 0.0, 'warmup_steps': 2, 'log_level': 'passive', 'log_level_replica': 'warning', 'log_on_each_node': True, 'logging_dir': '/kaggle/working/runs/Jun26_07-19-05_adc95cf38b20', 'logging_strategy': 'epoch', 'logging_first_step': False, 'logging_steps': 500, 'logging_nan_inf_filter': True, 'save_strategy': 'epoch', 'save_steps': 500, 'save_total_limit': None, 'save_safetensors': True, 'save_on_each_node': False, 'save_only_model': False, 'restore_callback_states_from_checkpoint': False, 'no_cuda': False, 'use_cpu': False, 'use_mps_device': False, 'seed': 42, 'data_seed': None, 'jit_mode_eval': False, 'use_ipex': False, 'bf16': False, 'fp16': True, 'fp16_opt_level': 'O1', 'half_precision_backend': 'auto', 'bf16_full_eval': False, 'fp16_full_eval': False, 'tf32': None, 'local_rank': 0, 'ddp_backend': None, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': None, 'dataloader_num_workers': 0, 'dataloader_prefetch_factor': None, 'past_index': -1, 'run_name': '/kaggle/working/', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': True, 'metric_for_best_model': 'loss', 'greater_is_better': False, 'ignore_data_skip': False, 'fsdp': [], 'fsdp_min_num_params': 0, 'fsdp_config': {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, 'fsdp_transformer_layer_cls_to_wrap': None, 'accelerator_config': {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}, 'deepspeed': None, 'label_smoothing_factor': 0.0, 'optim': 'paged_adamw_8bit', 'optim_args': None, 'adafactor': False, 'group_by_length': False, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'ddp_bucket_cap_mb': None, 'ddp_broadcast_buffers': None, 'dataloader_pin_memory': True, 'dataloader_persistent_workers': False, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': False, 'resume_from_checkpoint': None, 'hub_model_id': None, 'hub_strategy': 'every_save', 'hub_token': '<HUB_TOKEN>', 'hub_private_repo': False, 'hub_always_push': False, 'gradient_checkpointing': False, 'gradient_checkpointing_kwargs': None, 'include_inputs_for_metrics': False, 'eval_do_concat_batches': True, 'fp16_backend': 'auto', 'evaluation_strategy': None, 'push_to_hub_model_id': None, 'push_to_hub_organization': None, 'push_to_hub_token': '<PUSH_TO_HUB_TOKEN>', 'mp_parameters': '', 'auto_find_batch_size': False, 'full_determinism': False, 'torchdynamo': None, 'ray_scope': 'last', 'ddp_timeout': 1800, 'torch_compile': False, 'torch_compile_backend': None, 'torch_compile_mode': None, 'dispatch_batches': None, 'split_batches': None, 'include_tokens_per_second': False, 'include_num_input_tokens_seen': False, 'neftune_noise_alpha': None, 'optim_target_modules': None, 'batch_eval_metrics': False}
+2024-06-26 08:35:48,386 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:35:48,387 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:36:11,677 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:38:19,575 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:38:19,575 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:39:55,100 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:39:55,103 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:39:55,103 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:40:32,017 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:43:02,593 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:43:02,594 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:45:10,959 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:45:10,963 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:45:10,963 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:45:13,384 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:47:43,994 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:47:43,994 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:49:58,760 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:53:33,771 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:53:33,771 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:54:32,055 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:54:32,059 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:54:32,059 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 08:54:32,754 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 08:58:07,756 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 08:58:07,756 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 09:12:46,632 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 09:12:46,669 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 09:12:46,669 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 09:16:22,037 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 09:16:22,038 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 09:16:22,038 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 09:16:35,228 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 09:16:36,438 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 09:16:36,438 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 09:18:25,016 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 09:18:26,213 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 09:18:26,214 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 09:21:21,819 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 09:21:21,842 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 09:21:21,842 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 09:21:36,632 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend
+2024-06-26 09:21:36,633 INFO    MainThread:34 [jupyter.py:save_ipynb():373] not saving jupyter notebook
+2024-06-26 09:21:36,634 INFO    MainThread:34 [wandb_init.py:_pause_backend():431] pausing backend
+2024-06-26 09:21:37,142 INFO    MainThread:34 [wandb_init.py:_resume_backend():436] resuming backend