Update README.md

README.md

@@ -10,11 +10,10 @@ model-index:
   results: []
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
 [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
-<details><summary>See axolotl config</summary>
+
+
+# Axolotl Config 
 
 axolotl version: `0.3.0`
 ```yaml
@@ -107,75 +106,65 @@ special_tokens:
   eos_token: "</s>"
   unk_token: "<unk>"
 save_safetensors: true
+```
+
+# hc-mistral-alpaca
 
+This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1).
+
+See this [wandb run](https://wandb.ai/hamelsmu/hc-axolotl-mistral/runs/7dq9l9vu/overview) to see training metrics.
+
+# Usage
+
+You can use this model with the following code:
+
+First, download the model 
+
+```python
+from peft import AutoPeftModelForCausalLM
+from transformers import AutoTokenizer
+model_id='parlance-labs/hc-mistral-alpaca'
+model = AutoPeftModelForCausalLM.from_pretrained(model_id).cuda()
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+tokenizer.pad_token = tokenizer.eos_token
 ```
 
-</details><br>
+Then, construct the prompt template like so:
 
-# hc-mistral-alpaca
+```python
+def prompt(nlq, cols):
+    return f"""Honeycomb is an observability platform that allows you to write queries to inspect trace data. You are an assistant that takes a natural language query (NLQ) and a list of valid columns and produce a Honeycomb query.
+
+### Instruction:
+
+NLQ: "{nlq}"
+
+Columns: {cols}
+
+### Response:
+"""
+
+def prompt_tok(nlq, cols):
+    _p = prompt(nlq, cols)
+    input_ids = tokenizer(_p, return_tensors="pt", truncation=True).input_ids.cuda()
+    out_ids = model.generate(input_ids=input_ids, max_new_tokens=5000, 
+                          do_sample=False)
+    return tokenizer.batch_decode(out_ids.detach().cpu().numpy(), 
+                                  skip_special_tokens=True)[0][len(_p):]
+```
+
+Finally, you can get predictions like this:
+
+```python
+nlq = "Exception count by exception and caller"
+cols = ['error', 'exception.message', 'exception.type', 'exception.stacktrace', 'SampleRate', 'name', 'db.user', 'type', 'duration_ms', 'db.name', 'service.name', 'http.method', 'db.system', 'status_code', 'db.operation', 'library.name', 'process.pid', 'net.transport', 'messaging.system', 'rpc.system', 'http.target', 'db.statement', 'library.version', 'status_message', 'parent_name', 'aws.region', 'process.command', 'rpc.method', 'span.kind', 'serializer.name', 'net.peer.name', 'rpc.service', 'http.scheme', 'process.runtime.name', 'serializer.format', 'serializer.renderer', 'net.peer.port', 'process.runtime.version', 'http.status_code', 'telemetry.sdk.language', 'trace.parent_id', 'process.runtime.description', 'span.num_events', 'messaging.destination', 'net.peer.ip', 'trace.trace_id', 'telemetry.instrumentation_library', 'trace.span_id', 'span.num_links', 'meta.signal_type', 'http.route']
+```
+
+Alternatively, you can play with this model on Replicate: [hamelsmu/honeycomb-2](https://replicate.com/hamelsmu/honeycomb-2)
+
+# Hosted Inference
 
-This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the None dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.0283
-
-## Model description
-
-More information needed
-
-## Intended uses & limitations
-
-More information needed
-
-## Training and evaluation data
-
-More information needed
-
-## Training procedure
-
-
-The following `bitsandbytes` quantization config was used during training:
-- quant_method: bitsandbytes
-- load_in_8bit: False
-- load_in_4bit: True
-- llm_int8_threshold: 6.0
-- llm_int8_skip_modules: None
-- llm_int8_enable_fp32_cpu_offload: False
-- llm_int8_has_fp16_weight: False
-- bnb_4bit_quant_type: nf4
-- bnb_4bit_use_double_quant: True
-- bnb_4bit_compute_dtype: bfloat16
-
-### Training hyperparameters
-
-The following hyperparameters were used during training:
-- learning_rate: 0.0002
-- train_batch_size: 16
-- eval_batch_size: 16
-- seed: 49
-- gradient_accumulation_steps: 4
-- total_train_batch_size: 64
-- optimizer: Adam with betas=(0.9,0.95) and epsilon=1e-05
-- lr_scheduler_type: cosine
-- lr_scheduler_warmup_steps: 20
-- num_epochs: 3
-
-### Training results
-
-| Training Loss | Epoch | Step | Validation Loss |
-|:-------------:|:-----:|:----:|:---------------:|
-| 1.167         | 0.0   | 1    | 1.1641          |
-| 0.0518        | 0.25  | 450  | 0.0565          |
-| 0.0386        | 0.5   | 900  | 0.0414          |
-| 0.0314        | 0.75  | 1350 | 0.0357          |
-| 0.0278        | 1.0   | 1800 | 0.0326          |
-| 0.0217        | 1.25  | 2250 | 0.0311          |
-| 0.0206        | 1.5   | 2700 | 0.0296          |
-| 0.0175        | 1.75  | 3150 | 0.0285          |
-| 0.0271        | 2.0   | 3600 | 0.0274          |
-| 0.0146        | 2.25  | 4050 | 0.0288          |
-| 0.0126        | 2.5   | 4500 | 0.0284          |
-| 0.0168        | 2.75  | 4950 | 0.0284          |
-| 0.0118        | 3.0   | 5400 | 0.0283          |
+This model is hosted on Replicate: (hamelsmu/honeycomb-2)[https://replicate.com/hamelsmu/honeycomb-2], using [this config](https://github.com/hamelsmu/replicate-examples/tree/master/mistral-transformers-2).
 
 
 ### Framework versions