Create README.md

README.md

@@ -0,0 +1,128 @@
+---
+license: cc-by-sa-3.0
+datasets:
+- VMware/open-instruct-v1-oasst-dolly-hhrlhf
+language:
+- en
+library_name: transformers
+pipeline_tag: text-generation
+---
+
+# VMware/open-llama-7B-v2-open-instruct
+Instruction-tuned version of the fully trained Open LLama 7B v2  model. The model is open for <b>COMMERCIAL USE</b>. <br>
+
+- This model performs better on code compared to v1 due to the improvements made on the base model by the openlm-research team.
+- The instruction model is trained on an improved instruction tuning dataset compared to v1
+
+<b> NOTE </b> : The model was trained using the Alpaca prompt template
+<b> NOTE </b> : Fast tokenizer results in incorrect encoding, set the ```use_fast = False``` parameter, when instantiating the tokenizer
+
+## License
+- <b>Commercially Viable </b>
+
+- Open-instruct-v1
+- Mosaic/Dolly-HHRLHF + filtered  OASST1 - cc by 3.0 
+
+Subset of COT SUBMIX (FROM FLAN V2) Zeroshot examples
+- ESNLI -  MIT 
+- ECQA  - CDLA 1.0 - Sharing
+- Strategy  - MIT
+- CREAK  - MIT
+- gsmk8 - MIT
+- aqua  - MIT
+- qasc  - Apache 2.0
+- Language Model, ([openlm-research/open_llama_v2_7b](https://huggingface.co/openlm-research/open_llama_v2_7b)) is under apache-2.0
+
+
+## Nomenclature 
+
+- Model : Open-llama-v2
+- Model Size: 7B parameters
+- Dataset: Open-instruct(oasst,dolly, hhrlhf)
+
+## Use in Transformers
+
+```
+import os
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = 'VMware/open-llama-7b-open-instruct'
+
+
+tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=False)
+
+model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16, device_map='sequential')
+
+prompt_template = "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Response:"
+
+prompt = """What is attention mechanism of a transformer model? 
+ Write a python code to illustrate how attention works within a transformer model using numpy library. Donot use pytorch or tensorflow."""
+
+
+inputt = prompt_template.format(instruction= prompt)
+input_ids = tokenizer(inputt, return_tensors="pt").input_ids.to("cuda")
+
+output1 = model.generate(input_ids, max_length=512)
+input_length = input_ids.shape[1]
+output1 = output1[:, input_length:]
+output = tokenizer.decode(output1[0])
+
+print(output)
+
+'''
+Sure, I can help you with that!
+
+Attention mechanisms in transformer models are typically implemented using the attention mechanism in the self-attention layer. Self-attention allows the model to focus on different parts of the input sequence when processing it. This is achieved by computing a set of attention weights, which are used to weigh the contribution of each input element to the output.
+
+Here's an example code using NumPy to illustrate how attention works in a transformer model:
+
+```python
+import numpy as np
+
+def attention_weights(query, key, value, mask):
+    # Query, key, and value are input tensors. Mask is a tensor of zeros and ones that represents the attention mask.
+    # It is used to prevent the model from attending to certain positions in the input sequence if they are not relevant.
+    # The attention weights are the element-wise product of the query, key, and mask tensors.
+    # The result is a tensor of the same shape as the query tensor.
+    
+    # Compute the dot product between the query tensor and the key tensor
+    dot = np.matmul(query, key)
+    
+    # Compute the element-wise softmax of the dot product tensor
+    exp_dot = np.exp(dot)
+    
+    # Multiply the dot product and the softmax of the dot product tensors
+    weights = dot * exp_dot
+    
+    # Return the attention weights as a NumPy tensor
+    return weights
+
+# Define the input sequence
+query = np.array([[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]])
+key = np.array([[0.1, 0.2], [0.3, 0.4]])
+value = np.array([[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]])
+mask = np.array([[False, True, True], [False, True, True]])
+
+# Compute the attention weights
+weights = attention_weights(query, key, value, mask)
+
+# Print the attention weights
+print(weights)
+```
+
+In this example, the `attention_weights` function takes as input the query tensor, key tensor, value tensor, and mask tensor. It computes the dot product between the query and key tensors using the `np.matmul` function, and then applies a softmax function using the `np.exp` function to the element-wise dot product tensor. It then multiplies the dot product and softmax tensors using the `np.matmul` function, and returns the result as a NumPy tensor.
+
+The `query`, `key`, and `value` tensors represent the input sequence to the transformer model. The `mask` tensor represents the attention mask, which is used to prevent the model from attending to certain positions in the input sequence if they are not relevant.
+
+The output of the `attention_weights` function is a NumPy tensor that represents the attention weights for the input sequence. These weights are used by the transformer model to weigh the contribution of each input element to the output.
+
+I hope this helps!</s>
+'''
+```
+  
+## Finetuning details
+The finetuning scripts will be available in our [RAIL Github Repository](https://github.com/vmware-labs/research-and-development-artificial-intelligence-lab/tree/main/instruction-tuning)
+## Evaluation
+
+<B>TODO</B>