doberst commited on
Commit
2803d93
1 Parent(s): 8a83062

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +128 -0
README.md CHANGED
@@ -1,3 +1,131 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+
5
+ # Model Card for Model ID
6
+
7
+ <!-- Provide a quick summary of what the model is/does. -->
8
+
9
+ bling-falcon-1b-0.1 is part of the BLING ("Best Little Instruction-following No-GPU-required") model series, instruct trained on top of a falcon-rw-1b base model.
10
+
11
+ BLING models are fine-tuned with distilled high-quality custom instruct datasets, targeted at a specific subset of instruct tasks with
12
+ the objective of providing a high-quality Instruct model that is 'inference-ready' on a CPU laptop even
13
+ without using any advanced quantization optimizations.
14
+
15
+ ### **PERFORMANCE on BASIC RAG TEST DATASET**
16
+
17
+ | Model | Params (B) | Sourcing | GPU/CPU | Output Tokens | Out as % of Input | Process Time (secs) | Score (0-100) |
18
+ | :---------- | :--------: | :----: | :-----: | :---------: | :-------: | :--------: | :-------: |
19
+ | gpt-4 | <=1000 | Closed | Multi-GPU | 2665 | 10.53% | 183.8 | 100 |
20
+ | gpt-3.5-turbo-instruct| <=175 | Closed | Multi-GPU | 2621 | 11.49% | 62.7 | 100 |
21
+ | claude-instant-v1 | <=50 | Closed | Multi-GPU | 6337 | 26.50% | 154 | 100 |
22
+ | aib-read-gpt | 7 | Closed | GPU | 1964 | 9.30% | 114 | 96 |
23
+ | **bling_falcon-1b-0.1** | **1.3** | **Open** | **CPU** | **3204** | **14.55%** | **696** | **77** |
24
+ | bling_pythia-1.4b-0.1 | 1.4 | Open | CPU | 2589 | 11.75% | 593.5 | 65 |
25
+ | bling_pythia-1b-0.1 | 1.0 | Open | CPU | 2753 | 12.49% | 428 | 59 |
26
+ | bling_cerebras-1.3b | 1.3 | Open | CPU | 3202 | 20.01% | 690.1 | 52 |
27
+ | bling_pythia_410m | 0.41 | NA | CPU | 2349 | 10.66% | 189 | 36 |
28
+ | bling_cerebras_590m | 0.59 | NA | CPU | 4407 | 20.01% | 400.8 | 30 |
29
+
30
+ For more details on this evaluation, please see the dataset: **llmware/rag_instruct_test_dataset_0.1** and [BLOG](https://medium.com/@darrenoberst/evaluating-llm-performance-in-rag-instruct-use-cases-083dc272a31d)
31
+
32
+
33
+ ### Model Description
34
+
35
+ <!-- Provide a longer summary of what this model is. -->
36
+
37
+ - **Developed by:** llmware
38
+ - **Model type:** GPTNeoX instruct-trained decoder
39
+ - **Language(s) (NLP):** English
40
+ - **License:** Apache 2.0
41
+ - **Finetuned from model [optional]:** tiiuae/falcon-rw-1b
42
+
43
+ ## Uses
44
+
45
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
46
+
47
+ The intended use of BLING models is two-fold:
48
+
49
+ 1. Provide high-quality Instruct models that can run on a laptop for local testing. We have found it extremely useful when building a
50
+ proof-of-concept, or working with sensitive enterprise data that must be closely guarded, especially in RAG use cases.
51
+
52
+ 2. Push the state of the art for smaller Instruct-following models in the sub-7B parameter range, especially 1B-3B, as single-purpose
53
+ automation tools for specific tasks through targeted fine-tuning datasets and focused "instruction" tasks.
54
+
55
+
56
+ ### Direct Use
57
+
58
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
59
+
60
+ BLING is designed for enterprise automation use cases, especially in knowledge-intensive industries, such as financial services,
61
+ legal and regulatory industries with complex information sources. Rather than try to be "all things to all people," BLING models try to focus on a narrower set of Instructions more suitable to a ~1B parameter GPT model.
62
+
63
+ BLING is ideal for rapid prototyping, testing, and the ability to perform an end-to-end workflow locally on a laptop without
64
+ having to send sensitive information over an Internet-based API.
65
+
66
+ The first BLING models have been trained for common RAG scenarios, specifically: question-answering, key-value extraction, and basic summarization as the core instruction types
67
+ without the need for a lot of complex instruction verbiage - provide a text passage context, ask questions, and get clear fact-based responses.
68
+
69
+
70
+ ### Out-of-Scope Use
71
+
72
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
73
+
74
+ 1. BLING is not designed for 'chat-bot' or 'consumer-oriented' applications.
75
+
76
+ 2. BLING is not optimal for most production applications, other than simple and highly specific use cases.
77
+
78
+
79
+ ## Bias, Risks, and Limitations
80
+
81
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
82
+
83
+ BLING has not been designed for end consumer-oriented applications, and there has not been any focus in training on safeguards to mitigate potential bias. We would strongly discourage any use of BLING for any 'chatbot' use case.
84
+
85
+
86
+ ## How to Get Started with the Model
87
+
88
+ The fastest way to get started with BLING is through direct import in transformers:
89
+
90
+ from transformers import AutoTokenizer, AutoModelForCausalLM
91
+ tokenizer = AutoTokenizer.from_pretrained("llmware/bling-falcon-1b-0.1")
92
+ model = AutoModelForCausalLM.from_pretrained("llmware/bling-falcon-1b-0.1")
93
+
94
+
95
+ The BLING model was fine-tuned with a simple "\<human> and \<bot> wrapper", so to get the best results, wrap inference entries as:
96
+
97
+ full_prompt = "\<human>\: " + my_prompt + "\n" + "\<bot>\: "
98
+
99
+ The BLING model was fine-tuned with closed-context samples, which assume generally that the prompt consists of two sub-parts:
100
+
101
+ 1. Text Passage Context, and
102
+ 2. Specific question or instruction based on the text passage
103
+
104
+ To get the best results, package "my_prompt" as follows:
105
+
106
+ my_prompt = {{text_passage}} + "\n" + {{question/instruction}}
107
+
108
+
109
+ ## Citation [optional]
110
+
111
+ This BLING model was built on top of a Falcon model base - for more information about the Falcon model, please see the paper referenced below:
112
+
113
+ @article{refinedweb,
114
+ title={The {R}efined{W}eb dataset for {F}alcon {LLM}: outperforming curated corpora with web data, and web data only},
115
+ author={Guilherme Penedo and Quentin Malartic and Daniel Hesslow and Ruxandra Cojocaru and Alessandro Cappelli and Hamza Alobeidli and Baptiste Pannier and Ebtesam Almazrouei and Julien Launay},
116
+ journal={arXiv preprint arXiv:2306.01116},
117
+ eprint={2306.01116},
118
+ eprinttype = {arXiv},
119
+ url={https://arxiv.org/abs/2306.01116},
120
+ year={2023}
121
+ }
122
+
123
+
124
+ ## Model Card Contact
125
+
126
+ Darren Oberst & llmware team
127
+
128
+ Please reach out anytime if you are interested in this project and would like to participate and work with us!
129
+
130
+
131
+