r0ymanesco commited on
Commit
15bc23d
1 Parent(s): 0bcc6dd

update readme

Browse files
Files changed (1) hide show
  1. README.md +53 -0
README.md CHANGED
@@ -1,3 +1,56 @@
1
  ---
2
  license: gpl-3.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: gpl-3.0
3
  ---
4
+ # notdiamond-0001
5
+
6
+ notdiamond-0001 automatically determines whether to send queries to GPT-3.5 or GPT-4, depending on which model is best-suited for your task. We've trained notdiamond-0001 on hundreds of thousands of data points from robust, cross-domain evaluation benchmarks.
7
+
8
+ The model is a classifier and will return either GPT-3.5 or GPT-4. You determine which version of each model you want to use and make the calls client-side with your own keys.
9
+
10
+ To use notdiamond-0001, format your queries using the following prompt with your query appended at the end
11
+ ``` python
12
+ query = "Can you write a function that counts from 1 to 10?"
13
+
14
+ formatted_prompt = f"""I would like you to determine whether a given query should be sent to GPT-3.5 or GPT-4.
15
+ In general, the following types of queries should get sent to GPT-3.5:
16
+ Explanation
17
+ Summarization
18
+ Writing
19
+ Informal conversation
20
+ History, government, economics, literature, social studies
21
+ Simple math questions
22
+ Simple coding questions
23
+ Simple science questions
24
+
25
+ In general, the following types of queries should get sent to GPT-4:
26
+ Advanced coding questions
27
+ Advanced math questions
28
+ Advanced science questions
29
+ Legal questions
30
+ Medical questions
31
+ Sensitive/inappropriate queries
32
+
33
+ Your job is to determine whether the following query should be sent to GPT-3.5 or GPT-4.
34
+
35
+ Query:
36
+ {query}"""
37
+ ```
38
+
39
+
40
+ You can then determine the model to call as follows
41
+ ``` python
42
+ import torch
43
+ from transformers import AutoTokenizer, AutoModelForSequenceClassification
44
+
45
+ id2label = {0: 'gpt-3.5', 1: 'gpt-4'}
46
+ tokenizer = AutoTokenizer.from_pretrained("r0ymanesco/notdiamond-0001")
47
+ model = AutoModelForSequenceClassification.from_pretrained("r0ymanesco/notdiamond-0001")
48
+ max_length = self._get_max_length(model)
49
+
50
+ inputs = tokenizer(formatted_prompt, truncation=True, max_length=max_length, return_tensors="pt")
51
+ logits = model(**inputs).logits
52
+ model_id = logits.argmax().item()
53
+ model_to_call = id2label[model_id]
54
+ ```
55
+
56
+ For more details on how you can integrate this into your techstack and have notdiamond-0001 help you reduce latency and cost, check out our [documentation](https://notdiamond.readme.io/reference/introduction-1).