jeffreykthomas
/

llama2-7b-ubuntu-GPTQ

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

jeffreykthomas commited on Oct 24, 2023

Commit

e718166

·

1 Parent(s): f3c1668

Update handler.py

Files changed (1) hide show

handler.py +1 -1

handler.py CHANGED Viewed

@@ -19,7 +19,7 @@ class EndpointHandler:
             max_length=1024,
             max_new_tokens=250, do_sample=True, top_k=50,
             temperature=0.8, pad_token_id=2, num_return_sequences=1,
-            min_new_tokens=30, repetition_penalty=1.2, return_full_text: False
         )
         output = self.pipeline(inputs, **generation_config.to_dict())

             max_length=1024,
             max_new_tokens=250, do_sample=True, top_k=50,
             temperature=0.8, pad_token_id=2, num_return_sequences=1,
+            min_new_tokens=30, repetition_penalty=1.2, return_full_text=False
         )
         output = self.pipeline(inputs, **generation_config.to_dict())