Spaces:

joaogante
/

assisted_generation_demo

Starting on T4

joaogante HF staff commited on 29 days ago

Commit

817dcba

•

1 Parent(s): c93ec2a

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -57,8 +57,7 @@ with gr.Blocks() as demo:
         "# 🤗 Assisted Generation Demo\n"
         f"- Model: {model_id} (INT8, ~7GB)\n"
         f"- Assistant Model: {assistant_id} (FP16, ~0.3GB)\n"
-        "- Running on a T4 GPU\n"
-        "- Best results are obtained when the model size difference in parameters is >10x"
     )
     with gr.Row():

         "# 🤗 Assisted Generation Demo\n"
         f"- Model: {model_id} (INT8, ~7GB)\n"
         f"- Assistant Model: {assistant_id} (FP16, ~0.3GB)\n"
+        "- Recipe for speedup: a) >10x model size difference in parameters; b) assistant trained similarly; c) CPU is not a bottleneck"
     )
     with gr.Row():