Ensure N_GPU_LAYERS is int
9475016
Luke Stanley
commited on
Spaces:
Sleeping
Sleeping
Commit History
Update default GPU layer, temperature values
e327a9e
Add env vars to set GPU layer count and context size, make verbose
e01e28e
Add n_gpu_layers parameter to Llama initialization
88e6118
Fix: Move n_ctx parameter to model setup!
358cd20
Fix check for LLM_MODEL_PATH to avoid load error
ff938c3
Auto-downloads model if env var is not set
74d6e52
Luke Stanley
commited on
Make llm_stream_sans_network actually stream to stdout
a0f49a0
Luke Stanley
commited on
Default to in-memory LLM interface
ddb0d91
Luke Stanley
commited on