Revert expected serverless output metadata stripper
c013599
Luke Stanley
commited on
Spaces:
Sleeping
Sleeping
Commit History
Avoid unneeded imports, make serverless output more sensible, removing some debugging and comments
469f650
Luke Stanley
commited on
Fix RUNPOD_ENDPOINT_ID environment variable
ce5ad5f
Luke Stanley
commited on
Introduces worker mode env var
56e785c
Luke Stanley
commited on
Ensure N_GPU_LAYERS is int
9475016
Luke Stanley
commited on
Update default GPU layer, temperature values
e327a9e
Add env vars to set GPU layer count and context size, make verbose
e01e28e
Add n_gpu_layers parameter to Llama initialization
88e6118
Fix: Move n_ctx parameter to model setup!
358cd20
Fix check for LLM_MODEL_PATH to avoid load error
ff938c3
Auto-downloads model if env var is not set
74d6e52
Luke Stanley
commited on
Make llm_stream_sans_network actually stream to stdout
a0f49a0
Luke Stanley
commited on
Default to in-memory LLM interface
ddb0d91
Luke Stanley
commited on