Switch to serverless worker by default (PR #2 from lukestanley/serverless_json_llm)

Revert expected serverless output metadata stripper

Documents serverless motivation and testing instructions

Avoid unneeded imports, make serverless output more sensible, removing some debugging and comments

Fix RUNPOD_ENDPOINT_ID environment variable

Add more serverless GPU endpoint setup instruction detail

Document serverless setup

Rename serverless test file, set default model to Phi 2 for test, removed jq install, and env vars that are set the same in utils.py already, ignore .cache in git

Introduces worker mode env var

Make GPU detection and llama-cpp-python re-installation conditional

Initialise global variables in improvement_loop function

Ensure N_GPU_LAYERS is int

Expose json typed LLM interface for RunPod

RunPod Mixtral JSON output test

Add hello world RunPod setup

Update default GPU layer, temperature values

Add env vars to set GPU layer count and context size, make verbose

Fix gif link since LFS related gif binary purge due to HF requirments

Add n_gpu_layers parameter to Llama initialization

Fix: Move n_ctx parameter to model setup!

Fix check for LLM_MODEL_PATH to avoid load error

Correct Space metadata

Add HuggingFace space metadata

Adds Gradio app wrapper and Dockerfile

Auto-downloads model if env var is not set

Make llm_stream_sans_network actually stream to stdout

Default to in-memory LLM interface

Updates README with some realist hope

Bug fix for undefined last_edit

Add comment about adaptability

Documents command line usage and module import functionality

Print new line after LLM output end and some linting

Make reusable via CLI, and modue, moved core logic to improvement_loop.

Makes URL more obvious, update comments, lowers temp

Update comments and remove unused code

Make goal more clear

Removes unused line and tidying

Move prompt strings and types to own file, reorder code a bit

Move prompts to own file

Linting with Black

Removes unused code

Emoji refactor

Add more detailed setup notes with GPU, fork, and other pip dependencies

Adds demo gif

Add clone example

Removed unused lines

Slight refactor tidying

Emoji changes

Minor changes, typo fixes

Adds README