| # Persistent Memory Bot | |
| A chatbot that can remember all previous conversations. | |
| Useful for any application that requires an LM studio chatbot and functions identically to a traditional python call of a local AI Application. | |
| ## TO INSTALL: | |
| ``` | |
| Pip install flask install | |
| Pip3 install huggingface-hub | |
| CMAKE_ARGS="-DLLAMA_CUBLAS=on" pip install llama-cpp-python | |
| ``` | |
| ## Full docs: | |
| ``` | |
| # Base ctransformers with no GPU acceleration | |
| pip install llama-cpp-python | |
| # With NVidia CUDA acceleration | |
| CMAKE_ARGS="-DLLAMA_CUBLAS=on" pip install llama-cpp-python | |
| # Or with OpenBLAS acceleration | |
| CMAKE_ARGS="-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS" pip install llama-cpp-python | |
| # Or with CLBLast acceleration | |
| CMAKE_ARGS="-DLLAMA_CLBLAST=on" pip install llama-cpp-python | |
| # Or with AMD ROCm GPU acceleration (Linux only) | |
| CMAKE_ARGS="-DLLAMA_HIPBLAS=on" pip install llama-cpp-python | |
| # Or with Metal GPU acceleration for macOS systems only | |
| CMAKE_ARGS="-DLLAMA_METAL=on" pip install llama-cpp-python | |
| # In windows, to set the variables CMAKE_ARGS in PowerShell, follow this format; eg for NVidia CUDA: | |
| $env:CMAKE_ARGS = "-DLLAMA_OPENBLAS=on" | |
| pip install llama-cpp-python | |
| huggingface-cli download TheBloke/Silicon-Maid-7B-GGUF silicon-maid-7b.Q4_K_M.gguf --local-dir . --local-dir-use-symlinks False | |
| huggingface-cli download lmstudio-community/Meta-Llama-3-8B-Instruct-GGUF Meta-Llama-3-8B-Instruct-Q8_0.gguf --local-dir . --local-dir-use-symlinks False | |
| ``` | |