Upload 2 files

README.md

@@ -1,3 +1,57 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+language:
+- en
+tags:
+- mamba
+- mlx
+- cartesia
+---
+
+# Model Card for mamba2-780m-8bit-mlx
+
+This is an [MLX](https://ml-explore.github.io/mlx)-compatible version of the [mamba2-780m](https://huggingface.co/state-spaces/mamba2-780m) model, quantized to 8 bits. It uses the [EleutherAI/gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) tokenizer.
+For more details, see our [blog post](https://cartesia.ai/blog/on-device).
+
+## Usage
+### Installation
+This model requires the `cartesia-metal` and `cartesia-mlx` packages.
+
+Installation requires Xcode, which can be downloaded from https://developer.apple.com/xcode/. Accept the license agreement with:
+```shell 
+sudo xcodebuild -license
+```
+
+Install the required dependencies: the exact version of `nanobind`, followed by `cartesia-metal`, and finally `cartesia-mlx`, with the following commands:
+```shell 
+pip install nanobind@git+https://github.com/wjakob/nanobind.git@2f04eac452a6d9142dedb957701bdb20125561e4
+pip install git+https://github.com/cartesia-ai/edge.git#subdirectory=cartesia-metal
+pip install cartesia-mlx
+```
+
+Note: This package has been tested on macOS Sonoma 14.1 with the M3 chip.
+
+### Generation example
+```python 
+import mlx.core as mx
+import cartesia_mlx as cmx
+
+model = cmx.from_pretrained("cartesia-ai/mamba2-780m-8bit-mlx")
+model.set_dtype(mx.float32)   
+
+prompt = "Rene Descartes was"
+
+print(prompt, end="", flush=True)
+for text in model.generate(
+    prompt,
+    max_tokens=500,
+    eval_every_n=5,
+    verbose=True,
+    top_p=0.99,
+    temperature=0.85,
+):
+    print(text, end="", flush=True)
+```
+
+## About Cartesia
+At [Cartesia](https://cartesia.ai/), we're building real-time multimodal intelligence for every device.

mamba2-780m-8bit-mlx.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:370f6e52597809bc905dfd7017109bae3110ec2ae8733bc8aec344da7ac1315e
+size 984442444