anemll commited on
Commit
72c85fc
·
verified ·
1 Parent(s): b533bcf

Update README.md

Browse files

Mark up corrections, added unzip

Files changed (1) hide show
  1. README.md +57 -47
README.md CHANGED
@@ -1,69 +1,79 @@
1
- ---
2
- license: mit
3
- tags:
4
- - coreml
5
- - ANE
6
- - DeepSeek
7
- - Apple
8
- ---
9
- ANEMLL
10
-
11
- ANEMLL (pronounced like “animal”) is an open-source project
12
- focused on accelerating the porting of Large Language Models (LLMs)
13
- to tensor processors, starting with the Apple Neural Engine (ANE).
14
-
15
- The goal is to provide a fully open-source pipeline
16
- from model conversion to inference for common LLM architectures
17
- running on ANE.
18
-
19
- This enables seamless integration and on-device inference
20
- for low-power applications on edge devices,
21
- ensuring maximum privacy and security.
22
-
23
- This is critical for autonomous applications,
24
- where models run directly on the device
25
- without requiring an internet connection.
26
-
27
- License
28
-
29
- ANEMLL is licensed under the MIT License.
30
- https://opensource.org/license/mit
31
- The model is based on Meta’s LLaMA 3.1 8B architecture and may require a separate license.
32
 
 
33
 
 
 
 
 
 
 
34
 
35
- This test model is exclusively for the DeepSeek R1 8B model converted for CoreML,
36
- released before the official launch of the ANEMLL repository and minimal documentation.
37
  It is intended for early adopters only who requested an early release.
38
 
 
39
 
40
- Requirements
41
- • macOS Sequoia with Apple Neural Engine and 16GB RAM
42
- • CoreML Tools and HuggingFace Transformers libraries
43
- • Python 3.9
44
 
45
- chat.py provides a sample inference script.
46
- We apologize for the current quality of chat.py and appreciate your patience.
 
47
 
 
 
48
 
49
- Prerequisites:
 
 
 
 
 
 
 
 
50
  pip install coremltools transformers
 
 
 
51
 
52
- How to RUN:
53
  python chat.py
54
 
55
  Ctr-D to exit, Ctr-C to interrupt inference.
56
 
57
- alternative way to run:
 
 
58
  python chat.py Q123 -d /path/to/anemll-DeepSeek-8B-ctx1024 ctx=1024
59
 
 
 
60
  The first time the model loads, macOS will take some time to place it on the device.
61
  Subsequent loads will be instantaneous.
62
 
 
 
63
  Please check following links for later updates:
64
- https://huggingface.co/anemll
65
- https://x.com/anemll
66
- https://github.com/anemll
67
- https://anemll.com
68
 
69
- realanemll@gmail.com
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - coreml
5
+ - ANE
6
+ - DeepSeek
7
+ - Apple
8
+ - Apple Neural Engine
9
+ ---
10
+ # ANEMLL
11
+
12
+ **ANEMLL** (pronounced like “animal”) is an open-source project focused on accelerating the porting of Large Language Models (LLMs) to tensor processors, starting with the Apple Neural Engine (ANE).
13
+
14
+ The goal is to provide a fully open-source pipeline from model conversion to inference for common LLM architectures running on ANE.
15
+
16
+ This enables seamless integration and on-device inference for low-power applications on edge devices, ensuring maximum privacy and security.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
17
 
18
+ This is critical for autonomous applications, where models run directly on the device without requiring an internet connection.
19
 
20
+ ---
21
+
22
+ ## License
23
+
24
+ ANEMLL is licensed under the [MIT License](https://opensource.org/license/mit).
25
+ The model is based on Meta’s LLaMA 3.1 8B architecture and may require a separate license.
26
 
27
+ This test model is exclusively for the Meta's LLaMA 3.2 1B (1024 context) model converted for CoreML,
28
+ released before the official launch of the ANEMLL repository and minimal documentation.
29
  It is intended for early adopters only who requested an early release.
30
 
31
+ ---
32
 
33
+ ## Requirements
 
 
 
34
 
35
+ - **macOS Sequoia** with Apple Neural Engine and 16GB RAM
36
+ - **CoreML Tools** and **HuggingFace Transformers** libraries
37
+ - **Python 3.9**
38
 
39
+ `chat.py` provides a sample inference script.
40
+ *We apologize for the current quality of `chat.py` and appreciate your patience.*
41
 
42
+ **Installation**
43
+
44
+ Unzip all ZIP files with CoreML Models files using Finder or via bash
45
+ ```bash
46
+ cd ./anemll-DeepSeek-8B-ctx1024
47
+ find . -type f -name "*.zip" -exec unzip {} \;
48
+ ```
49
+
50
+ ```bash
51
  pip install coremltools transformers
52
+ ```
53
+
54
+ **How to RUN:**
55
 
 
56
  python chat.py
57
 
58
  Ctr-D to exit, Ctr-C to interrupt inference.
59
 
60
+ **Alternative way to run:**
61
+
62
+ ```bash
63
  python chat.py Q123 -d /path/to/anemll-DeepSeek-8B-ctx1024 ctx=1024
64
 
65
+ ```
66
+
67
  The first time the model loads, macOS will take some time to place it on the device.
68
  Subsequent loads will be instantaneous.
69
 
70
+
71
+ ** More Info **
72
  Please check following links for later updates:
 
 
 
 
73
 
74
+ • https://huggingface.co/anemll
75
+ • https://x.com/anemll
76
+ • https://github.com/anemll
77
+ • https://anemll.com
78
+
79