GGUF
Inference Endpoints
conversational
File size: 1,572 Bytes
400e138
7cbc35a
400e138
 
 
 
89f3e10
400e138
 
 
 
 
5d61621
 
 
 
400e138
 
 
 
f934bf9
 
5d61621
f934bf9
 
400e138
 
 
f934bf9
 
 
 
 
400e138
 
eed3c4d
400e138
89f3e10
eed3c4d
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
---
license: ms-pl
---

## Overview

The [Phi-3](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct), state-of-the-art open model trained with the Phi-3 datasets that includes both synthetic data and the filtered publicly available websites data with a focus on high-quality and reasoning dense properties. The model belongs to the Phi-3 family with the 4B, 7B version in two variants 8K and 128K which is the context length (in tokens) that it can support.

## Variants

| No | Variant | Cortex CLI command |
| --- | --- | --- |
| 1 | [mini-onnx](https://huggingface.co/cortexso/phi3/tree/mini-onnx) | `cortex run phi3:mini-onnx` |
| 2 | [mini-gguf](https://huggingface.co/cortexso/phi3/tree/mini-gguf) | `cortex run phi3:mini-gguf` |
| 3 | [medium-gguf](https://huggingface.co/cortexso/phi3/tree/medium-gguf) | `cortex run phi3:medium-gguf` |
| 4 | [medium-onnx](https://huggingface.co/cortexso/phi3/tree/medium-onnx) | `cortex run phi3:medium-onnx` |

## Use it with Jan (UI)

1. Install **Jan** using [Quickstart](https://jan.ai/docs/quickstart)
2. Use in Jan model Hub:
    ```
    cortexso/phi3
    ```
    
## Use it with Cortex (CLI)

1. Install **Cortex** using [Quickstart](https://cortex.jan.ai/docs/quickstart)
2. Run the model with command:
    ```
    cortex run phi3
    ```
    
## Credits

- **Author:** Microsoft
- **Converter:** [Homebrew](https://www.homebrew.ltd/)
- **Original License:** [License](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/blob/main/LICENSE)
- **Papers:** [Phi-3 Technical Report](https://arxiv.org/abs/2404.14219)