phi-4-Q3_K_S-GGUF

Original Model: /microsoft/phi-4
Quantization Method: GGUF

Overview

This is an GGUF Q5 KM quantized version of phi-4.

Quantization By

I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models.

I hope the community finds these quantizations useful.

Andrew Webby @ RolePlai

Downloads last month: 6

GGUF

Model size

14.7B params

Architecture

phi3

5-bit

Inference Examples

Text Generation

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for roleplaiapp/phi-4-Q5_K_M-GGUF

Base model

microsoft/phi-4

Quantized

(65)

this model