Phi-4 mini does not work inside of unsloth.
The Phi-4 mini release is very promising, but sadly it cannot load inside of the Unsloth framework: "RuntimeError: rope_scaling's short_factor field must have length 64, got 48".
Will unsloth release a fixed version possibly?
seems like the modelling_phi3.py is not included
The Phi-4 mini release is very promising, but sadly it cannot load inside of the Unsloth framework: "RuntimeError:
rope_scaling's short_factor field must have length 64, got 48".Will unsloth release a fixed version possibly?
seems like the modelling_phi3.py is not included
unfortunately doesnt work atm in any framework. doesnt work in unsloth, ollama, llama.cpp etc because of new arch
will update u guys when it does
The architecture isn’t even particularly new, just none of these frameworks respect the “partial_rotary_factor” config line. (Only 3/4 of the embeddings are subject to RoPE, presumably to weight latest context more heavily than long context). I took a crack at adding it to Exllamav2, and while quantization now appears to work inference is wildly broken. Guess it’ll take a while before we see this in a usable state if all the upstream packages need to be updated to support it.
Hello @shimmyshimmer
We already added this 'partial_rotary_factor' support to the latest HF and VLLM before the release.
The new model feature is added to the latest HF(v4.49.0) and vllm (v0.7.3) already.
Can you take a look at the PRs?
They are relatively simple if the new config is utilized.
VLLM: https://github.com/vllm-project/vllm/pull/12718
HF: https://github.com/huggingface/transformers/pull/35947