NeuML/language-id-quantized

Language Detection with StaticVectors

This model is an export of this FastText Language Identification model for staticvectors. staticvectors enables running inference in Python with NumPy. This helps it maintain solid runtime performance.

Language detection is an important task and identification with n-gram models is an efficient and highly accurate way to do it.

This model is a quantized version of the base language id model. It's using 2x256 Product Quantization like the original quantized model from FastText. This shrinks this model down to 4MB with only a minor hit on accuracy.

Usage with StaticVectors

from staticvectors import StaticVectors

model = StaticVectors("neuml/language-id-quantized")
model.predict(["What language is this text?"])