Mrw33554432
/

bitLinear-phi-1.5

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Mrw33554432 commited on Apr 23

Commit

bf9d895

•

1 Parent(s): 9d04c92

Update README.md

Files changed (1) hide show

README.md +3 -1

README.md CHANGED Viewed

@@ -7,7 +7,9 @@ datasets:
 BitLinear-phi-1.5 is a model trained partially using the method described in [The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits](https://arxiv.org/abs/2402.17764).
-Our BitLinear layer will only apply 1-bit quantization to the weight, all other computations in the paper is discarded.
 The model structure is from [phi-1.5](https://huggingface.co/microsoft/phi-1_5), with all linear layers except lm_head replaced with our custom BitLinear layer.

 BitLinear-phi-1.5 is a model trained partially using the method described in [The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits](https://arxiv.org/abs/2402.17764).
+### Notice: Our BitLinear layer will only apply 1-bit quantization to the weight
+### Other components (RMSnorm, activation quant) in the paper is discarded.
+Idea behind: The major contribution in their paper is introduced a valid binary weight quantization, we don't want to mix it with other components to make it difficult to evaluate the major part.
 The model structure is from [phi-1.5](https://huggingface.co/microsoft/phi-1_5), with all linear layers except lm_head replaced with our custom BitLinear layer.