readme

README.md

@@ -1,6 +1,7 @@
 ---
 language: en
 tags:
+- distilbert
 - long context
 ---
 
@@ -9,6 +10,8 @@ tags:
 **This model relies on a custom modeling file, you need to add trust_remote_code=True**\
 **See [\#13467](https://github.com/huggingface/transformers/pull/13467)**
 
+Conversion script is available at this [link](https://github.com/ccdv-ai/convert_checkpoint_to_lsg).
+
 * [Usage](#usage)
 * [Parameters](#parameters)
 * [Sparse selection type](#sparse-selection-type)
@@ -20,8 +23,7 @@ This model is adapted from [distilbert-base-uncased](https://huggingface.co/dist
 
 This model can handle long sequences but faster and more efficiently than Longformer or BigBird (from Transformers) and relies on Local + Sparse + Global attention (LSG).
 
-
-The model requires sequences whose length is a multiple of the block size. The model is "adaptive" and automatically pads the sequences if needed (adaptive=True in config). It is however recommended, thanks to the tokenizer, to truncate the inputs (truncation=True) and optionally to pad with a multiple of the block size (pad_to_multiple_of=...). \
+The model requires sequences whose length is a multiple of the block size. The model is "adaptive" and automatically pads the sequences if needed (adaptive=True in config). It is however recommended, thanks to the tokenizer, to truncate the inputs (truncation=True) and optionally to pad with a multiple of the block size (pad_to_multiple_of=...). 
 
 
 Support encoder-decoder and causal masking but I didnt test it extensively.\