Hugging Face's logo

youssefkhalil320
/
all-MiniLM-L6-v4-pairscore

Sentence Similarity
sentence-transformers
Safetensors
English
bert
feature-extraction
Generated from Trainer
dataset_size:1363306
loss:CoSENTLoss
text-embeddings-inference
Inference Endpoints
Model card Files Community
all-MiniLM-L6-v4-pairscore / README.md
youssefkhalil320's picture
youssefkhalil320
Upload folder using huggingface_hub
98bc080 verified
preview code
|
raw
history blame contribute delete
27.8 kB
metadata
base_model: sentence-transformers/all-MiniLM-L6-v2
language:
  - en
library_name: sentence-transformers
license: apache-2.0
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:1363306
  - loss:CoSENTLoss
widget:
  - source_sentence: labneh
    sentences:
      - iftar
      - bathing suit
      - coffee cup
  - source_sentence: Velvet flock Veil
    sentences:
      - mermaid purse
      - veil
      - mobile bag
  - source_sentence: Red lipstick
    sentences:
      - chemise dress
      - tote
      - rouge
  - source_sentence: Unisex Travel bag
    sentences:
      - spf
      - basic vega ring
      - travel backpack
  - source_sentence: jeremy hush book
    sentences:
      - chinese jumper
      - perfume
      - home automation device

all-MiniLM-L6-v4-pair_score

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: sentence-transformers/all-MiniLM-L6-v2
  • Maximum Sequence Length: 256 tokens
  • Output Dimensionality: 384 tokens
  • Similarity Function: Cosine Similarity
  • Language: en
  • License: apache-2.0

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'jeremy hush book',
    'chinese jumper',
    'perfume',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Hyperparameters

Non-Default Hyperparameters

  • eval_strategy: steps
  • per_device_train_batch_size: 128
  • per_device_eval_batch_size: 128
  • learning_rate: 2e-05
  • num_train_epochs: 4
  • warmup_ratio: 0.1
  • fp16: True

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: steps
  • prediction_loss_only: True
  • per_device_train_batch_size: 128
  • per_device_eval_batch_size: 128
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 2e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1.0
  • num_train_epochs: 4
  • max_steps: -1
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.1
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: False
  • fp16: True
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: False
  • hub_always_push: False
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • dispatch_batches: None
  • split_batches: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • use_liger_kernel: False
  • eval_use_gather_object: False
  • batch_sampler: batch_sampler
  • multi_dataset_batch_sampler: proportional

Training Logs

Click to expand
Epoch Step Training Loss loss
0.0094 100 17.0123 -
0.0188 200 16.3963 -
0.0282 300 14.9883 -
0.0376 400 12.5378 -
0.0469 500 9.8375 -
0.0563 600 8.4884 -
0.0657 700 8.2217 -
0.0751 800 8.1311 -
0.0845 900 8.104 -
0.0939 1000 8.0921 -
0.1033 1100 8.0568 -
0.1127 1200 8.0567 -
0.1221 1300 8.0534 -
0.1314 1400 8.0189 -
0.1408 1500 8.0172 -
0.1502 1600 8.0291 -
0.1596 1700 8.0396 -
0.1690 1800 8.0527 -
0.1784 1900 8.0543 -
0.1878 2000 8.0244 -
0.1972 2100 8.0294 -
0.2066 2200 8.019 -
0.2159 2300 7.9946 -
0.2253 2400 8.0233 -
0.2347 2500 8.0058 -
0.2441 2600 8.0146 -
0.2535 2700 8.0116 -
0.2629 2800 7.9843 -
0.2723 2900 8.0226 -
0.2817 3000 7.991 -
0.2911 3100 8.0041 -
0.3004 3200 8.025 -
0.3098 3300 7.9913 -
0.3192 3400 7.9852 -
0.3286 3500 8.0103 -
0.3380 3600 7.9911 -
0.3474 3700 7.9892 -
0.3568 3800 7.9605 -
0.3662 3900 8.011 -
0.3756 4000 7.9894 -
0.3849 4100 7.9658 -
0.3943 4200 7.9791 -
0.4037 4300 7.9717 -
0.4131 4400 8.0139 -
0.4225 4500 7.9691 -
0.4319 4600 8.0115 -
0.4413 4700 8.0245 -
0.4507 4800 8.0289 -
0.4601 4900 7.9644 -
0.4694 5000 7.9851 7.9703
0.4788 5100 7.9594 -
0.4882 5200 7.9618 -
0.4976 5300 7.9917 -
0.5070 5400 7.988 -
0.5164 5500 8.0203 -
0.5258 5600 7.9738 -
0.5352 5700 7.9614 -
0.5445 5800 7.9567 -
0.5539 5900 7.9721 -
0.5633 6000 7.96 -
0.5727 6100 7.9376 -
0.5821 6200 7.9901 -
0.5915 6300 7.9559 -
0.6009 6400 7.9548 -
0.6103 6500 8.0004 -
0.6197 6600 7.9607 -
0.6290 6700 7.9779 -
0.6384 6800 7.9401 -
0.6478 6900 7.9695 -
0.6572 7000 7.9667 -
0.6666 7100 7.9679 -
0.6760 7200 7.9821 -
0.6854 7300 7.9981 -
0.6948 7400 7.975 -
0.7042 7500 7.9438 -
0.7135 7600 7.9611 -
0.7229 7700 7.9501 -
0.7323 7800 7.9565 -
0.7417 7900 7.9199 -
0.7511 8000 7.9601 -
0.7605 8100 7.9208 -
0.7699 8200 7.9488 -
0.7793 8300 7.9519 -
0.7887 8400 7.9806 -
0.7980 8500 7.9557 -
0.8074 8600 7.9383 -
0.8168 8700 7.9541 -
0.8262 8800 7.9529 -
0.8356 8900 7.9463 -
0.8450 9000 7.9674 -
0.8544 9100 7.9454 -
0.8638 9200 7.9613 -
0.8732 9300 7.9119 -
0.8825 9400 7.9806 -
0.8919 9500 7.9449 -
0.9013 9600 7.9254 -
0.9107 9700 7.9156 -
0.9201 9800 7.9105 -
0.9295 9900 7.9668 -
0.9389 10000 7.9922 7.9137
0.9483 10100 7.9261 -
0.9577 10200 7.9134 -
0.9670 10300 7.8968 -
0.9764 10400 7.9086 -
0.9858 10500 7.9609 -
0.9952 10600 7.9125 -
1.0046 10700 7.8816 -
1.0140 10800 7.9558 -
1.0234 10900 7.9357 -
1.0328 11000 7.9212 -
1.0422 11100 7.9305 -
1.0515 11200 7.9073 -
1.0609 11300 7.9016 -
1.0703 11400 7.9321 -
1.0797 11500 7.8765 -
1.0891 11600 7.8907 -
1.0985 11700 7.9338 -
1.1079 11800 7.9163 -
1.1173 11900 7.8892 -
1.1267 12000 7.9261 -
1.1360 12100 7.8846 -
1.1454 12200 7.8976 -
1.1548 12300 7.8796 -
1.1642 12400 7.9041 -
1.1736 12500 7.9181 -
1.1830 12600 7.8944 -
1.1924 12700 7.9168 -
1.2018 12800 7.9122 -
1.2112 12900 7.9006 -
1.2205 13000 7.916 -
1.2299 13100 7.8994 -
1.2393 13200 7.8785 -
1.2487 13300 7.8751 -
1.2581 13400 7.9022 -
1.2675 13500 7.8806 -
1.2769 13600 7.9056 -
1.2863 13700 7.889 -
1.2957 13800 7.9155 -
1.3050 13900 7.9346 -
1.3144 14000 7.8537 -
1.3238 14100 7.8961 -
1.3332 14200 7.8977 -
1.3426 14300 7.887 -
1.3520 14400 7.8839 -
1.3614 14500 7.9331 -
1.3708 14600 7.8964 -
1.3802 14700 7.8773 -
1.3895 14800 7.8749 -
1.3989 14900 7.8824 -
1.4083 15000 7.8987 7.8832
1.4177 15100 7.8683 -
1.4271 15200 7.9177 -
1.4365 15300 7.8573 -
1.4459 15400 7.8797 -
1.4553 15500 7.8577 -
1.4647 15600 7.8827 -
1.4740 15700 7.8548 -
1.4834 15800 7.906 -
1.4928 15900 7.8808 -
1.5022 16000 7.8886 -
1.5116 16100 7.872 -
1.5210 16200 7.8746 -
1.5304 16300 7.8855 -
1.5398 16400 7.8549 -
1.5492 16500 7.8727 -
1.5585 16600 7.887 -
1.5679 16700 7.8534 -
1.5773 16800 7.888 -
1.5867 16900 7.8525 -
1.5961 17000 7.8818 -
1.6055 17100 7.9097 -
1.6149 17200 7.855 -
1.6243 17300 7.8925 -
1.6336 17400 7.8652 -
1.6430 17500 7.866 -
1.6524 17600 7.8411 -
1.6618 17700 7.8525 -
1.6712 17800 7.8651 -
1.6806 17900 7.8411 -
1.6900 18000 7.8622 -
1.6994 18100 7.8833 -
1.7088 18200 7.9135 -
1.7181 18300 7.8527 -
1.7275 18400 7.8451 -
1.7369 18500 7.8766 -
1.7463 18600 7.8375 -
1.7557 18700 7.8433 -
1.7651 18800 7.8321 -
1.7745 18900 7.8594 -
1.7839 19000 7.8398 -
1.7933 19100 7.8764 -
1.8026 19200 7.841 -
1.8120 19300 7.8515 -
1.8214 19400 7.8458 -
1.8308 19500 7.8409 -
1.8402 19600 7.8768 -
1.8496 19700 7.8533 -
1.8590 19800 7.8538 -
1.8684 19900 7.8547 -
1.8778 20000 7.8522 7.8474
1.8871 20100 7.87 -
1.8965 20200 7.8586 -
1.9059 20300 7.8529 -
1.9153 20400 7.8373 -
1.9247 20500 7.8239 -
1.9341 20600 7.8782 -
1.9435 20700 7.8533 -
1.9529 20800 7.8403 -
1.9623 20900 7.8904 -
1.9716 21000 7.8287 -
1.9810 21100 7.8844 -
1.9904 21200 7.8625 -
1.9998 21300 7.8568 -
2.0092 21400 7.841 -
2.0186 21500 7.8214 -
2.0280 21600 7.8255 -
2.0374 21700 7.8196 -
2.0468 21800 7.8441 -
2.0561 21900 7.8785 -
2.0655 22000 7.8331 -
2.0749 22100 7.8516 -
2.0843 22200 7.8164 -
2.0937 22300 7.8206 -
2.1031 22400 7.815 -
2.1125 22500 7.8048 -
2.1219 22600 7.8218 -
2.1313 22700 7.8371 -
2.1406 22800 7.7967 -
2.1500 22900 7.8182 -
2.1594 23000 7.8352 -
2.1688 23100 7.8565 -
2.1782 23200 7.8293 -
2.1876 23300 7.8216 -
2.1970 23400 7.8155 -
2.2064 23500 7.8269 -
2.2158 23600 7.8378 -
2.2251 23700 7.8056 -
2.2345 23800 7.827 -
2.2439 23900 7.8095 -
2.2533 24000 7.8292 -
2.2627 24100 7.8349 -
2.2721 24200 7.8391 -
2.2815 24300 7.8161 -
2.2909 24400 7.8053 -
2.3003 24500 7.8641 -
2.3096 24600 7.855 -
2.3190 24700 7.8286 -
2.3284 24800 7.8605 -
2.3378 24900 7.828 -
2.3472 25000 7.8274 7.8454
2.3566 25100 7.8104 -
2.3660 25200 7.873 -
2.3754 25300 7.7956 -
2.3848 25400 7.8135 -
2.3941 25500 7.8033 -
2.4035 25600 7.812 -
2.4129 25700 7.8285 -
2.4223 25800 7.8062 -
2.4317 25900 7.8178 -
2.4411 26000 7.8051 -
2.4505 26100 7.8255 -
2.4599 26200 7.8026 -
2.4693 26300 7.8627 -
2.4786 26400 7.8018 -
2.4880 26500 7.787 -
2.4974 26600 7.8374 -
2.5068 26700 7.8227 -
2.5162 26800 7.8076 -
2.5256 26900 7.7875 -
2.5350 27000 7.7908 -
2.5444 27100 7.8162 -
2.5538 27200 7.7919 -
2.5631 27300 7.8033 -
2.5725 27400 7.8147 -
2.5819 27500 7.8013 -
2.5913 27600 7.777 -
2.6007 27700 7.7982 -
2.6101 27800 7.8025 -
2.6195 27900 7.79 -
2.6289 28000 7.8124 -
2.6382 28100 7.7936 -
2.6476 28200 7.7793 -
2.6570 28300 7.8126 -
2.6664 28400 7.8149 -
2.6758 28500 7.7919 -
2.6852 28600 7.8127 -
2.6946 28700 7.8339 -
2.7040 28800 7.805 -
2.7134 28900 7.794 -
2.7227 29000 7.777 -
2.7321 29100 7.7888 -
2.7415 29200 7.8384 -
2.7509 29300 7.8175 -
2.7603 29400 7.8394 -
2.7697 29500 7.7813 -
2.7791 29600 7.8205 -
2.7885 29700 7.7982 -
2.7979 29800 7.7904 -
2.8072 29900 7.8107 -
2.8166 30000 7.8217 7.8158
2.8260 30100 7.7893 -
2.8354 30200 7.8139 -
2.8448 30300 7.8097 -
2.8542 30400 7.7966 -
2.8636 30500 7.7895 -
2.8730 30600 7.7914 -
2.8824 30700 7.8095 -
2.8917 30800 7.7943 -
2.9011 30900 7.8001 -
2.9105 31000 7.8299 -
2.9199 31100 7.7804 -
2.9293 31200 7.8015 -
2.9387 31300 7.8038 -
2.9481 31400 7.7731 -
2.9575 31500 7.7856 -
2.9669 31600 7.7935 -
2.9762 31700 7.7896 -
2.9856 31800 7.8216 -
2.9950 31900 7.7841 -
3.0044 32000 7.7569 -
3.0138 32100 7.7929 -
3.0232 32200 7.7738 -
3.0326 32300 7.7837 -
3.0420 32400 7.7777 -
3.0514 32500 7.7829 -
3.0607 32600 7.7585 -
3.0701 32700 7.7896 -
3.0795 32800 7.7873 -
3.0889 32900 7.7904 -
3.0983 33000 7.7808 -
3.1077 33100 7.7871 -
3.1171 33200 7.7835 -
3.1265 33300 7.7819 -
3.1359 33400 7.8037 -
3.1452 33500 7.7585 -
3.1546 33600 7.7928 -
3.1640 33700 7.7751 -
3.1734 33800 7.7829 -
3.1828 33900 7.7723 -
3.1922 34000 7.7999 -
3.2016 34100 7.757 -
3.2110 34200 7.7682 -
3.2204 34300 7.784 -
3.2297 34400 7.7962 -
3.2391 34500 7.7913 -
3.2485 34600 7.7768 -
3.2579 34700 7.7749 -
3.2673 34800 7.7724 -
3.2767 34900 7.7786 -
3.2861 35000 7.775 7.8301
3.2955 35100 7.7702 -
3.3049 35200 7.7689 -
3.3142 35300 7.7676 -
3.3236 35400 7.8029 -
3.3330 35500 7.7945 -
3.3424 35600 7.7765 -
3.3518 35700 7.7799 -
3.3612 35800 7.7701 -
3.3706 35900 7.7572 -
3.3800 36000 7.7656 -
3.3894 36100 7.8075 -
3.3987 36200 7.771 -
3.4081 36300 7.7757 -
3.4175 36400 7.7713 -
3.4269 36500 7.7885 -
3.4363 36600 7.7547 -
3.4457 36700 7.761 -
3.4551 36800 7.7797 -
3.4645 36900 7.7576 -
3.4739 37000 7.7578 -
3.4832 37100 7.736 -
3.4926 37200 7.7532 -
3.5020 37300 7.7747 -
3.5114 37400 7.7578 -
3.5208 37500 7.7632 -
3.5302 37600 7.7689 -
3.5396 37700 7.7796 -
3.5490 37800 7.7897 -
3.5584 37900 7.7824 -
3.5677 38000 7.7479 -
3.5771 38100 7.781 -
3.5865 38200 7.769 -
3.5959 38300 7.8087 -
3.6053 38400 7.7742 -
3.6147 38500 7.7974 -
3.6241 38600 7.7661 -
3.6335 38700 7.758 -
3.6429 38800 7.7659 -
3.6522 38900 7.753 -
3.6616 39000 7.819 -
3.6710 39100 7.766 -
3.6804 39200 7.7649 -
3.6898 39300 7.7684 -
3.6992 39400 7.7716 -
3.7086 39500 7.7781 -
3.7180 39600 7.788 -
3.7273 39700 7.7834 -
3.7367 39800 7.7566 -
3.7461 39900 7.7567 -
3.7555 40000 7.7804 7.8189
3.7649 40100 7.7559 -
3.7743 40200 7.7793 -
3.7837 40300 7.7749 -
3.7931 40400 7.7773 -
3.8025 40500 7.7836 -
3.8118 40600 7.7817 -
3.8212 40700 7.8036 -
3.8306 40800 7.7693 -
3.8400 40900 7.7895 -
3.8494 41000 7.789 -
3.8588 41100 7.7432 -
3.8682 41200 7.7777 -
3.8776 41300 7.7399 -
3.8870 41400 7.7629 -
3.8963 41500 7.7578 -
3.9057 41600 7.787 -
3.9151 41700 7.7984 -
3.9245 41800 7.7842 -
3.9339 41900 7.7992 -
3.9433 42000 7.7921 -
3.9527 42100 7.7986 -
3.9621 42200 7.7571 -
3.9715 42300 7.7783 -
3.9808 42400 7.7636 -
3.9902 42500 7.7633 -
3.9996 42600 7.7673 -

Framework Versions

  • Python: 3.8.10
  • Sentence Transformers: 3.1.1
  • Transformers: 4.45.2
  • PyTorch: 2.4.1+cu118
  • Accelerate: 1.0.1
  • Datasets: 3.0.1
  • Tokenizers: 0.20.3

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CoSENTLoss 

@online{kexuefm-8847,
    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
    author={Su Jianlin},
    year={2022},
    month={Jan},
    url={https://kexue.fm/archives/8847},
}