metadata

base_model: microsoft/deberta-v3-base
datasets:
  - tals/vitaminc
  - allenai/scitail
  - allenai/sciq
  - allenai/qasc
  - sentence-transformers/msmarco-msmarco-distilbert-base-v3
  - sentence-transformers/natural-questions
  - sentence-transformers/trivia-qa
  - sentence-transformers/gooaq
  - google-research-datasets/paws
language:
  - en
library_name: sentence-transformers
metrics:
  - pearson_cosine
  - spearman_cosine
  - pearson_manhattan
  - spearman_manhattan
  - pearson_euclidean
  - spearman_euclidean
  - pearson_dot
  - spearman_dot
  - pearson_max
  - spearman_max
  - cosine_accuracy
  - cosine_accuracy_threshold
  - cosine_f1
  - cosine_f1_threshold
  - cosine_precision
  - cosine_recall
  - cosine_ap
  - dot_accuracy
  - dot_accuracy_threshold
  - dot_f1
  - dot_f1_threshold
  - dot_precision
  - dot_recall
  - dot_ap
  - manhattan_accuracy
  - manhattan_accuracy_threshold
  - manhattan_f1
  - manhattan_f1_threshold
  - manhattan_precision
  - manhattan_recall
  - manhattan_ap
  - euclidean_accuracy
  - euclidean_accuracy_threshold
  - euclidean_f1
  - euclidean_f1_threshold
  - euclidean_precision
  - euclidean_recall
  - euclidean_ap
  - max_accuracy
  - max_accuracy_threshold
  - max_f1
  - max_f1_threshold
  - max_precision
  - max_recall
  - max_ap
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:123245
  - loss:CachedGISTEmbedLoss
widget:
  - source_sentence: what type of inheritance does haemochromatosis
    sentences:
      - >-
        Nestled on the tranquil banks of the Pamlico River, Moss Landing is a
        vibrant new community of thoughtfully conceived, meticulously crafted
        single-family homes in Washington, North Carolina. Washington is
        renowned for its historic architecture and natural beauty.
      - >-
        1 Microwave on high for 8 to 10 minutes or until tender, turning the
        yams once. 2  To microwave sliced yams: Wash, peel, and cut off the
        woody portions and ends. 3  Cut yams into quarters. 4  Place the yams
        and 1/2 cup water in a microwave-safe casserole.ake the Yams. 1  Place
        half the yams in a 1-quart casserole. 2  Layer with half the brown sugar
        and half the margarine. 3  Repeat the layers. 4  Bake, uncovered, in a
        375 degree F oven for 30 to 35 minutes or until the yams are glazed,
        spooning the liquid over the yams once or twice during cooking.
      - >-
        Types 1, 2, and 3 hemochromatosis are inherited in an autosomal
        recessive pattern, which means both copies of the gene in each cell have
        mutations. Most often, the parents of an individual with an autosomal
        recessive condition each carry one copy of the mutated gene but do not
        show signs and symptoms of the condition.Type 4 hemochromatosis is
        distinguished by its autosomal dominant inheritance pattern.With this
        type of inheritance, one copy of the altered gene in each cell is
        sufficient to cause the disorder. In most cases, an affected person has
        one parent with the condition.ype 1, the most common form of the
        disorder, and type 4 (also called ferroportin disease) begin in
        adulthood. Men with type 1 or type 4 hemochromatosis typically develop
        symptoms between the ages of 40 and 60, and women usually develop
        symptoms after menopause. Type 2 hemochromatosis is a juvenile-onset
        disorder.
  - source_sentence: >-
      More than 273 people have died from the 2019-20 coronavirus outside
      mainland China .
    sentences:
      - >-
        More than 3,700 people have died : around 3,100 in mainland China and
        around 550 in all other countries combined .
      - >-
        More than 3,200 people have died : almost 3,000 in mainland China and
        around 275 in other countries .
      - more than 4,900 deaths have been attributed to COVID-19 .
  - source_sentence: >-
      The male reproductive system consists of structures that produce sperm and
      secrete testosterone.
    sentences:
      - What does the male reproductive system consist of?
      - What facilitates the diffusion of ions across a membrane?
      - >-
        Autoimmunity can develop with time, and its causes may be rooted in
        this?
  - source_sentence: Nitrogen gas comprises about three-fourths of earth's atmosphere.
    sentences:
      - What do all cells have in common?
      - What gas comprises about three-fourths of earth's atmosphere?
      - >-
        What do you call an animal in which the embryo, often termed a joey, is
        born immature and must complete its development outside the mother's
        body?
  - source_sentence: What device is used to regulate a person's heart rate?
    sentences:
      - >-
        Marie Antoinette and the French Revolution    .   Famous Faces    .  
        Mad Max: Maximilien Robespierre   |   PBS Extended Interviews >
        Resources > For Educators > Mad Max: Maximilien Robespierre Maximilien
        Robespierre was born May 6, 1758 in Arras, France. Educated at the Lycée
        Louis-le-Grand in Paris as a lawyer, Robespierre became a disciple of
        philosopher Jean-Jacques Rousseau and a passionate advocate for the
        poor. Called "the Incorruptible" because of his unwavering dedication to
        the Revolution, Robespierre joined the Jacobin Club and earned a loyal
        following. In contrast to the more republican Girondins and Marie
        Antoinette, Robespierre fiercely opposed declaring war on Austria,
        feeling it would distract from revolutionary progress in France.
        Robespierre's exemplary oratory skills influenced the National
        Convention in 1792 to avoid seeking public opinion about the
        Convention’s decision to execute King Louis XVI.  In 1793, the
        Convention elected Robespierre to the Committee of Public Defense. He
        was a highly controversial member, developing radical policies, warning
        of conspiracies, and suggesting restructuring the Convention. This
        behavior eventually led to his downfall, and he was guillotined without
        trial on 10th Thermidor An II (July 28, 1794), marking the end of the
        Reign of Terror. Famous Faces
      - >-
        Devices for Arrhythmia Devices for Arrhythmia Updated:Dec 21,2016 In a
        medical emergency, life-threatening arrhythmias may be stopped by giving
        the heart an electric shock (as with a defibrillator ). For people with
        recurrent arrhythmias, medical devices such as a pacemaker and
        implantable cardioverter defibrillator (ICD) can help by continuously
        monitoring the heart's electrical system and providing automatic
        correction when an arrhythmia starts to occur. This section covers
        everything you need to know about these devices. Implantable
        Cardioverter Defibrillator (ICD)
      - >-
        vintage cleats | eBay vintage cleats: 1 2 3 4 5 eBay determines this
        price through a machine learned model of the product's sale prices
        within the last 90 days. eBay determines trending price through a
        machine learned model of the product’s sale prices within the last 90
        days. "New" refers to a brand-new, unused, unopened, undamaged item, and
        "Used" refers to an item that has been used previously. Top Rated Plus
        Sellers with highest buyer ratings Returns, money back Sellers with
        highest buyer ratings Returns, money back
model-index:
  - name: SentenceTransformer based on microsoft/deberta-v3-base
    results:
      - task:
          type: semantic-similarity
          name: Semantic Similarity
        dataset:
          name: sts test
          type: sts-test
        metrics:
          - type: pearson_cosine
            value: 0.8253431554642914
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.870857890879963
            name: Spearman Cosine
          - type: pearson_manhattan
            value: 0.8653068915625914
            name: Pearson Manhattan
          - type: spearman_manhattan
            value: 0.8667110599943904
            name: Spearman Manhattan
          - type: pearson_euclidean
            value: 0.8671346646296434
            name: Pearson Euclidean
          - type: spearman_euclidean
            value: 0.8681442638917114
            name: Spearman Euclidean
          - type: pearson_dot
            value: 0.7826717704847901
            name: Pearson Dot
          - type: spearman_dot
            value: 0.7685403521338614
            name: Spearman Dot
          - type: pearson_max
            value: 0.8671346646296434
            name: Pearson Max
          - type: spearman_max
            value: 0.870857890879963
            name: Spearman Max
      - task:
          type: binary-classification
          name: Binary Classification
        dataset:
          name: allNLI dev
          type: allNLI-dev
        metrics:
          - type: cosine_accuracy
            value: 0.71875
            name: Cosine Accuracy
          - type: cosine_accuracy_threshold
            value: 0.8745474815368652
            name: Cosine Accuracy Threshold
          - type: cosine_f1
            value: 0.617169373549884
            name: Cosine F1
          - type: cosine_f1_threshold
            value: 0.7519949674606323
            name: Cosine F1 Threshold
          - type: cosine_precision
            value: 0.5155038759689923
            name: Cosine Precision
          - type: cosine_recall
            value: 0.7687861271676301
            name: Cosine Recall
          - type: cosine_ap
            value: 0.6116004689391709
            name: Cosine Ap
          - type: dot_accuracy
            value: 0.693359375
            name: Dot Accuracy
          - type: dot_accuracy_threshold
            value: 401.3755187988281
            name: Dot Accuracy Threshold
          - type: dot_f1
            value: 0.566735112936345
            name: Dot F1
          - type: dot_f1_threshold
            value: 295.2575988769531
            name: Dot F1 Threshold
          - type: dot_precision
            value: 0.4394904458598726
            name: Dot Precision
          - type: dot_recall
            value: 0.7976878612716763
            name: Dot Recall
          - type: dot_ap
            value: 0.5243551756921989
            name: Dot Ap
          - type: manhattan_accuracy
            value: 0.724609375
            name: Manhattan Accuracy
          - type: manhattan_accuracy_threshold
            value: 228.3092498779297
            name: Manhattan Accuracy Threshold
          - type: manhattan_f1
            value: 0.6267281105990783
            name: Manhattan F1
          - type: manhattan_f1_threshold
            value: 266.0207824707031
            name: Manhattan F1 Threshold
          - type: manhattan_precision
            value: 0.5210727969348659
            name: Manhattan Precision
          - type: manhattan_recall
            value: 0.7861271676300579
            name: Manhattan Recall
          - type: manhattan_ap
            value: 0.6101425904568746
            name: Manhattan Ap
          - type: euclidean_accuracy
            value: 0.720703125
            name: Euclidean Accuracy
          - type: euclidean_accuracy_threshold
            value: 9.726119041442871
            name: Euclidean Accuracy Threshold
          - type: euclidean_f1
            value: 0.6303854875283447
            name: Euclidean F1
          - type: euclidean_f1_threshold
            value: 14.837699890136719
            name: Euclidean F1 Threshold
          - type: euclidean_precision
            value: 0.5186567164179104
            name: Euclidean Precision
          - type: euclidean_recall
            value: 0.8034682080924855
            name: Euclidean Recall
          - type: euclidean_ap
            value: 0.6172110045723997
            name: Euclidean Ap
          - type: max_accuracy
            value: 0.724609375
            name: Max Accuracy
          - type: max_accuracy_threshold
            value: 401.3755187988281
            name: Max Accuracy Threshold
          - type: max_f1
            value: 0.6303854875283447
            name: Max F1
          - type: max_f1_threshold
            value: 295.2575988769531
            name: Max F1 Threshold
          - type: max_precision
            value: 0.5210727969348659
            name: Max Precision
          - type: max_recall
            value: 0.8034682080924855
            name: Max Recall
          - type: max_ap
            value: 0.6172110045723997
            name: Max Ap
      - task:
          type: binary-classification
          name: Binary Classification
        dataset:
          name: Qnli dev
          type: Qnli-dev
        metrics:
          - type: cosine_accuracy
            value: 0.673828125
            name: Cosine Accuracy
          - type: cosine_accuracy_threshold
            value: 0.7472400069236755
            name: Cosine Accuracy Threshold
          - type: cosine_f1
            value: 0.6863468634686347
            name: Cosine F1
          - type: cosine_f1_threshold
            value: 0.7334084510803223
            name: Cosine F1 Threshold
          - type: cosine_precision
            value: 0.6078431372549019
            name: Cosine Precision
          - type: cosine_recall
            value: 0.788135593220339
            name: Cosine Recall
          - type: cosine_ap
            value: 0.7293502303398447
            name: Cosine Ap
          - type: dot_accuracy
            value: 0.6484375
            name: Dot Accuracy
          - type: dot_accuracy_threshold
            value: 392.88726806640625
            name: Dot Accuracy Threshold
          - type: dot_f1
            value: 0.6634920634920635
            name: Dot F1
          - type: dot_f1_threshold
            value: 310.97833251953125
            name: Dot F1 Threshold
          - type: dot_precision
            value: 0.5304568527918782
            name: Dot Precision
          - type: dot_recall
            value: 0.885593220338983
            name: Dot Recall
          - type: dot_ap
            value: 0.6331200610041253
            name: Dot Ap
          - type: manhattan_accuracy
            value: 0.671875
            name: Manhattan Accuracy
          - type: manhattan_accuracy_threshold
            value: 277.69342041015625
            name: Manhattan Accuracy Threshold
          - type: manhattan_f1
            value: 0.6830122591943958
            name: Manhattan F1
          - type: manhattan_f1_threshold
            value: 301.36639404296875
            name: Manhattan F1 Threshold
          - type: manhattan_precision
            value: 0.582089552238806
            name: Manhattan Precision
          - type: manhattan_recall
            value: 0.826271186440678
            name: Manhattan Recall
          - type: manhattan_ap
            value: 0.7276384343706648
            name: Manhattan Ap
          - type: euclidean_accuracy
            value: 0.68359375
            name: Euclidean Accuracy
          - type: euclidean_accuracy_threshold
            value: 15.343950271606445
            name: Euclidean Accuracy Threshold
          - type: euclidean_f1
            value: 0.6895238095238095
            name: Euclidean F1
          - type: euclidean_f1_threshold
            value: 15.738676071166992
            name: Euclidean F1 Threshold
          - type: euclidean_precision
            value: 0.6262975778546713
            name: Euclidean Precision
          - type: euclidean_recall
            value: 0.7669491525423728
            name: Euclidean Recall
          - type: euclidean_ap
            value: 0.7307379367367225
            name: Euclidean Ap
          - type: max_accuracy
            value: 0.68359375
            name: Max Accuracy
          - type: max_accuracy_threshold
            value: 392.88726806640625
            name: Max Accuracy Threshold
          - type: max_f1
            value: 0.6895238095238095
            name: Max F1
          - type: max_f1_threshold
            value: 310.97833251953125
            name: Max F1 Threshold
          - type: max_precision
            value: 0.6262975778546713
            name: Max Precision
          - type: max_recall
            value: 0.885593220338983
            name: Max Recall
          - type: max_ap
            value: 0.7307379367367225
            name: Max Ap

SentenceTransformer based on microsoft/deberta-v3-base

This is a sentence-transformers model finetuned from microsoft/deberta-v3-base on the negation-triplets, vitaminc-pairs, scitail-pairs-qa, scitail-pairs-pos, xsum-pairs, sciq_pairs, qasc_pairs, openbookqa_pairs, msmarco_pairs, nq_pairs, trivia_pairs, gooaq_pairs, paws-pos and global_dataset datasets. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: microsoft/deberta-v3-base
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 tokens
Similarity Function: Cosine Similarity
Training Datasets:
- negation-triplets
- vitaminc-pairs
- scitail-pairs-qa
- scitail-pairs-pos
- xsum-pairs
- sciq_pairs
- qasc_pairs
- openbookqa_pairs
- msmarco_pairs
- nq_pairs
- trivia_pairs
- gooaq_pairs
- paws-pos
- global_dataset
Language: en

Evaluation

Metrics

Semantic Similarity

Dataset: sts-test
Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.8253
spearman_cosine	0.8709
pearson_manhattan	0.8653
spearman_manhattan	0.8667
pearson_euclidean	0.8671
spearman_euclidean	0.8681
pearson_dot	0.7827
spearman_dot	0.7685
pearson_max	0.8671
spearman_max	0.8709

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 96
per_device_eval_batch_size: 68
learning_rate: 3.5e-05
weight_decay: 0.0005
num_train_epochs: 2
lr_scheduler_type: cosine_with_min_lr
lr_scheduler_kwargs: {'num_cycles': 3.5, 'min_lr': 1.5e-05}
warmup_ratio: 0.33
save_safetensors: False
fp16: True
push_to_hub: True
hub_model_id: bobox/DeBERTa3-base-STr-CosineWaves-checkpoints-tmp
hub_strategy: all_checkpoints
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 96
per_device_eval_batch_size: 68
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 3.5e-05
weight_decay: 0.0005
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 2
max_steps: -1
lr_scheduler_type: cosine_with_min_lr
lr_scheduler_kwargs: {'num_cycles': 3.5, 'min_lr': 1.5e-05}
warmup_ratio: 0.33
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: False
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: True
resume_from_checkpoint: None
hub_model_id: bobox/DeBERTa3-base-STr-CosineWaves-checkpoints-tmp
hub_strategy: all_checkpoints
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
eval_use_gather_object: False
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Framework Versions

Python: 3.10.14
Sentence Transformers: 3.0.1
Transformers: 4.44.0
PyTorch: 2.4.0
Accelerate: 0.33.0
Datasets: 2.21.0
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}