README.md

metadata

tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:98
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
base_model: Snowflake/snowflake-arctic-embed-l
widget:
  - source_sentence: >-
      What are some common attitudes and beliefs that can create personal
      barriers to self-care for family caregivers?
    sentences:
      - "Support for nutrition, breathing, and feeding\nPeople with ALS may have trouble chewing and swallowing their food, and getting the nutrients they need. Nutritionists and registered dieticians can help plan small, nutritious meals\_throughout the day and identify foods to avoid. When the person can no longer eat with help, a feeding tube can reduce the person’s risk of choking and pneumonia."
      - >-
        Amyotrophic Lateral Sclerosis (ALS) | National Institute of Neurological
        Disorders and Stroke








              Skip to main content
            














        An official website of the United States government

                    Here’s how you know











        Official websites use .gov 
                        A
                        .gov
                        website belongs to an official government organization in the United States.
                      








        Secure .gov websites use HTTPS

                          A lock
                          (

        )
                           or
                          https://
                           means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
                        




















        Search



        Menu










        Search NINDS























        Search NINDS















        Main navigation
      - >-
        Identifying Personal Barriers 

        Many times, attitudes and beliefs form personal barriers that stand in
        the 

        way of caring for yourself.  Not taking care of yourself may be a
        lifelong 

        pattern, with taking care of others an easier option.  However, as a
        family 

        caregiver you must ask yourself, "What good will I be to the person I
        care 

        for if I become ill? If I die?"  Breaking old patterns and overcoming 

        obstacles is not an easy proposition, but it can be done – regardless
        of 

        your age or situation.  The first task in removing personal barriers to
        self-

        care is to identify what is in your way.  For example, 

        • Do you feel you have to prove that you are worthy of the care
        recipient's 

        affection?  

        • Do you think you are being selfish if you put your needs first?  

        • Is it frightening to think of your own needs? What is the fear about?
  - source_sentence: What role does the SOD1 gene play in the body?
    sentences:
      - "Migraine Trainer® Shareable Resources\n\n\n\nMind Your Risks®\n\n\nNINDS Brain Educational Resources\n\n\nStroke\n\n\n\n\n\n\nStroke Overview\n\n\nPrevention\n\n\nSigns and Symptoms\n\n\nAssess and Treat\n\n\n\n\n\n\nNIH Stroke Scale\n\n\n\nRecovery\n\n\nResearch\n\n\nOutreach\n\n\n\n\n\n\n\n\nDid you find the content you were looking for?\n\n\n\n\n\nYes, I did find the content I was looking for\n\n\n\n\nNo, I did not find the content I was looking for\n\n\n\n\n\n\n\nPlease rate how easy it was to navigate the NINDS website\n\n\n\n\n\nVery easy to navigate\n\n\n\n\nEasy to navigate\n\n\n\n\nNeutral\n\n\n\n\nDifficult to navigate\n\n\n\n\nVery difficult to navigate\n\n\n\n\n\n\nThank you for letting us know! Any other feedback?\n\n\n\n\nSubmit\n\n\n\n\n\nThis site is protected by reCAPTCHA and the Google\_Privacy Policyand Terms of Serviceapply.\n\n\n\n\n\n\n\n\n\n\n\n                          Last reviewed on July 19, 2024\n    \n\n\n\n\n\n\n\n\n\n\n\nContact Us"
      - >-
        Muscle twitches in the arm, leg, shoulder, or tongue

        Muscle cramps

        Tight and stiff muscles (spasticity)

        Muscle weakness affecting an arm, a leg, or the neck

        Slurred and nasal speech

        Difficulty chewing or swallowing


        As the disease progresses, muscle weakness and atrophy spread to other
        parts of your body. People with ALS may develop problems with:


        Chewing food and swallowing (dysphagia)

        Drooling (sialorrhea)

        Speaking or forming words (dysarthria)

        Breathing (dyspnea)

        Unintended crying, laughing, or other emotional displays (pseudobulbar
        symptoms)

        Constipation

        Maintaining weight and getting enough nutrients
      - >-
        About 25-40% of all familial cases (and a small percentage of sporadic
        cases) are caused by a defect in the C9orf72 gene. C9orf72 makes a
        protein found in motor neurons and nerve cells in the brain. 

        Another 12-20% of familial cases result from mutations in the SOD1 gene.
        SOD1 is involved in production of the enzyme copper-zinc superoxide
        dismutase 1.
  - source_sentence: >-
      What types of resources are available for caregivers of individuals with
      ALS?
    sentences:
      - >-
        Eventually, people with ALS will not be able to stand or walk, get in or
        out of bed on their own, use their hands and arms, or breathe on their
        own. Because they usually remain able to reason, remember, and
        understand, they are aware of their progressive loss of function. This
        can cause anxiety and depression in the person with ALS and their loved
        ones. Although not as common, people with ALS also may experience
        problems with language or decision-making. Some also develop a form of
        dementia known as FTD-ALS.

        Most people with ALS die from being unable to breathe on their own
        (known as respiratory failure,) usually within three to five years from
        when the symptoms first appear. However, about 10% survive for a decade
        or more.

        Who is more likely to get amyotrophic lateral sclerosis (ALS)?
      - "Motor Neuron Diseases\_\n\n\n\n\n\n\n\n\n\n\n\n\nOrder publications from the NINDS Catalog\nThe NINDS Publication Catalog offers printed materials on neurological disorders for patients, health professionals, and the general public. All materials are free of charge, and a downloadable PDF version is also available for most publications.\nOrder NINDS Publications\n\n\n\_\n\n\n\n\n\n\n\nHealth Information\n\n\n\n\n\n\nDisorders\n\n\n\n\n\n\nGlossary of Neurological Terms\n\n\nOrder Publications\n\n\n\nClinical Trials\n\n\n\n\n\n\nClinical Trials in the Spotlight\n\n\nFind NINDS Clinical Trials\n\n\n\nPatient & Caregiver Education\n\n\n\n\n\n\nBrain Attack Coalition\n\n\nBrain Donation\n\n\n\nPublic Education\n\n\n\n\n\n\nBrain Basics\n\n\n\n\n\n\nKnow Your Brain\n\n\nUnderstanding Sleep\n\n\nPreventing Stroke\n\n\nThe Life and Death of a Neuron\n\n\nGenes and the Brain\n\n\n\nMigraine Trainer®\n\n\n\n\n\n\nMigraine Trainer® Shareable Resources"
      - >-
        Caring for a person living with ALS

        As the person with ALS progresses in their disease, they will need more
        and more help with daily activities. Being a caregiver for a person with
        ALS, while rewarding, can be challenging for the person’s loved ones and
        caregivers. It is important for caregivers take care of themselves and
        to seek support when needed. Free and paid resources are available to
        provide home health care services and support. Visit the organizations
        listed at the end of this article to find support in your area. 

        What are the latest updates on amyotrophic lateral sclerosis (ALS)?
  - source_sentence: >-
      How can prospective donors participate in ALS research through brain
      donation?
    sentences:
      - "Doctors may use the following medications approved by the U.S. Food and Drug Administration (FDA) to support a treatment plan for\_ALS:"
      - "NINDS\_also supports the\_NIH NeuroBioBank, a collaborative effort involving several brain banks across the U.S. that supply investigators with tissue from people with neurological and other disorders. Tissue from individuals\_with\_ALS\_is needed to help advance critical research on the disease. A single donated brain can make a huge impact on ALS research, potentially providing information for hundreds of studies. The goal is to increase the availability of, and access to, high quality specimens for research to understand the neurological basis of the disease. Prospective donors can begin the enrollment process by visiting\_Learn How to Become a Brain Donor."
      - "The\_National\_ALS\_Registry\_collects, manages, and analyzes de-identified data about people with\_ALS\_in the United States. Developed by the Center for Disease Control and Prevention's Agency for Toxic Substances and Disease Registry (ATSDR), this registry establishes information about the number of\_ALS\_cases, collects demographic, occupational, and environmental exposure data from people with\_ALS\_to learn about potential risk factors for the disease, and notifies participants about research opportunities. The Registry includes data from national databases as well as de-identified information provided by individuals\_with\_ALS. All information is kept confidential. People with\_ALS\_can add their information to the registry and sign up to receive for more information."
  - source_sentence: Does having a risk factor guarantee that a person will develop a disorder?
    sentences:
      - "Doctors may use the following medications approved by the U.S. Food and Drug Administration (FDA) to support a treatment plan for\_ALS:"
      - >-
        possible decline in quality of life. 
         
        But despite these risks, family caregivers of any age are less likely
        than 

        non-caregivers to practice preventive healthcare and self-care
        behavior.  

        Regardless of age, sex, and race and ethnicity, caregivers report
        problems 

        attending to their own health and well-being while managing caregiving 

        responsibilities. They report: 

        • sleep deprivation  

        • poor eating habits  

        • failure to exercise  

        • failure to stay in bed when ill  

        • postponement of or failure to make medical appointments .
      - >-
        A risk factor is a condition or behavior that occurs more frequently in
        those who have a disease, or who are at greater risk of getting a
        disease, than in those who don't have the risk factor. Having a risk
        factor doesn't mean a person will develop a disorder, and not having a
        risk factor doesn't mean you won’t. Risk factors for ALS include:
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
model-index:
  - name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-l
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: Unknown
          type: unknown
        metrics:
          - type: cosine_accuracy@1
            value: 0.9166666666666666
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 1
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 1
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 1
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.9166666666666666
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.3333333333333333
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.20000000000000004
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.10000000000000002
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.9166666666666666
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 1
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 1
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 1
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.9637887397321441
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.951388888888889
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.9513888888888888
            name: Cosine Map@100

SentenceTransformer based on Snowflake/snowflake-arctic-embed-l

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-l. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Snowflake/snowflake-arctic-embed-l
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ernestobs7/caregiver-ft-v1")
# Run inference
sentences = [
    'Does having a risk factor guarantee that a person will develop a disorder?',
    "A risk factor is a condition or behavior that occurs more frequently in those who have a disease, or who are at greater risk of getting a disease, than in those who don't have the risk factor. Having a risk factor doesn't mean a person will develop a disorder, and not having a risk factor doesn't mean you won’t. Risk factors for ALS include:",
    'possible decline in quality of life. \n \nBut despite these risks, family caregivers of any age are less likely than \nnon-caregivers to practice preventive healthcare and self-care behavior.  \nRegardless of age, sex, and race and ethnicity, caregivers report problems \nattending to their own health and well-being while managing caregiving \nresponsibilities. They report: \n• sleep deprivation  \n• poor eating habits  \n• failure to exercise  \n• failure to stay in bed when ill  \n• postponement of or failure to make medical appointments .',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.9167
cosine_accuracy@3	1.0
cosine_accuracy@5	1.0
cosine_accuracy@10	1.0
cosine_precision@1	0.9167
cosine_precision@3	0.3333
cosine_precision@5	0.2
cosine_precision@10	0.1
cosine_recall@1	0.9167
cosine_recall@3	1.0
cosine_recall@5	1.0
cosine_recall@10	1.0
cosine_ndcg@10	0.9638
cosine_mrr@10	0.9514
cosine_map@100	0.9514

Training Details

Training Dataset

Unnamed Dataset

• Size: 98 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 98 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 12 tokens mean: 19.21 tokens max: 34 tokens	min: 30 tokens mean: 120.29 tokens max: 181 tokens

• Samples:

  sentence_0	sentence_1
  What are some common symptoms experienced by individuals with ALS related to muscle function?	Muscle twitches in the arm, leg, shoulder, or tongue Muscle cramps Tight and stiff muscles (spasticity) Muscle weakness affecting an arm, a leg, or the neck Slurred and nasal speech Difficulty chewing or swallowing As the disease progresses, muscle weakness and atrophy spread to other parts of your body. People with ALS may develop problems with: Chewing food and swallowing (dysphagia) Drooling (sialorrhea) Speaking or forming words (dysarthria) Breathing (dyspnea) Unintended crying, laughing, or other emotional displays (pseudobulbar symptoms) Constipation Maintaining weight and getting enough nutrients
  How does ALS affect a person's ability to chew and swallow food?	Muscle twitches in the arm, leg, shoulder, or tongue Muscle cramps Tight and stiff muscles (spasticity) Muscle weakness affecting an arm, a leg, or the neck Slurred and nasal speech Difficulty chewing or swallowing As the disease progresses, muscle weakness and atrophy spread to other parts of your body. People with ALS may develop problems with: Chewing food and swallowing (dysphagia) Drooling (sialorrhea) Speaking or forming words (dysarthria) Breathing (dyspnea) Unintended crying, laughing, or other emotional displays (pseudobulbar symptoms) Constipation Maintaining weight and getting enough nutrients
  What percentage of ALS cases are classified as familial?	About 10% of all ALS cases are familial (also called inherited or genetic). Changes in more than a dozen genes have been found to cause familial ALS.

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• num_train_epochs: 10
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 10
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	cosine_ndcg@10
1.0	10	0.9382
2.0	20	0.9539
3.0	30	0.9484
4.0	40	0.9484
5.0	50	0.9638
6.0	60	0.9638
7.0	70	0.9638
8.0	80	0.9638
9.0	90	0.9638
10.0	100	0.9638

Framework Versions

• Python: 3.11.4
• Sentence Transformers: 3.4.1
• Transformers: 4.49.0
• PyTorch: 2.6.0+cu124
• Accelerate: 1.4.0
• Datasets: 3.3.2
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}