README.md

metadata

tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:98
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
base_model: Snowflake/snowflake-arctic-embed-l
widget:
  - source_sentence: >-
      What are some potential health effects that caregivers may experience as a
      result of their caregiving responsibilities?
    sentences:
      - |-
        Who We Are


        What We Do


        Job Opportunities


        75th Anniversary
























        Home


                  Health Information
                







        SHARE: 

















        Amyotrophic Lateral Sclerosis (ALS)











        On this page
      - >-
        The U.S. Food and Drug Administration has approved several drugs for ALS
        that may prolong survival, reduce the rate of decline, or help manage
        symptoms. However, there is currently no known treatment that stops or
        reverses the progression of ALS.

        Early symptoms include:
      - >-
        1 

        Fact Sheet:  Taking Care of YOU:  Self-Care for 

        Family Caregivers 
         
         
        First, Care for Yourself 

        On an airplane, an oxygen mask descends in front of you.  What do you 

        do? As we all know, the first rule is to put on your own oxygen mask
        before 

        you assist anyone else.  Only when we first help ourselves can we 

        effectively help others.  Caring for yourself is one of the most
        important – 

        and one of the most often forgotten – things you can do as a caregiver. 

        When your needs are taken care of, the person you care for will
        benefit, 

        too. 
         
        Effects of Caregiving on Health and Well Being 

        We hear this often: "My husband is the person with Alzheimer's, but now 

        I'm the one in the hospital!"  Such a situation is all too common. 

        Researchers know a lot about the effects of caregiving on health and
        well
  - source_sentence: What are clinical trials and what purpose do they serve in healthcare?
    sentences:
      - "NINDS\_also supports the\_NIH NeuroBioBank, a collaborative effort involving several brain banks across the U.S. that supply investigators with tissue from people with neurological and other disorders. Tissue from individuals\_with\_ALS\_is needed to help advance critical research on the disease. A single donated brain can make a huge impact on ALS research, potentially providing information for hundreds of studies. The goal is to increase the availability of, and access to, high quality specimens for research to understand the neurological basis of the disease. Prospective donors can begin the enrollment process by visiting\_Learn How to Become a Brain Donor."
      - >-
        being.  For example, if you are a caregiving spouse between the ages of
        66 

        and 96 and are experiencing mental or emotional strain, you have a risk
        of 

        dying that is 63 percent higher than that of people your age who are
        not 

        caregivers.1  The combination of loss, prolonged stress, the physical 

        demands of caregiving, and the biological vulnerabilities that come with
        age 

        place you at risk for significant health problems as well as an earlier
        death.  
         
        Older caregivers are not the only ones who put their health and well
        being 

        at risk.  If you are a baby boomer who has assumed a caregiver role for 

        your parents while simultaneously juggling work and raising adolescent 

        children, you face an increased risk for depression, chronic illness and
        a 

        possible decline in quality of life.
      - >-
        Learn About Clinical Trials

        Clinical trials are studies that allow us to learn more about disorders
        and improve care. They can help connect patients with new and upcoming
        treatment options.

        Search Clinical Trials
  - source_sentence: >-
      What are some common symptoms experienced by individuals with ALS related
      to muscle function?
    sentences:
      - >-
        • Do you have trouble asking for what you need? Do you feel inadequate 

        if you ask for help? Why?  
         
        Sometimes caregivers have misconceptions that increase their stress and 

        get in the way of good self-care.  Here are some of the most commonly 

        expressed: 

        • I am responsible for my parent's health.  

        • If I don't do it, no one will.  

        • If I do it right, I will get the love, attention, and respect I
        deserve.
      - >-
        possible decline in quality of life. 
         
        But despite these risks, family caregivers of any age are less likely
        than 

        non-caregivers to practice preventive healthcare and self-care
        behavior.  

        Regardless of age, sex, and race and ethnicity, caregivers report
        problems 

        attending to their own health and well-being while managing caregiving 

        responsibilities. They report: 

        • sleep deprivation  

        • poor eating habits  

        • failure to exercise  

        • failure to stay in bed when ill  

        • postponement of or failure to make medical appointments .
      - >-
        Muscle twitches in the arm, leg, shoulder, or tongue

        Muscle cramps

        Tight and stiff muscles (spasticity)

        Muscle weakness affecting an arm, a leg, or the neck

        Slurred and nasal speech

        Difficulty chewing or swallowing


        As the disease progresses, muscle weakness and atrophy spread to other
        parts of your body. People with ALS may develop problems with:


        Chewing food and swallowing (dysphagia)

        Drooling (sialorrhea)

        Speaking or forming words (dysarthria)

        Breathing (dyspnea)

        Unintended crying, laughing, or other emotional displays (pseudobulbar
        symptoms)

        Constipation

        Maintaining weight and getting enough nutrients
  - source_sentence: >-
      Why is it important for family caregivers to prioritize their own health
      and well-being?
    sentences:
      - "Cellular defects\nOngoing studies seek to understand the mechanisms that selectively trigger motor neurons to degenerate in\_ALS, which may lead to effective approaches to stop this process. Research using cellular culture systems and animal models suggests that motor neuron death is caused by a variety of cellular defects, including those involved in protein recycling and gene regulation, as well as structural impairments of motor neurons. Increasing evidence\_also suggests that glial support cells and inflammation cells of the nervous system may play an important role in\_ALS.\nStem cells"
      - >-
        Identifying Personal Barriers 

        Many times, attitudes and beliefs form personal barriers that stand in
        the 

        way of caring for yourself.  Not taking care of yourself may be a
        lifelong 

        pattern, with taking care of others an easier option.  However, as a
        family 

        caregiver you must ask yourself, "What good will I be to the person I
        care 

        for if I become ill? If I die?"  Breaking old patterns and overcoming 

        obstacles is not an easy proposition, but it can be done – regardless
        of 

        your age or situation.  The first task in removing personal barriers to
        self-

        care is to identify what is in your way.  For example, 

        • Do you feel you have to prove that you are worthy of the care
        recipient's 

        affection?  

        • Do you think you are being selfish if you put your needs first?  

        • Is it frightening to think of your own needs? What is the fear about?
      - >-
        Caring for a person living with ALS

        As the person with ALS progresses in their disease, they will need more
        and more help with daily activities. Being a caregiver for a person with
        ALS, while rewarding, can be challenging for the person’s loved ones and
        caregivers. It is important for caregivers take care of themselves and
        to seek support when needed. Free and paid resources are available to
        provide home health care services and support. Visit the organizations
        listed at the end of this article to find support in your area. 

        What are the latest updates on amyotrophic lateral sclerosis (ALS)?
  - source_sentence: >-
      What are some common health issues reported by family caregivers while
      managing their caregiving responsibilities?
    sentences:
      - >-
        Speech and communication support

        Speech therapists can help people with ALS learn strategies to speak
        louder and more clearly and help maintain the ability to communicate.
        Computer-based speech synthesizers use eye-tracking devices that allow a
        person to navigate the web and to type on custom screens to communicate.
        Voice banking is a process sometimes used by people with ALS to store
        their own voice for future use in computer-based speech synthesizers.

        A brain-computer interface (BCI) is a system that allows individuals to
        communicate or control equipment such as a wheelchair using only brain
        activity. Researchers are developing more efficient, mobile BCIs for
        people with severe paralysis and/or visual impairments.

        Support for nutrition, breathing, and feeding
      - "Consider participating in a clinical trial so clinicians and scientists can learn more about\_ALS.\_Clinical research uses human study participants to help researchers learn more about a disorder and perhaps find better ways to safely detect, treat, or prevent disease.\nAll types of study participants are needed—those who are healthy or may have an illness or disease—of all different ages, sexes, races, and ethnicities to ensure that study results apply to as many people as possible, and that treatments will be safe and effective for everyone who will use them.\nFor information about participating in clinical research visit\_NIH Clinical Research Trials\_and You. Learn about clinical trials\_currently looking for people with\_ALS\_at\_Clinicaltrial.gov."
      - >-
        possible decline in quality of life. 
         
        But despite these risks, family caregivers of any age are less likely
        than 

        non-caregivers to practice preventive healthcare and self-care
        behavior.  

        Regardless of age, sex, and race and ethnicity, caregivers report
        problems 

        attending to their own health and well-being while managing caregiving 

        responsibilities. They report: 

        • sleep deprivation  

        • poor eating habits  

        • failure to exercise  

        • failure to stay in bed when ill  

        • postponement of or failure to make medical appointments .
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
model-index:
  - name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-l
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: Unknown
          type: unknown
        metrics:
          - type: cosine_accuracy@1
            value: 0.9166666666666666
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 1
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 1
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 1
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.9166666666666666
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.3333333333333333
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.19999999999999998
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09999999999999999
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.9166666666666666
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 1
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 1
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 1
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.9692441461309548
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.9583333333333334
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.9583333333333334
            name: Cosine Map@100

SentenceTransformer based on Snowflake/snowflake-arctic-embed-l

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-l. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Snowflake/snowflake-arctic-embed-l
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ernestobs7/caregiver-ft-v0")
# Run inference
sentences = [
    'What are some common health issues reported by family caregivers while managing their caregiving responsibilities?',
    'possible decline in quality of life. \n \nBut despite these risks, family caregivers of any age are less likely than \nnon-caregivers to practice preventive healthcare and self-care behavior.  \nRegardless of age, sex, and race and ethnicity, caregivers report problems \nattending to their own health and well-being while managing caregiving \nresponsibilities. They report: \n• sleep deprivation  \n• poor eating habits  \n• failure to exercise  \n• failure to stay in bed when ill  \n• postponement of or failure to make medical appointments .',
    'Consider participating in a clinical trial so clinicians and scientists can learn more about\xa0ALS.\xa0Clinical research uses human study participants to help researchers learn more about a disorder and perhaps find better ways to safely detect, treat, or prevent disease.\nAll types of study participants are needed—those who are healthy or may have an illness or disease—of all different ages, sexes, races, and ethnicities to ensure that study results apply to as many people as possible, and that treatments will be safe and effective for everyone who will use them.\nFor information about participating in clinical research visit\xa0NIH Clinical Research Trials\xa0and You. Learn about clinical trials\xa0currently looking for people with\xa0ALS\xa0at\xa0Clinicaltrial.gov.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.9167
cosine_accuracy@3	1.0
cosine_accuracy@5	1.0
cosine_accuracy@10	1.0
cosine_precision@1	0.9167
cosine_precision@3	0.3333
cosine_precision@5	0.2
cosine_precision@10	0.1
cosine_recall@1	0.9167
cosine_recall@3	1.0
cosine_recall@5	1.0
cosine_recall@10	1.0
cosine_ndcg@10	0.9692
cosine_mrr@10	0.9583
cosine_map@100	0.9583

Training Details

Training Dataset

Unnamed Dataset

• Size: 98 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 98 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 13 tokens mean: 19.41 tokens max: 34 tokens	min: 30 tokens mean: 120.29 tokens max: 181 tokens

• Samples:

  sentence_0	sentence_1
  What types of examinations are performed by healthcare providers to assess for ALS?	There is no single test that can definitely diagnose ALS. A healthcare provider will conduct a physical exam and review the person’s full medical history. A neurologic examination will test reflexes, muscle strength, and other responses. These tests should be performed at regular intervals to assess whether symptoms are getting worse over time. A healthcare provider may conduct muscle and imaging tests to rule out other diseases. This can help support an ALS diagnosis. These tests include:
  Why are muscle and imaging tests conducted in the process of diagnosing ALS?	There is no single test that can definitely diagnose ALS. A healthcare provider will conduct a physical exam and review the person’s full medical history. A neurologic examination will test reflexes, muscle strength, and other responses. These tests should be performed at regular intervals to assess whether symptoms are getting worse over time. A healthcare provider may conduct muscle and imaging tests to rule out other diseases. This can help support an ALS diagnosis. These tests include:
  What are some factors that influence an individual's level of stress in a caregiving situation?	Moving Forward Once you've started to identify any personal barriers to good self -care, you can begin to change your behavior, moving forward one small step at a time. Following are some effective tools for self-care that can start you on your way. Tool #1: Reducing Personal Stress How we perceive and respond to an event is a significant factor in how we adjust and cope with it. The stress you feel is not only the result of your caregiving situation but also the result of your perception of it – whether you see the glass as half-full or half-empty. It is important to remember that you are not alone in your experiences. Your level of stress is influenced by many factors, including the following: • Whether your caregiving is voluntary. If you feel you had no choice in

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• num_train_epochs: 10
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 10
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	cosine_ndcg@10
1.0	10	0.9728
2.0	20	0.9728
3.0	30	0.9434
4.0	40	0.9462
5.0	50	0.9616
6.0	60	0.9616
7.0	70	0.9588
8.0	80	0.9616
9.0	90	0.9692
10.0	100	0.9692

Framework Versions

• Python: 3.11.4
• Sentence Transformers: 3.4.1
• Transformers: 4.49.0
• PyTorch: 2.6.0+cu124
• Accelerate: 1.4.0
• Datasets: 3.3.2
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}