---
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:98
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
base_model: Snowflake/snowflake-arctic-embed-l
widget:
- source_sentence: What are some common attitudes and beliefs that can create personal
barriers to self-care for family caregivers?
sentences:
- 'Support for nutrition, breathing, and feeding
People with ALS may have trouble chewing and swallowing their food, and getting
the nutrients they need. Nutritionists and registered dieticians can help plan
small, nutritious meals throughout the day and identify foods to avoid. When the
person can no longer eat with help, a feeding tube can reduce the person’s risk
of choking and pneumonia.'
- "Amyotrophic Lateral Sclerosis (ALS) | National Institute of Neurological Disorders\
\ and Stroke\n\n\n\n\n\n\n\n\n Skip to main content\n \n\n\n\n\n\n\n\n\
\n\n\n\n\n\n\nAn official website of the United States government\n\n \
\ Here’s how you know\n\n\n\n\n\n\n\n\n\n\n\nOfficial websites use .gov \n\
\ A\n .gov\n website belongs to an\
\ official government organization in the United States.\n \n\n\n\
\n\n\n\n\n\nSecure .gov websites use HTTPS\n\n A lock\n \
\ (\n\n)\n or\n https://\n \
\ means you’ve safely connected to the .gov website. Share sensitive\
\ information only on official, secure websites.\n \n\n\n\n\n\n\
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nSearch\n\n\nMenu\n\n\n\n\n\n\n\n\n\nSearch NINDS\n\
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nSearch NINDS\n\n\n\n\n\n\n\n\n\n\n\
\n\n\n\nMain navigation"
- "Identifying Personal Barriers \nMany times, attitudes and beliefs form personal\
\ barriers that stand in the \nway of caring for yourself. Not taking care of\
\ yourself may be a lifelong \npattern, with taking care of others an easier option.\
\ However, as a family \ncaregiver you must ask yourself, \"What good will I\
\ be to the person I care \nfor if I become ill? If I die?\" Breaking old patterns\
\ and overcoming \nobstacles is not an easy proposition, but it can be done –\
\ regardless of \nyour age or situation. The first task in removing personal\
\ barriers to self-\ncare is to identify what is in your way. For example, \n\
• Do you feel you have to prove that you are worthy of the care recipient's \n\
affection? \n• Do you think you are being selfish if you put your needs first?\
\ \n• Is it frightening to think of your own needs? What is the fear about?"
- source_sentence: What role does the SOD1 gene play in the body?
sentences:
- "Migraine Trainer® Shareable Resources\n\n\n\nMind Your Risks®\n\n\nNINDS Brain\
\ Educational Resources\n\n\nStroke\n\n\n\n\n\n\nStroke Overview\n\n\nPrevention\n\
\n\nSigns and Symptoms\n\n\nAssess and Treat\n\n\n\n\n\n\nNIH Stroke Scale\n\n\
\n\nRecovery\n\n\nResearch\n\n\nOutreach\n\n\n\n\n\n\n\n\nDid you find the content\
\ you were looking for?\n\n\n\n\n\nYes, I did find the content I was looking for\n\
\n\n\n\nNo, I did not find the content I was looking for\n\n\n\n\n\n\n\nPlease\
\ rate how easy it was to navigate the NINDS website\n\n\n\n\n\nVery easy to navigate\n\
\n\n\n\nEasy to navigate\n\n\n\n\nNeutral\n\n\n\n\nDifficult to navigate\n\n\n\
\n\nVery difficult to navigate\n\n\n\n\n\n\nThank you for letting us know! Any\
\ other feedback?\n\n\n\n\nSubmit\n\n\n\n\n\nThis site is protected by reCAPTCHA\
\ and the Google Privacy Policyand Terms of Serviceapply.\n\n\n\n\n\n\n\n\n\n\n\
\n Last reviewed on July 19, 2024\n \n\n\n\n\n\n\n\
\n\n\n\n\nContact Us"
- 'Muscle twitches in the arm, leg, shoulder, or tongue
Muscle cramps
Tight and stiff muscles (spasticity)
Muscle weakness affecting an arm, a leg, or the neck
Slurred and nasal speech
Difficulty chewing or swallowing
As the disease progresses, muscle weakness and atrophy spread to other parts of
your body. People with ALS may develop problems with:
Chewing food and swallowing (dysphagia)
Drooling (sialorrhea)
Speaking or forming words (dysarthria)
Breathing (dyspnea)
Unintended crying, laughing, or other emotional displays (pseudobulbar symptoms)
Constipation
Maintaining weight and getting enough nutrients'
- "About 25-40% of all familial cases (and a small percentage of sporadic cases)\
\ are caused by a defect in the C9orf72 gene. C9orf72 makes a protein found in\
\ motor neurons and nerve cells in the brain. \nAnother 12-20% of familial cases\
\ result from mutations in the SOD1 gene. SOD1 is involved in production of the\
\ enzyme copper-zinc superoxide dismutase 1."
- source_sentence: What types of resources are available for caregivers of individuals
with ALS?
sentences:
- 'Eventually, people with ALS will not be able to stand or walk, get in or out
of bed on their own, use their hands and arms, or breathe on their own. Because
they usually remain able to reason, remember, and understand, they are aware of
their progressive loss of function. This can cause anxiety and depression in the
person with ALS and their loved ones. Although not as common, people with ALS
also may experience problems with language or decision-making. Some also develop
a form of dementia known as FTD-ALS.
Most people with ALS die from being unable to breathe on their own (known as respiratory
failure,) usually within three to five years from when the symptoms first appear.
However, about 10% survive for a decade or more.
Who is more likely to get amyotrophic lateral sclerosis (ALS)?'
- 'Motor Neuron Diseases
Order publications from the NINDS Catalog
The NINDS Publication Catalog offers printed materials on neurological disorders
for patients, health professionals, and the general public. All materials are
free of charge, and a downloadable PDF version is also available for most publications.
Order NINDS Publications
Health Information
Disorders
Glossary of Neurological Terms
Order Publications
Clinical Trials
Clinical Trials in the Spotlight
Find NINDS Clinical Trials
Patient & Caregiver Education
Brain Attack Coalition
Brain Donation
Public Education
Brain Basics
Know Your Brain
Understanding Sleep
Preventing Stroke
The Life and Death of a Neuron
Genes and the Brain
Migraine Trainer®
Migraine Trainer® Shareable Resources'
- "Caring for a person living with ALS\nAs the person with ALS progresses in their\
\ disease, they will need more and more help with daily activities. Being a caregiver\
\ for a person with ALS, while rewarding, can be challenging for the person’s\
\ loved ones and caregivers. It is important for caregivers take care of themselves\
\ and to seek support when needed. Free and paid resources are available to provide\
\ home health care services and support. Visit the organizations listed at the\
\ end of this article to find support in your area. \nWhat are the latest updates\
\ on amyotrophic lateral sclerosis (ALS)?"
- source_sentence: How can prospective donors participate in ALS research through
brain donation?
sentences:
- 'Doctors may use the following medications approved by the U.S. Food and Drug
Administration (FDA) to support a treatment plan for ALS:'
- NINDS also supports the NIH NeuroBioBank, a collaborative effort involving several
brain banks across the U.S. that supply investigators with tissue from people
with neurological and other disorders. Tissue from individuals with ALS is needed
to help advance critical research on the disease. A single donated brain can make
a huge impact on ALS research, potentially providing information for hundreds
of studies. The goal is to increase the availability of, and access to, high quality
specimens for research to understand the neurological basis of the disease. Prospective
donors can begin the enrollment process by visiting Learn How to Become a Brain
Donor.
- The National ALS Registry collects, manages, and analyzes de-identified data about
people with ALS in the United States. Developed by the Center for Disease Control
and Prevention's Agency for Toxic Substances and Disease Registry (ATSDR), this
registry establishes information about the number of ALS cases, collects demographic,
occupational, and environmental exposure data from people with ALS to learn about
potential risk factors for the disease, and notifies participants about research
opportunities. The Registry includes data from national databases as well as de-identified
information provided by individuals with ALS. All information is kept confidential.
People with ALS can add their information to the registry and sign up to receive
for more information.
- source_sentence: Does having a risk factor guarantee that a person will develop
a disorder?
sentences:
- 'Doctors may use the following medications approved by the U.S. Food and Drug
Administration (FDA) to support a treatment plan for ALS:'
- "possible decline in quality of life. \n \nBut despite these risks, family caregivers\
\ of any age are less likely than \nnon-caregivers to practice preventive healthcare\
\ and self-care behavior. \nRegardless of age, sex, and race and ethnicity, caregivers\
\ report problems \nattending to their own health and well-being while managing\
\ caregiving \nresponsibilities. They report: \n• sleep deprivation \n• poor\
\ eating habits \n• failure to exercise \n• failure to stay in bed when ill\
\ \n• postponement of or failure to make medical appointments ."
- 'A risk factor is a condition or behavior that occurs more frequently in those
who have a disease, or who are at greater risk of getting a disease, than in those
who don''t have the risk factor. Having a risk factor doesn''t mean a person will
develop a disorder, and not having a risk factor doesn''t mean you won’t. Risk
factors for ALS include:'
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
model-index:
- name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-l
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: Unknown
type: unknown
metrics:
- type: cosine_accuracy@1
value: 0.9166666666666666
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 1.0
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 1.0
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 1.0
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.9166666666666666
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.3333333333333333
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.20000000000000004
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.10000000000000002
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.9166666666666666
name: Cosine Recall@1
- type: cosine_recall@3
value: 1.0
name: Cosine Recall@3
- type: cosine_recall@5
value: 1.0
name: Cosine Recall@5
- type: cosine_recall@10
value: 1.0
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.9637887397321441
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.951388888888889
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.9513888888888888
name: Cosine Map@100
---
# SentenceTransformer based on Snowflake/snowflake-arctic-embed-l
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l). It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l)
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 1024 dimensions
- **Similarity Function:** Cosine Similarity
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("ernestobs7/caregiver-ft-v1")
# Run inference
sentences = [
'Does having a risk factor guarantee that a person will develop a disorder?',
"A risk factor is a condition or behavior that occurs more frequently in those who have a disease, or who are at greater risk of getting a disease, than in those who don't have the risk factor. Having a risk factor doesn't mean a person will develop a disorder, and not having a risk factor doesn't mean you won’t. Risk factors for ALS include:",
'possible decline in quality of life. \n \nBut despite these risks, family caregivers of any age are less likely than \nnon-caregivers to practice preventive healthcare and self-care behavior. \nRegardless of age, sex, and race and ethnicity, caregivers report problems \nattending to their own health and well-being while managing caregiving \nresponsibilities. They report: \n• sleep deprivation \n• poor eating habits \n• failure to exercise \n• failure to stay in bed when ill \n• postponement of or failure to make medical appointments .',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
## Evaluation
### Metrics
#### Information Retrieval
* Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.9167 |
| cosine_accuracy@3 | 1.0 |
| cosine_accuracy@5 | 1.0 |
| cosine_accuracy@10 | 1.0 |
| cosine_precision@1 | 0.9167 |
| cosine_precision@3 | 0.3333 |
| cosine_precision@5 | 0.2 |
| cosine_precision@10 | 0.1 |
| cosine_recall@1 | 0.9167 |
| cosine_recall@3 | 1.0 |
| cosine_recall@5 | 1.0 |
| cosine_recall@10 | 1.0 |
| **cosine_ndcg@10** | **0.9638** |
| cosine_mrr@10 | 0.9514 |
| cosine_map@100 | 0.9514 |
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 98 training samples
* Columns: sentence_0 and sentence_1
* Approximate statistics based on the first 98 samples:
| | sentence_0 | sentence_1 |
|:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
| type | string | string |
| details | - min: 12 tokens
- mean: 19.21 tokens
- max: 34 tokens
| - min: 30 tokens
- mean: 120.29 tokens
- max: 181 tokens
|
* Samples:
| sentence_0 | sentence_1 |
|:-----------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| What are some common symptoms experienced by individuals with ALS related to muscle function? | Muscle twitches in the arm, leg, shoulder, or tongue
Muscle cramps
Tight and stiff muscles (spasticity)
Muscle weakness affecting an arm, a leg, or the neck
Slurred and nasal speech
Difficulty chewing or swallowing
As the disease progresses, muscle weakness and atrophy spread to other parts of your body. People with ALS may develop problems with:
Chewing food and swallowing (dysphagia)
Drooling (sialorrhea)
Speaking or forming words (dysarthria)
Breathing (dyspnea)
Unintended crying, laughing, or other emotional displays (pseudobulbar symptoms)
Constipation
Maintaining weight and getting enough nutrients |
| How does ALS affect a person's ability to chew and swallow food? | Muscle twitches in the arm, leg, shoulder, or tongue
Muscle cramps
Tight and stiff muscles (spasticity)
Muscle weakness affecting an arm, a leg, or the neck
Slurred and nasal speech
Difficulty chewing or swallowing
As the disease progresses, muscle weakness and atrophy spread to other parts of your body. People with ALS may develop problems with:
Chewing food and swallowing (dysphagia)
Drooling (sialorrhea)
Speaking or forming words (dysarthria)
Breathing (dyspnea)
Unintended crying, laughing, or other emotional displays (pseudobulbar symptoms)
Constipation
Maintaining weight and getting enough nutrients |
| What percentage of ALS cases are classified as familial? | About 10% of all ALS cases are familial (also called inherited or genetic). Changes in more than a dozen genes have been found to cause familial ALS. |
* Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
```json
{
"loss": "MultipleNegativesRankingLoss",
"matryoshka_dims": [
768,
512,
256,
128,
64
],
"matryoshka_weights": [
1,
1,
1,
1,
1
],
"n_dims_per_step": -1
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: steps
- `per_device_train_batch_size`: 10
- `per_device_eval_batch_size`: 10
- `num_train_epochs`: 10
- `multi_dataset_batch_sampler`: round_robin
#### All Hyperparameters
Click to expand
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 10
- `per_device_eval_batch_size`: 10
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 10
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin