|
--- |
|
tags: |
|
- sentence-transformers |
|
- sentence-similarity |
|
- feature-extraction |
|
- generated_from_trainer |
|
- dataset_size:98 |
|
- loss:MatryoshkaLoss |
|
- loss:MultipleNegativesRankingLoss |
|
base_model: Snowflake/snowflake-arctic-embed-l |
|
widget: |
|
- source_sentence: What are some potential health effects that caregivers may experience |
|
as a result of their caregiving responsibilities? |
|
sentences: |
|
- "Who We Are\n\n\nWhat We Do\n\n\nJob Opportunities\n\n\n75th Anniversary\n\n\n\ |
|
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nHome\n\n\n Health Information\n\ |
|
\ \n\n\n\n\n\n\n\nSHARE: \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nAmyotrophic\ |
|
\ Lateral Sclerosis (ALS)\n\n\n\n\n\n\n\n\n\n\n\nOn this page" |
|
- 'The U.S. Food and Drug Administration has approved several drugs for ALS that |
|
may prolong survival, reduce the rate of decline, or help manage symptoms. However, |
|
there is currently no known treatment that stops or reverses the progression of |
|
ALS. |
|
|
|
Early symptoms include:' |
|
- "1 \nFact Sheet: Taking Care of YOU: Self-Care for \nFamily Caregivers \n \n\ |
|
\ \nFirst, Care for Yourself \nOn an airplane, an oxygen mask descends in front\ |
|
\ of you. What do you \ndo? As we all know, the first rule is to put on your\ |
|
\ own oxygen mask before \nyou assist anyone else. Only when we first help ourselves\ |
|
\ can we \neffectively help others. Caring for yourself is one of the most important\ |
|
\ – \nand one of the most often forgotten – things you can do as a caregiver.\ |
|
\ \nWhen your needs are taken care of, the person you care for will benefit, \n\ |
|
too. \n \nEffects of Caregiving on Health and Well Being \nWe hear this often:\ |
|
\ \"My husband is the person with Alzheimer's, but now \nI'm the one in the hospital!\"\ |
|
\ Such a situation is all too common. \nResearchers know a lot about the effects\ |
|
\ of caregiving on health and well" |
|
- source_sentence: What are clinical trials and what purpose do they serve in healthcare? |
|
sentences: |
|
- NINDS also supports the NIH NeuroBioBank, a collaborative effort involving several |
|
brain banks across the U.S. that supply investigators with tissue from people |
|
with neurological and other disorders. Tissue from individuals with ALS is needed |
|
to help advance critical research on the disease. A single donated brain can make |
|
a huge impact on ALS research, potentially providing information for hundreds |
|
of studies. The goal is to increase the availability of, and access to, high quality |
|
specimens for research to understand the neurological basis of the disease. Prospective |
|
donors can begin the enrollment process by visiting Learn How to Become a Brain |
|
Donor. |
|
- "being. For example, if you are a caregiving spouse between the ages of 66 \n\ |
|
and 96 and are experiencing mental or emotional strain, you have a risk of \n\ |
|
dying that is 63 percent higher than that of people your age who are not \ncaregivers.1\ |
|
\ The combination of loss, prolonged stress, the physical \ndemands of caregiving,\ |
|
\ and the biological vulnerabilities that come with age \nplace you at risk for\ |
|
\ significant health problems as well as an earlier death. \n \nOlder caregivers\ |
|
\ are not the only ones who put their health and well being \nat risk. If you\ |
|
\ are a baby boomer who has assumed a caregiver role for \nyour parents while\ |
|
\ simultaneously juggling work and raising adolescent \nchildren, you face an\ |
|
\ increased risk for depression, chronic illness and a \npossible decline in quality\ |
|
\ of life." |
|
- 'Learn About Clinical Trials |
|
|
|
Clinical trials are studies that allow us to learn more about disorders and improve |
|
care. They can help connect patients with new and upcoming treatment options. |
|
|
|
Search Clinical Trials' |
|
- source_sentence: What are some common symptoms experienced by individuals with ALS |
|
related to muscle function? |
|
sentences: |
|
- "• Do you have trouble asking for what you need? Do you feel inadequate \nif you\ |
|
\ ask for help? Why? \n \nSometimes caregivers have misconceptions that increase\ |
|
\ their stress and \nget in the way of good self-care. Here are some of the most\ |
|
\ commonly \nexpressed: \n• I am responsible for my parent's health. \n• If I\ |
|
\ don't do it, no one will. \n• If I do it right, I will get the love, attention,\ |
|
\ and respect I deserve." |
|
- "possible decline in quality of life. \n \nBut despite these risks, family caregivers\ |
|
\ of any age are less likely than \nnon-caregivers to practice preventive healthcare\ |
|
\ and self-care behavior. \nRegardless of age, sex, and race and ethnicity, caregivers\ |
|
\ report problems \nattending to their own health and well-being while managing\ |
|
\ caregiving \nresponsibilities. They report: \n• sleep deprivation \n• poor\ |
|
\ eating habits \n• failure to exercise \n• failure to stay in bed when ill\ |
|
\ \n• postponement of or failure to make medical appointments ." |
|
- 'Muscle twitches in the arm, leg, shoulder, or tongue |
|
|
|
Muscle cramps |
|
|
|
Tight and stiff muscles (spasticity) |
|
|
|
Muscle weakness affecting an arm, a leg, or the neck |
|
|
|
Slurred and nasal speech |
|
|
|
Difficulty chewing or swallowing |
|
|
|
|
|
As the disease progresses, muscle weakness and atrophy spread to other parts of |
|
your body. People with ALS may develop problems with: |
|
|
|
|
|
Chewing food and swallowing (dysphagia) |
|
|
|
Drooling (sialorrhea) |
|
|
|
Speaking or forming words (dysarthria) |
|
|
|
Breathing (dyspnea) |
|
|
|
Unintended crying, laughing, or other emotional displays (pseudobulbar symptoms) |
|
|
|
Constipation |
|
|
|
Maintaining weight and getting enough nutrients' |
|
- source_sentence: Why is it important for family caregivers to prioritize their own |
|
health and well-being? |
|
sentences: |
|
- 'Cellular defects |
|
|
|
Ongoing studies seek to understand the mechanisms that selectively trigger motor |
|
neurons to degenerate in ALS, which may lead to effective approaches to stop this |
|
process. Research using cellular culture systems and animal models suggests that |
|
motor neuron death is caused by a variety of cellular defects, including those |
|
involved in protein recycling and gene regulation, as well as structural impairments |
|
of motor neurons. Increasing evidence also suggests that glial support cells and |
|
inflammation cells of the nervous system may play an important role in ALS. |
|
|
|
Stem cells' |
|
- "Identifying Personal Barriers \nMany times, attitudes and beliefs form personal\ |
|
\ barriers that stand in the \nway of caring for yourself. Not taking care of\ |
|
\ yourself may be a lifelong \npattern, with taking care of others an easier option.\ |
|
\ However, as a family \ncaregiver you must ask yourself, \"What good will I\ |
|
\ be to the person I care \nfor if I become ill? If I die?\" Breaking old patterns\ |
|
\ and overcoming \nobstacles is not an easy proposition, but it can be done –\ |
|
\ regardless of \nyour age or situation. The first task in removing personal\ |
|
\ barriers to self-\ncare is to identify what is in your way. For example, \n\ |
|
• Do you feel you have to prove that you are worthy of the care recipient's \n\ |
|
affection? \n• Do you think you are being selfish if you put your needs first?\ |
|
\ \n• Is it frightening to think of your own needs? What is the fear about?" |
|
- "Caring for a person living with ALS\nAs the person with ALS progresses in their\ |
|
\ disease, they will need more and more help with daily activities. Being a caregiver\ |
|
\ for a person with ALS, while rewarding, can be challenging for the person’s\ |
|
\ loved ones and caregivers. It is important for caregivers take care of themselves\ |
|
\ and to seek support when needed. Free and paid resources are available to provide\ |
|
\ home health care services and support. Visit the organizations listed at the\ |
|
\ end of this article to find support in your area. \nWhat are the latest updates\ |
|
\ on amyotrophic lateral sclerosis (ALS)?" |
|
- source_sentence: What are some common health issues reported by family caregivers |
|
while managing their caregiving responsibilities? |
|
sentences: |
|
- 'Speech and communication support |
|
|
|
Speech therapists can help people with ALS learn strategies to speak louder and |
|
more clearly and help maintain the ability to communicate. Computer-based speech |
|
synthesizers use eye-tracking devices that allow a person to navigate the web |
|
and to type on custom screens to communicate. Voice banking is a process sometimes |
|
used by people with ALS to store their own voice for future use in computer-based |
|
speech synthesizers. |
|
|
|
A brain-computer interface (BCI) is a system that allows individuals to communicate |
|
or control equipment such as a wheelchair using only brain activity. Researchers |
|
are developing more efficient, mobile BCIs for people with severe paralysis and/or |
|
visual impairments. |
|
|
|
Support for nutrition, breathing, and feeding' |
|
- 'Consider participating in a clinical trial so clinicians and scientists can learn |
|
more about ALS. Clinical research uses human study participants to help researchers |
|
learn more about a disorder and perhaps find better ways to safely detect, treat, |
|
or prevent disease. |
|
|
|
All types of study participants are needed—those who are healthy or may have an |
|
illness or disease—of all different ages, sexes, races, and ethnicities to ensure |
|
that study results apply to as many people as possible, and that treatments will |
|
be safe and effective for everyone who will use them. |
|
|
|
For information about participating in clinical research visit NIH Clinical Research |
|
Trials and You. Learn about clinical trials currently looking for people with ALS at Clinicaltrial.gov.' |
|
- "possible decline in quality of life. \n \nBut despite these risks, family caregivers\ |
|
\ of any age are less likely than \nnon-caregivers to practice preventive healthcare\ |
|
\ and self-care behavior. \nRegardless of age, sex, and race and ethnicity, caregivers\ |
|
\ report problems \nattending to their own health and well-being while managing\ |
|
\ caregiving \nresponsibilities. They report: \n• sleep deprivation \n• poor\ |
|
\ eating habits \n• failure to exercise \n• failure to stay in bed when ill\ |
|
\ \n• postponement of or failure to make medical appointments ." |
|
pipeline_tag: sentence-similarity |
|
library_name: sentence-transformers |
|
metrics: |
|
- cosine_accuracy@1 |
|
- cosine_accuracy@3 |
|
- cosine_accuracy@5 |
|
- cosine_accuracy@10 |
|
- cosine_precision@1 |
|
- cosine_precision@3 |
|
- cosine_precision@5 |
|
- cosine_precision@10 |
|
- cosine_recall@1 |
|
- cosine_recall@3 |
|
- cosine_recall@5 |
|
- cosine_recall@10 |
|
- cosine_ndcg@10 |
|
- cosine_mrr@10 |
|
- cosine_map@100 |
|
model-index: |
|
- name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-l |
|
results: |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: Unknown |
|
type: unknown |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.9166666666666666 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 1.0 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 1.0 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 1.0 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.9166666666666666 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.3333333333333333 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.19999999999999998 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.09999999999999999 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.9166666666666666 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 1.0 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 1.0 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 1.0 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.9692441461309548 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.9583333333333334 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.9583333333333334 |
|
name: Cosine Map@100 |
|
--- |
|
|
|
# SentenceTransformer based on Snowflake/snowflake-arctic-embed-l |
|
|
|
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l). It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
- **Model Type:** Sentence Transformer |
|
- **Base model:** [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l) <!-- at revision d8fb21ca8d905d2832ee8b96c894d3298964346b --> |
|
- **Maximum Sequence Length:** 512 tokens |
|
- **Output Dimensionality:** 1024 dimensions |
|
- **Similarity Function:** Cosine Similarity |
|
<!-- - **Training Dataset:** Unknown --> |
|
<!-- - **Language:** Unknown --> |
|
<!-- - **License:** Unknown --> |
|
|
|
### Model Sources |
|
|
|
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net) |
|
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) |
|
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) |
|
|
|
### Full Model Architecture |
|
|
|
``` |
|
SentenceTransformer( |
|
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel |
|
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) |
|
(2): Normalize() |
|
) |
|
``` |
|
|
|
## Usage |
|
|
|
### Direct Usage (Sentence Transformers) |
|
|
|
First install the Sentence Transformers library: |
|
|
|
```bash |
|
pip install -U sentence-transformers |
|
``` |
|
|
|
Then you can load this model and run inference. |
|
```python |
|
from sentence_transformers import SentenceTransformer |
|
|
|
# Download from the 🤗 Hub |
|
model = SentenceTransformer("ernestobs7/caregiver-ft-v0") |
|
# Run inference |
|
sentences = [ |
|
'What are some common health issues reported by family caregivers while managing their caregiving responsibilities?', |
|
'possible decline in quality of life. \n \nBut despite these risks, family caregivers of any age are less likely than \nnon-caregivers to practice preventive healthcare and self-care behavior. \nRegardless of age, sex, and race and ethnicity, caregivers report problems \nattending to their own health and well-being while managing caregiving \nresponsibilities. They report: \n• sleep deprivation \n• poor eating habits \n• failure to exercise \n• failure to stay in bed when ill \n• postponement of or failure to make medical appointments .', |
|
'Consider participating in a clinical trial so clinicians and scientists can learn more about\xa0ALS.\xa0Clinical research uses human study participants to help researchers learn more about a disorder and perhaps find better ways to safely detect, treat, or prevent disease.\nAll types of study participants are needed—those who are healthy or may have an illness or disease—of all different ages, sexes, races, and ethnicities to ensure that study results apply to as many people as possible, and that treatments will be safe and effective for everyone who will use them.\nFor information about participating in clinical research visit\xa0NIH Clinical Research Trials\xa0and You. Learn about clinical trials\xa0currently looking for people with\xa0ALS\xa0at\xa0Clinicaltrial.gov.', |
|
] |
|
embeddings = model.encode(sentences) |
|
print(embeddings.shape) |
|
# [3, 1024] |
|
|
|
# Get the similarity scores for the embeddings |
|
similarities = model.similarity(embeddings, embeddings) |
|
print(similarities.shape) |
|
# [3, 3] |
|
``` |
|
|
|
<!-- |
|
### Direct Usage (Transformers) |
|
|
|
<details><summary>Click to see the direct usage in Transformers</summary> |
|
|
|
</details> |
|
--> |
|
|
|
<!-- |
|
### Downstream Usage (Sentence Transformers) |
|
|
|
You can finetune this model on your own dataset. |
|
|
|
<details><summary>Click to expand</summary> |
|
|
|
</details> |
|
--> |
|
|
|
<!-- |
|
### Out-of-Scope Use |
|
|
|
*List how the model may foreseeably be misused and address what users ought not to do with the model.* |
|
--> |
|
|
|
## Evaluation |
|
|
|
### Metrics |
|
|
|
#### Information Retrieval |
|
|
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.9167 | |
|
| cosine_accuracy@3 | 1.0 | |
|
| cosine_accuracy@5 | 1.0 | |
|
| cosine_accuracy@10 | 1.0 | |
|
| cosine_precision@1 | 0.9167 | |
|
| cosine_precision@3 | 0.3333 | |
|
| cosine_precision@5 | 0.2 | |
|
| cosine_precision@10 | 0.1 | |
|
| cosine_recall@1 | 0.9167 | |
|
| cosine_recall@3 | 1.0 | |
|
| cosine_recall@5 | 1.0 | |
|
| cosine_recall@10 | 1.0 | |
|
| **cosine_ndcg@10** | **0.9692** | |
|
| cosine_mrr@10 | 0.9583 | |
|
| cosine_map@100 | 0.9583 | |
|
|
|
<!-- |
|
## Bias, Risks and Limitations |
|
|
|
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* |
|
--> |
|
|
|
<!-- |
|
### Recommendations |
|
|
|
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* |
|
--> |
|
|
|
## Training Details |
|
|
|
### Training Dataset |
|
|
|
#### Unnamed Dataset |
|
|
|
* Size: 98 training samples |
|
* Columns: <code>sentence_0</code> and <code>sentence_1</code> |
|
* Approximate statistics based on the first 98 samples: |
|
| | sentence_0 | sentence_1 | |
|
|:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------| |
|
| type | string | string | |
|
| details | <ul><li>min: 13 tokens</li><li>mean: 19.41 tokens</li><li>max: 34 tokens</li></ul> | <ul><li>min: 30 tokens</li><li>mean: 120.29 tokens</li><li>max: 181 tokens</li></ul> | |
|
* Samples: |
|
| sentence_0 | sentence_1 | |
|
|:-------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |
|
| <code>What types of examinations are performed by healthcare providers to assess for ALS?</code> | <code>There is no single test that can definitely diagnose ALS. A healthcare provider will conduct a physical exam and review the person’s full medical history. A neurologic examination will test reflexes, muscle strength, and other responses. These tests should be performed at regular intervals to assess whether symptoms are getting worse over time.<br>A healthcare provider may conduct muscle and imaging tests to rule out other diseases. This can help support an ALS diagnosis. These tests include:</code> | |
|
| <code>Why are muscle and imaging tests conducted in the process of diagnosing ALS?</code> | <code>There is no single test that can definitely diagnose ALS. A healthcare provider will conduct a physical exam and review the person’s full medical history. A neurologic examination will test reflexes, muscle strength, and other responses. These tests should be performed at regular intervals to assess whether symptoms are getting worse over time.<br>A healthcare provider may conduct muscle and imaging tests to rule out other diseases. This can help support an ALS diagnosis. These tests include:</code> | |
|
| <code>What are some factors that influence an individual's level of stress in a caregiving situation?</code> | <code>Moving Forward <br>Once you've started to identify any personal barriers to good self -care, you <br>can begin to change your behavior, moving forward one small step at a <br>time. Following are some effective tools for self-care that can start you on <br>your way. <br> <br>Tool #1: Reducing Personal Stress <br>How we perceive and respond to an event is a significant factor in how we <br>adjust and cope with it. The stress you feel is not only the result of your <br>caregiving situation but also the result of your perception of it – whether <br>you see the glass as half-full or half-empty. It is important to remember <br>that you are not alone in your experiences. <br> <br>Your level of stress is influenced by many factors, including the following: <br>• Whether your caregiving is voluntary. If you feel you had no choice in</code> | |
|
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: |
|
```json |
|
{ |
|
"loss": "MultipleNegativesRankingLoss", |
|
"matryoshka_dims": [ |
|
768, |
|
512, |
|
256, |
|
128, |
|
64 |
|
], |
|
"matryoshka_weights": [ |
|
1, |
|
1, |
|
1, |
|
1, |
|
1 |
|
], |
|
"n_dims_per_step": -1 |
|
} |
|
``` |
|
|
|
### Training Hyperparameters |
|
#### Non-Default Hyperparameters |
|
|
|
- `eval_strategy`: steps |
|
- `per_device_train_batch_size`: 10 |
|
- `per_device_eval_batch_size`: 10 |
|
- `num_train_epochs`: 10 |
|
- `multi_dataset_batch_sampler`: round_robin |
|
|
|
#### All Hyperparameters |
|
<details><summary>Click to expand</summary> |
|
|
|
- `overwrite_output_dir`: False |
|
- `do_predict`: False |
|
- `eval_strategy`: steps |
|
- `prediction_loss_only`: True |
|
- `per_device_train_batch_size`: 10 |
|
- `per_device_eval_batch_size`: 10 |
|
- `per_gpu_train_batch_size`: None |
|
- `per_gpu_eval_batch_size`: None |
|
- `gradient_accumulation_steps`: 1 |
|
- `eval_accumulation_steps`: None |
|
- `torch_empty_cache_steps`: None |
|
- `learning_rate`: 5e-05 |
|
- `weight_decay`: 0.0 |
|
- `adam_beta1`: 0.9 |
|
- `adam_beta2`: 0.999 |
|
- `adam_epsilon`: 1e-08 |
|
- `max_grad_norm`: 1 |
|
- `num_train_epochs`: 10 |
|
- `max_steps`: -1 |
|
- `lr_scheduler_type`: linear |
|
- `lr_scheduler_kwargs`: {} |
|
- `warmup_ratio`: 0.0 |
|
- `warmup_steps`: 0 |
|
- `log_level`: passive |
|
- `log_level_replica`: warning |
|
- `log_on_each_node`: True |
|
- `logging_nan_inf_filter`: True |
|
- `save_safetensors`: True |
|
- `save_on_each_node`: False |
|
- `save_only_model`: False |
|
- `restore_callback_states_from_checkpoint`: False |
|
- `no_cuda`: False |
|
- `use_cpu`: False |
|
- `use_mps_device`: False |
|
- `seed`: 42 |
|
- `data_seed`: None |
|
- `jit_mode_eval`: False |
|
- `use_ipex`: False |
|
- `bf16`: False |
|
- `fp16`: False |
|
- `fp16_opt_level`: O1 |
|
- `half_precision_backend`: auto |
|
- `bf16_full_eval`: False |
|
- `fp16_full_eval`: False |
|
- `tf32`: None |
|
- `local_rank`: 0 |
|
- `ddp_backend`: None |
|
- `tpu_num_cores`: None |
|
- `tpu_metrics_debug`: False |
|
- `debug`: [] |
|
- `dataloader_drop_last`: False |
|
- `dataloader_num_workers`: 0 |
|
- `dataloader_prefetch_factor`: None |
|
- `past_index`: -1 |
|
- `disable_tqdm`: False |
|
- `remove_unused_columns`: True |
|
- `label_names`: None |
|
- `load_best_model_at_end`: False |
|
- `ignore_data_skip`: False |
|
- `fsdp`: [] |
|
- `fsdp_min_num_params`: 0 |
|
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} |
|
- `fsdp_transformer_layer_cls_to_wrap`: None |
|
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} |
|
- `deepspeed`: None |
|
- `label_smoothing_factor`: 0.0 |
|
- `optim`: adamw_torch |
|
- `optim_args`: None |
|
- `adafactor`: False |
|
- `group_by_length`: False |
|
- `length_column_name`: length |
|
- `ddp_find_unused_parameters`: None |
|
- `ddp_bucket_cap_mb`: None |
|
- `ddp_broadcast_buffers`: False |
|
- `dataloader_pin_memory`: True |
|
- `dataloader_persistent_workers`: False |
|
- `skip_memory_metrics`: True |
|
- `use_legacy_prediction_loop`: False |
|
- `push_to_hub`: False |
|
- `resume_from_checkpoint`: None |
|
- `hub_model_id`: None |
|
- `hub_strategy`: every_save |
|
- `hub_private_repo`: None |
|
- `hub_always_push`: False |
|
- `gradient_checkpointing`: False |
|
- `gradient_checkpointing_kwargs`: None |
|
- `include_inputs_for_metrics`: False |
|
- `include_for_metrics`: [] |
|
- `eval_do_concat_batches`: True |
|
- `fp16_backend`: auto |
|
- `push_to_hub_model_id`: None |
|
- `push_to_hub_organization`: None |
|
- `mp_parameters`: |
|
- `auto_find_batch_size`: False |
|
- `full_determinism`: False |
|
- `torchdynamo`: None |
|
- `ray_scope`: last |
|
- `ddp_timeout`: 1800 |
|
- `torch_compile`: False |
|
- `torch_compile_backend`: None |
|
- `torch_compile_mode`: None |
|
- `dispatch_batches`: None |
|
- `split_batches`: None |
|
- `include_tokens_per_second`: False |
|
- `include_num_input_tokens_seen`: False |
|
- `neftune_noise_alpha`: None |
|
- `optim_target_modules`: None |
|
- `batch_eval_metrics`: False |
|
- `eval_on_start`: False |
|
- `use_liger_kernel`: False |
|
- `eval_use_gather_object`: False |
|
- `average_tokens_across_devices`: False |
|
- `prompts`: None |
|
- `batch_sampler`: batch_sampler |
|
- `multi_dataset_batch_sampler`: round_robin |
|
|
|
</details> |
|
|
|
### Training Logs |
|
| Epoch | Step | cosine_ndcg@10 | |
|
|:-----:|:----:|:--------------:| |
|
| 1.0 | 10 | 0.9728 | |
|
| 2.0 | 20 | 0.9728 | |
|
| 3.0 | 30 | 0.9434 | |
|
| 4.0 | 40 | 0.9462 | |
|
| 5.0 | 50 | 0.9616 | |
|
| 6.0 | 60 | 0.9616 | |
|
| 7.0 | 70 | 0.9588 | |
|
| 8.0 | 80 | 0.9616 | |
|
| 9.0 | 90 | 0.9692 | |
|
| 10.0 | 100 | 0.9692 | |
|
|
|
|
|
### Framework Versions |
|
- Python: 3.11.4 |
|
- Sentence Transformers: 3.4.1 |
|
- Transformers: 4.49.0 |
|
- PyTorch: 2.6.0+cu124 |
|
- Accelerate: 1.4.0 |
|
- Datasets: 3.3.2 |
|
- Tokenizers: 0.21.0 |
|
|
|
## Citation |
|
|
|
### BibTeX |
|
|
|
#### Sentence Transformers |
|
```bibtex |
|
@inproceedings{reimers-2019-sentence-bert, |
|
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", |
|
author = "Reimers, Nils and Gurevych, Iryna", |
|
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", |
|
month = "11", |
|
year = "2019", |
|
publisher = "Association for Computational Linguistics", |
|
url = "https://arxiv.org/abs/1908.10084", |
|
} |
|
``` |
|
|
|
#### MatryoshkaLoss |
|
```bibtex |
|
@misc{kusupati2024matryoshka, |
|
title={Matryoshka Representation Learning}, |
|
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, |
|
year={2024}, |
|
eprint={2205.13147}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.LG} |
|
} |
|
``` |
|
|
|
#### MultipleNegativesRankingLoss |
|
```bibtex |
|
@misc{henderson2017efficient, |
|
title={Efficient Natural Language Response Suggestion for Smart Reply}, |
|
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, |
|
year={2017}, |
|
eprint={1705.00652}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CL} |
|
} |
|
``` |
|
|
|
<!-- |
|
## Glossary |
|
|
|
*Clearly define terms in order to be accessible across audiences.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Authors |
|
|
|
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Contact |
|
|
|
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* |
|
--> |
