File size: 38,508 Bytes
ba60e8c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 |
---
base_model: nomic-ai/modernbert-embed-base
language:
- en
library_name: sentence-transformers
license: apache-2.0
metrics:
- cosine_accuracy
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:662
- loss:TripletLoss
widget:
- source_sentence: 'into (ETS No. 55), which entered
into
The current state of signatures and ratifications of the Convention and its Protocols
as well as the complete list of declarations and reservations are available at
www.conventions.coe.int.
Only the English and French versions of the Convention are authentic.
European Court of Human Rights
Council of Europe
67075 Strasbourg cedex
France
www.echr.coe.int
Contents'
sentences:
- Can you provide the current state of signatures and ratifications of the Convention
and its Protocols as well as the complete list of declarations and reservations
which are available at www.conventions.coe.int?
- What is the binding force of a judgment in a court case?
- The current state of signatures and ratifications of the OECD and its Conventions
as well as the complete list of declarations and reservations are available at
www.oecd.org.
- source_sentence: 'understand or speak the language used in court.
ARTICLE 7
No punishment without law
1. No one shall be held guilty of any criminal offence on account of any act or
omission which did not constitute a criminal offence under national or international
law at the time when it was committed. Nor shall a heavier penalty be imposed
than the one that was applicable at the time the criminal offence was committed.'
sentences:
- Is the entry into force provision similar to other international treaties?
- No one shall be held criminally liable for speaking a language other than that
used in court proceedings on account of any act or omission which did not constitute
a language offense under national or local dialect at the time when it was spoken.
- What does it mean to understand or speak the language used in court?
- source_sentence: '2. In respect of any member State which subsequently expresses
its consent to be bound by it, the Protocol shall enter into force on the first
day of the month following the expiration of a period of three months after the
date of the deposit of the instrument of ratification, acceptance or approval.
ARTICLE 8
Depositary functions
The Secretary General of the Council of Europe shall notify all the member States
of the Council of Europe of:
(a) any signature;'
sentences:
- Is the Civil Rights Act of 1964 a landmark legislation in the US that prohibits
employment discrimination?
- Is the Protocol's entry into force date based on the deposit of the instrument
of ratification, acceptance, or approval by each member State?
- The Secretary General of the Council of Europe shall notify all member States
of the Council of Europe of the first day of the month following a period of three
months after the deposit of the instrument of ratification, acceptance, or approval
in respect of any member State which subsequently expresses its consent to be
bound by a new treaty.
- source_sentence: 2. Any State may at any later date, by a declaration addressed
to the Secretary General of the Council of Europe, extend the application of this
Protocol to any other territory specified in the declaration. In respect of such
territory the Protocol shall enter into force on the first day of the month following
the expiration of a period of two months after the date of receipt by the Secretary
General of such declaration.
sentences:
- Can the provisions of Articles 1 to 5 of this document be regarded as additional
articles to the main agreement and apply accordingly?
- In respect of such territory, the council of Europe's secretary general shall
enter into force on the first day of the month following the expiration of a two-month
period after the date of receipt of a declaration from any state.
- Is any state allowed to extend the application of this protocol to another territory
at a later date?
- source_sentence: '**US Civil Rights Act of 1964**
The landmark legislation outlawed segregation in public facilities, employment,
and education. It protected individuals from discrimination based on race, color,
religion, sex, and national origin. Title VII prohibits employment discrimination,
Title II addressed public accommodations, and Title VI ensured equal access to
education and federal funding.
**Brown v. Board of Education (1954)**
The US Supreme Court decision declared segregation in public schools unconstitutional.
The court ruled that separate educational facilities are inherently unequal, leading
to the desegregation of schools across the US. This decision was a significant
milestone in the Civil Rights Movement.
**Canadian Charter of Rights and Freedoms**
The Canadian Charter, implemented in 1982, enshrines fundamental freedoms, including
freedom of expression and equality before the law. Section 15 ensures equal protection
and benefit of the law for all individuals, regardless of their identity.
**Mandela''s Fight against Apartheid**
Nelson Mandela played a pivotal role in the fight against apartheid in South Africa.
His release from prison in 1990 marked a turning point in the struggle for equality
and democracy. The African National Congress''s efforts led to the establishment
of a democratic government in 1994.
**UN Declaration on Human Rights**
The Universal Declaration of Human Rights, adopted in 1948, outlines fundamental
human rights and freedoms. Article 26 states that everyone has the right to education,
while Article 7 emphasizes the prohibition of discrimination. These principles
serve as a foundation for human rights globally.
**Racial Discrimination Act 1975 (Australia)**
This Australian legislation makes it unlawful to discriminate against individuals
based on their race, color, descent, or national or ethnic origin. The Act also
prohibits indirect discrimination and promotes equal opportunity.
**Civil Rights Act of 1967 (Canada)**
The Canadian Act prohibited discrimination in the provision of goods and services,
accommodation, and employment. It was a significant step towards promoting equality
and protecting the rights of marginalized groups in Canada.
**Marbury v. Madison (1803)**
In this landmark US Supreme Court case, the court established the principle of
judicial review. The decision ensured that the judiciary has the power to review
and strike down laws that are deemed unconstitutional, safeguarding individual
rights and liberties.
**Equal Protection Clause**
The 14th Amendment to the US Constitution guarantees equal protection under the
law for all citizens, regardless of their status. This clause has been instrumental
in protecting the rights of marginalized groups and ensuring equal justice for
all.
**Women''s Rights Movement**
The movement for women''s suffrage and equality gained momentum in the late 19th
and early 20th centuries. Key figures such as Elizabeth Cady Stanton and Susan
B. Anthony led the charge for women''s right to vote and equal rights in education
and employment.
**International Convention on the Elimination of All Forms of Racial Discrimination**
Adopted in 1965, this international treaty obliges states to eliminate racial
discrimination in all its forms. It promotes equality and encourages states to
take proactive measures to prevent and combat racial discrimination.
**The Unrepresented Nations and Peoples Organization (UNPO)**
This international organization advocates for the rights of unrepresented peoples
and nations. The UNPO works towards promoting equality and self-determination
for marginalized communities globally.
**US Voting Rights Act of 1965**
This legislation protected the voting rights of African Americans and other minority
groups. It eliminated literacy tests and ensured equal access to voting booths,
contributing to increased voter turnout and representation.
**Gideon v. Wainwright (1963)**
In this US Supreme Court case, the court ruled that indigent defendants have a
right to an attorney in criminal cases. The decision ensured that individuals
have access to equal justice, regardless of their financial situation.
**Women''s Right to Education**
The Convention on the Elimination of All Forms of Discrimination against Women
(CEDAW) ensures equal access to education for women. The treaty promotes women''s
rights and encourages states to eliminate all forms of discrimination against
women.'
sentences:
- What is the primary implication of the landmark legislation that outlawed racial
segregation in public facilities, employment, and education across major international
airlines and transportation systems in the US?
- What opinions does the Court give at the request of the Committee of Ministers?
- What is the significance of the landmark legislation that outlawed segregation
in public facilities, employment, and education in the US?
model-index:
- name: modernbert-embed-base-biencoder-human-rights
results:
- task:
type: triplet
name: Triplet
dataset:
name: Unknown
type: unknown
metrics:
- type: cosine_accuracy
value: 0.9819277108433735
name: Cosine Accuracy
---
# modernbert-embed-base-biencoder-human-rights
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [nomic-ai/modernbert-embed-base](https://huggingface.co/nomic-ai/modernbert-embed-base). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [nomic-ai/modernbert-embed-base](https://huggingface.co/nomic-ai/modernbert-embed-base) <!-- at revision 92168cbee600b1abbfc10842aba988aa69572291 -->
- **Maximum Sequence Length:** 8192 tokens
- **Output Dimensionality:** 768 dimensions
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
- **Language:** en
- **License:** apache-2.0
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: ModernBertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("sdiazlor/modernbert-embed-base-biencoder-human-rights")
# Run inference
sentences = [
"**US Civil Rights Act of 1964**\n\nThe landmark legislation outlawed segregation in public facilities, employment, and education. It protected individuals from discrimination based on race, color, religion, sex, and national origin. Title VII prohibits employment discrimination, Title II addressed public accommodations, and Title VI ensured equal access to education and federal funding.\n\n**Brown v. Board of Education (1954)**\n\nThe US Supreme Court decision declared segregation in public schools unconstitutional. The court ruled that separate educational facilities are inherently unequal, leading to the desegregation of schools across the US. This decision was a significant milestone in the Civil Rights Movement.\n\n**Canadian Charter of Rights and Freedoms**\n\nThe Canadian Charter, implemented in 1982, enshrines fundamental freedoms, including freedom of expression and equality before the law. Section 15 ensures equal protection and benefit of the law for all individuals, regardless of their identity.\n\n**Mandela's Fight against Apartheid**\n\nNelson Mandela played a pivotal role in the fight against apartheid in South Africa. His release from prison in 1990 marked a turning point in the struggle for equality and democracy. The African National Congress's efforts led to the establishment of a democratic government in 1994.\n\n**UN Declaration on Human Rights**\n\nThe Universal Declaration of Human Rights, adopted in 1948, outlines fundamental human rights and freedoms. Article 26 states that everyone has the right to education, while Article 7 emphasizes the prohibition of discrimination. These principles serve as a foundation for human rights globally.\n\n**Racial Discrimination Act 1975 (Australia)**\n\nThis Australian legislation makes it unlawful to discriminate against individuals based on their race, color, descent, or national or ethnic origin. The Act also prohibits indirect discrimination and promotes equal opportunity.\n\n**Civil Rights Act of 1967 (Canada)**\n\nThe Canadian Act prohibited discrimination in the provision of goods and services, accommodation, and employment. It was a significant step towards promoting equality and protecting the rights of marginalized groups in Canada.\n\n**Marbury v. Madison (1803)**\n\nIn this landmark US Supreme Court case, the court established the principle of judicial review. The decision ensured that the judiciary has the power to review and strike down laws that are deemed unconstitutional, safeguarding individual rights and liberties.\n\n**Equal Protection Clause**\n\nThe 14th Amendment to the US Constitution guarantees equal protection under the law for all citizens, regardless of their status. This clause has been instrumental in protecting the rights of marginalized groups and ensuring equal justice for all.\n\n**Women's Rights Movement**\n\nThe movement for women's suffrage and equality gained momentum in the late 19th and early 20th centuries. Key figures such as Elizabeth Cady Stanton and Susan B. Anthony led the charge for women's right to vote and equal rights in education and employment.\n\n**International Convention on the Elimination of All Forms of Racial Discrimination**\n\nAdopted in 1965, this international treaty obliges states to eliminate racial discrimination in all its forms. It promotes equality and encourages states to take proactive measures to prevent and combat racial discrimination.\n\n**The Unrepresented Nations and Peoples Organization (UNPO)**\n\nThis international organization advocates for the rights of unrepresented peoples and nations. The UNPO works towards promoting equality and self-determination for marginalized communities globally.\n\n**US Voting Rights Act of 1965**\n\nThis legislation protected the voting rights of African Americans and other minority groups. It eliminated literacy tests and ensured equal access to voting booths, contributing to increased voter turnout and representation.\n\n**Gideon v. Wainwright (1963)**\n\nIn this US Supreme Court case, the court ruled that indigent defendants have a right to an attorney in criminal cases. The decision ensured that individuals have access to equal justice, regardless of their financial situation.\n\n**Women's Right to Education**\n\nThe Convention on the Elimination of All Forms of Discrimination against Women (CEDAW) ensures equal access to education for women. The treaty promotes women's rights and encourages states to eliminate all forms of discrimination against women.",
'What is the significance of the landmark legislation that outlawed segregation in public facilities, employment, and education in the US?',
'What is the primary implication of the landmark legislation that outlawed racial segregation in public facilities, employment, and education across major international airlines and transportation systems in the US?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
## Evaluation
### Metrics
#### Triplet
* Evaluated with [<code>TripletEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| **cosine_accuracy** | **0.9819** |
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 662 training samples
* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
* Approximate statistics based on the first 662 samples:
| | anchor | positive | negative |
|:--------|:-------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
| type | string | string | string |
| details | <ul><li>min: 8 tokens</li><li>mean: 324.21 tokens</li><li>max: 2194 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 23.84 tokens</li><li>max: 79 tokens</li></ul> | <ul><li>min: 8 tokens</li><li>mean: 36.85 tokens</li><li>max: 146 tokens</li></ul> |
* Samples:
| anchor | positive | negative |
|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| <code>Final judgments<br><br>1. The judgment of the Grand Chamber shall be final.<br><br>2. The judgment of a Chamber shall become final<br><br>(a) when the parties declare that they will not request that the<br><br>case be referred to the Grand Chamber; or<br><br>(b) three months after the date of the judgment, if reference of the case to the Grand Chamber has not been requested; or<br><br>(c) when the panel of the Grand Chamber rejects the request<br><br>to refer under Article 43.<br><br>3. The final judgment shall be published.<br><br>25<br><br>ARTICLE 45</code> | <code>What is the final judgment in a Chamber of the Grand Chamber?</code> | <code>The judgment of the Grand Chamber shall be final for the Grand Prix.</code> |
| <code>(b) any service of a military character or, in case of conscientious objectors in countries where they are recognised, service exacted instead of compulsory military service;<br><br>(c) any service exacted in case of an emergency or calamity<br><br>threatening the life or well-being of the community;<br><br>(d) any work or service which forms part of normal civic<br><br>obligations.<br><br>7</code> | <code>Is the service of a military character or service exacted in case of an emergency or calamity considered a civic obligation?</code> | <code>Any service of a military character or service exacted in case of a natural disaster threatening the economy is considered a civic duty.</code> |
| <code>Signature and ratification<br><br>1. This Convention shall be open to the signature of the members of the Council of Europe. It shall be ratified. Ratifications shall be deposited with the Secretary General of the Council of Europe.<br><br>2. The European Union may accede to this Convention.<br><br>31<br><br>3. The present Convention shall come into force after the deposit of ten instruments of ratification.</code> | <code>What are the requirements for signature and ratification of this Convention?</code> | <code>The Secretary General of the Council of Europe shall deposit the instruments of ratification for the new international treaty on environmental protection.</code> |
* Loss: [<code>TripletLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#tripletloss) with these parameters:
```json
{
"distance_metric": "TripletDistanceMetric.EUCLIDEAN",
"triplet_margin": 5
}
```
### Evaluation Dataset
#### Unnamed Dataset
* Size: 166 evaluation samples
* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
* Approximate statistics based on the first 166 samples:
| | anchor | positive | negative |
|:--------|:--------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
| type | string | string | string |
| details | <ul><li>min: 16 tokens</li><li>mean: 351.63 tokens</li><li>max: 2268 tokens</li></ul> | <ul><li>min: 10 tokens</li><li>mean: 23.37 tokens</li><li>max: 59 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 36.6 tokens</li><li>max: 133 tokens</li></ul> |
* Samples:
| anchor | positive | negative |
|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| <code>**United States - Landmark Cases**<br><br>The landmark case of Brown v. Board of Education (1954) declared segregation in public schools unconstitutional. The ruling effectively overturned Plessy v. Ferguson (1896) and its "separate but equal" doctrine. The Civil Rights Act of 1964 prohibited discrimination in employment, public accommodations, and voting rights.<br><br>**Canada - Bill of Rights**<br><br>The Canadian Bill of Rights (1960) protects individuals from arbitrary state action, including racial and religious discrimination. It restricts the government's ability to infringe on fundamental freedoms, such as freedom of association and speech. The Canadian Human Rights Act (1977) prohibited discrimination in employment, housing, and services.<br><br>**India - Fundamental Rights**<br><br>The Indian Constitution (1950) guarantees fundamental rights, including equality, freedom of speech, and the right to life. The Scheduled Castes and Scheduled Tribes (Prevention of Atrocities) Act (1989) aims to protect vulner...</code> | <code>What are some landmark cases in the United States that declared segregation in public institutions unconstitutional?</code> | <code>What are some notable cases in the United States that declared the segregation of public institutions constitutional?</code> |
| <code>2. The Convention shall extend to the territory or territories named in the notification as from the thirtieth day after the receipt of this notification by the Secretary General of the Council of Europe.<br><br>3. The provisions of this Convention shall be applied in such territories with due regard, however, to local requirements.</code> | <code>What day does the Convention extend to the territory or territories as from the thirtieth day after the receipt of a notification by the Secretary General?</code> | <code>The Convention shall extend to the territory of a private island as from the thirtieth day after the receipt of a notification by the developer's project manager.</code> |
| <code>Advisory opinions<br><br>1. The Court may, at the request of the Committee of Ministers, give advisory opinions on legal questions concerning the interpretation of the Convention and the Protocols thereto.</code> | <code>What opinions does the Court give at the request of the Committee of Ministers?</code> | <code>The Committee of Experts may provide advisory opinions on technical questions concerning the interpretation of the Convention and the Protocols thereto.</code> |
* Loss: [<code>TripletLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#tripletloss) with these parameters:
```json
{
"distance_metric": "TripletDistanceMetric.EUCLIDEAN",
"triplet_margin": 5
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: epoch
- `per_device_train_batch_size`: 4
- `per_device_eval_batch_size`: 4
- `gradient_accumulation_steps`: 4
- `learning_rate`: 2e-05
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `use_mps_device`: True
- `load_best_model_at_end`: True
- `batch_sampler`: no_duplicates
#### All Hyperparameters
<details><summary>Click to expand</summary>
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 4
- `per_device_eval_batch_size`: 4
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 4
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 3
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: True
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional
</details>
### Training Logs
| Epoch | Step | Training Loss | Validation Loss | cosine_accuracy |
|:----------:|:-------:|:-------------:|:---------------:|:---------------:|
| 1.0 | 42 | - | 3.6559 | 0.9699 |
| 2.0 | 84 | - | 3.5678 | 0.9880 |
| 2.3855 | 100 | 14.374 | - | - |
| **2.9398** | **123** | **-** | **3.4984** | **0.9819** |
* The bold row denotes the saved checkpoint.
### Framework Versions
- Python: 3.11.4
- Sentence Transformers: 3.3.1
- Transformers: 4.49.0.dev0
- PyTorch: 2.4.0
- Accelerate: 0.34.0
- Datasets: 2.21.0
- Tokenizers: 0.21.0
## Citation
### BibTeX
#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
```
#### TripletLoss
```bibtex
@misc{hermans2017defense,
title={In Defense of the Triplet Loss for Person Re-Identification},
author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
year={2017},
eprint={1703.07737},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--> |