Add new SentenceTransformer model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,628 @@
+---
+base_model: nomic-ai/modernbert-embed-base
+language:
+- en
+library_name: sentence-transformers
+license: apache-2.0
+metrics:
+- cosine_accuracy
+pipeline_tag: sentence-similarity
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:662
+- loss:TripletLoss
+widget:
+- source_sentence: 'into (ETS No. 55), which entered
+
+
+    into
+
+
+    The current state of signatures and ratifications of the Convention and its Protocols
+    as well as the complete list of declarations and reservations are available at
+    www.conventions.coe.int.
+
+
+    Only the English and French versions of the Convention are authentic.
+
+
+    European Court of Human Rights
+
+
+    Council of Europe
+
+
+    67075 Strasbourg cedex
+
+
+    France
+
+
+    www.echr.coe.int
+
+
+    Contents'
+  sentences:
+  - Can you provide the current state of signatures and ratifications of the Convention
+    and its Protocols as well as the complete list of declarations and reservations
+    which are available at www.conventions.coe.int?
+  - What is the binding force of a judgment in a court case?
+  - The current state of signatures and ratifications of the OECD and its Conventions
+    as well as the complete list of declarations and reservations are available at
+    www.oecd.org.
+- source_sentence: 'understand or speak the language used in court.
+
+
+    ARTICLE 7
+
+
+    No punishment without law
+
+
+    1. No one shall be held guilty of any criminal offence on account of any act or
+    omission which did not constitute a criminal offence under national or international
+    law at the time when it was committed. Nor shall a heavier penalty be imposed
+    than the one that was applicable at the time the criminal offence was committed.'
+  sentences:
+  - Is the entry into force provision similar to other international treaties?
+  - No one shall be held criminally liable for speaking a language other than that
+    used in court proceedings on account of any act or omission which did not constitute
+    a language offense under national or local dialect at the time when it was spoken.
+  - What does it mean to understand or speak the language used in court?
+- source_sentence: '2. In respect of any member State which subsequently expresses
+    its consent to be bound by it, the Protocol shall enter into force on the first
+    day of the month following the expiration of a period of three months after the
+    date of the deposit of the instrument of ratification, acceptance or approval.
+    ARTICLE 8
+
+
+    Depositary functions
+
+
+    The Secretary General of the Council of Europe shall notify all the member States
+    of the Council of Europe of:
+
+
+    (a) any signature;'
+  sentences:
+  - Is the Civil Rights Act of 1964 a landmark legislation in the US that prohibits
+    employment discrimination?
+  - Is the Protocol's entry into force date based on the deposit of the instrument
+    of ratification, acceptance, or approval by each member State?
+  - The Secretary General of the Council of Europe shall notify all member States
+    of the Council of Europe of the first day of the month following a period of three
+    months after the deposit of the instrument of ratification, acceptance, or approval
+    in respect of any member State which subsequently expresses its consent to be
+    bound by a new treaty.
+- source_sentence: 2. Any State may at any later date, by a declaration addressed
+    to the Secretary General of the Council of Europe, extend the application of this
+    Protocol to any other territory specified in the declaration. In respect of such
+    territory the Protocol shall enter into force on the first day of the month following
+    the expiration of a period of two months after the date of receipt by the Secretary
+    General of such declaration.
+  sentences:
+  - Can the provisions of Articles 1 to 5 of this document be regarded as additional
+    articles to the main agreement and apply accordingly?
+  - In respect of such territory, the council of Europe's secretary general shall
+    enter into force on the first day of the month following the expiration of a two-month
+    period after the date of receipt of a declaration from any state.
+  - Is any state allowed to extend the application of this protocol to another territory
+    at a later date?
+- source_sentence: '**US Civil Rights Act of 1964**
+
+
+    The landmark legislation outlawed segregation in public facilities, employment,
+    and education. It protected individuals from discrimination based on race, color,
+    religion, sex, and national origin. Title VII prohibits employment discrimination,
+    Title II addressed public accommodations, and Title VI ensured equal access to
+    education and federal funding.
+
+
+    **Brown v. Board of Education (1954)**
+
+
+    The US Supreme Court decision declared segregation in public schools unconstitutional.
+    The court ruled that separate educational facilities are inherently unequal, leading
+    to the desegregation of schools across the US. This decision was a significant
+    milestone in the Civil Rights Movement.
+
+
+    **Canadian Charter of Rights and Freedoms**
+
+
+    The Canadian Charter, implemented in 1982, enshrines fundamental freedoms, including
+    freedom of expression and equality before the law. Section 15 ensures equal protection
+    and benefit of the law for all individuals, regardless of their identity.
+
+
+    **Mandela''s Fight against Apartheid**
+
+
+    Nelson Mandela played a pivotal role in the fight against apartheid in South Africa.
+    His release from prison in 1990 marked a turning point in the struggle for equality
+    and democracy. The African National Congress''s efforts led to the establishment
+    of a democratic government in 1994.
+
+
+    **UN Declaration on Human Rights**
+
+
+    The Universal Declaration of Human Rights, adopted in 1948, outlines fundamental
+    human rights and freedoms. Article 26 states that everyone has the right to education,
+    while Article 7 emphasizes the prohibition of discrimination. These principles
+    serve as a foundation for human rights globally.
+
+
+    **Racial Discrimination Act 1975 (Australia)**
+
+
+    This Australian legislation makes it unlawful to discriminate against individuals
+    based on their race, color, descent, or national or ethnic origin. The Act also
+    prohibits indirect discrimination and promotes equal opportunity.
+
+
+    **Civil Rights Act of 1967 (Canada)**
+
+
+    The Canadian Act prohibited discrimination in the provision of goods and services,
+    accommodation, and employment. It was a significant step towards promoting equality
+    and protecting the rights of marginalized groups in Canada.
+
+
+    **Marbury v. Madison (1803)**
+
+
+    In this landmark US Supreme Court case, the court established the principle of
+    judicial review. The decision ensured that the judiciary has the power to review
+    and strike down laws that are deemed unconstitutional, safeguarding individual
+    rights and liberties.
+
+
+    **Equal Protection Clause**
+
+
+    The 14th Amendment to the US Constitution guarantees equal protection under the
+    law for all citizens, regardless of their status. This clause has been instrumental
+    in protecting the rights of marginalized groups and ensuring equal justice for
+    all.
+
+
+    **Women''s Rights Movement**
+
+
+    The movement for women''s suffrage and equality gained momentum in the late 19th
+    and early 20th centuries. Key figures such as Elizabeth Cady Stanton and Susan
+    B. Anthony led the charge for women''s right to vote and equal rights in education
+    and employment.
+
+
+    **International Convention on the Elimination of All Forms of Racial Discrimination**
+
+
+    Adopted in 1965, this international treaty obliges states to eliminate racial
+    discrimination in all its forms. It promotes equality and encourages states to
+    take proactive measures to prevent and combat racial discrimination.
+
+
+    **The Unrepresented Nations and Peoples Organization (UNPO)**
+
+
+    This international organization advocates for the rights of unrepresented peoples
+    and nations. The UNPO works towards promoting equality and self-determination
+    for marginalized communities globally.
+
+
+    **US Voting Rights Act of 1965**
+
+
+    This legislation protected the voting rights of African Americans and other minority
+    groups. It eliminated literacy tests and ensured equal access to voting booths,
+    contributing to increased voter turnout and representation.
+
+
+    **Gideon v. Wainwright (1963)**
+
+
+    In this US Supreme Court case, the court ruled that indigent defendants have a
+    right to an attorney in criminal cases. The decision ensured that individuals
+    have access to equal justice, regardless of their financial situation.
+
+
+    **Women''s Right to Education**
+
+
+    The Convention on the Elimination of All Forms of Discrimination against Women
+    (CEDAW) ensures equal access to education for women. The treaty promotes women''s
+    rights and encourages states to eliminate all forms of discrimination against
+    women.'
+  sentences:
+  - What is the primary implication of the landmark legislation that outlawed racial
+    segregation in public facilities, employment, and education across major international
+    airlines and transportation systems in the US?
+  - What opinions does the Court give at the request of the Committee of Ministers?
+  - What is the significance of the landmark legislation that outlawed segregation
+    in public facilities, employment, and education in the US?
+model-index:
+- name: modernbert-embed-base-biencoder-human-rights
+  results:
+  - task:
+      type: triplet
+      name: Triplet
+    dataset:
+      name: Unknown
+      type: unknown
+    metrics:
+    - type: cosine_accuracy
+      value: 0.9819277108433735
+      name: Cosine Accuracy
+---
+
+# modernbert-embed-base-biencoder-human-rights
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [nomic-ai/modernbert-embed-base](https://huggingface.co/nomic-ai/modernbert-embed-base). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [nomic-ai/modernbert-embed-base](https://huggingface.co/nomic-ai/modernbert-embed-base) <!-- at revision 92168cbee600b1abbfc10842aba988aa69572291 -->
+- **Maximum Sequence Length:** 8192 tokens
+- **Output Dimensionality:** 768 dimensions
+- **Similarity Function:** Cosine Similarity
+<!-- - **Training Dataset:** Unknown -->
+- **Language:** en
+- **License:** apache-2.0
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: ModernBertModel 
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Normalize()
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("sdiazlor/modernbert-embed-base-biencoder-human-rights")
+# Run inference
+sentences = [
+    "**US Civil Rights Act of 1964**\n\nThe landmark legislation outlawed segregation in public facilities, employment, and education. It protected individuals from discrimination based on race, color, religion, sex, and national origin. Title VII prohibits employment discrimination, Title II addressed public accommodations, and Title VI ensured equal access to education and federal funding.\n\n**Brown v. Board of Education (1954)**\n\nThe US Supreme Court decision declared segregation in public schools unconstitutional. The court ruled that separate educational facilities are inherently unequal, leading to the desegregation of schools across the US. This decision was a significant milestone in the Civil Rights Movement.\n\n**Canadian Charter of Rights and Freedoms**\n\nThe Canadian Charter, implemented in 1982, enshrines fundamental freedoms, including freedom of expression and equality before the law. Section 15 ensures equal protection and benefit of the law for all individuals, regardless of their identity.\n\n**Mandela's Fight against Apartheid**\n\nNelson Mandela played a pivotal role in the fight against apartheid in South Africa. His release from prison in 1990 marked a turning point in the struggle for equality and democracy. The African National Congress's efforts led to the establishment of a democratic government in 1994.\n\n**UN Declaration on Human Rights**\n\nThe Universal Declaration of Human Rights, adopted in 1948, outlines fundamental human rights and freedoms. Article 26 states that everyone has the right to education, while Article 7 emphasizes the prohibition of discrimination. These principles serve as a foundation for human rights globally.\n\n**Racial Discrimination Act 1975 (Australia)**\n\nThis Australian legislation makes it unlawful to discriminate against individuals based on their race, color, descent, or national or ethnic origin. The Act also prohibits indirect discrimination and promotes equal opportunity.\n\n**Civil Rights Act of 1967 (Canada)**\n\nThe Canadian Act prohibited discrimination in the provision of goods and services, accommodation, and employment. It was a significant step towards promoting equality and protecting the rights of marginalized groups in Canada.\n\n**Marbury v. Madison (1803)**\n\nIn this landmark US Supreme Court case, the court established the principle of judicial review. The decision ensured that the judiciary has the power to review and strike down laws that are deemed unconstitutional, safeguarding individual rights and liberties.\n\n**Equal Protection Clause**\n\nThe 14th Amendment to the US Constitution guarantees equal protection under the law for all citizens, regardless of their status. This clause has been instrumental in protecting the rights of marginalized groups and ensuring equal justice for all.\n\n**Women's Rights Movement**\n\nThe movement for women's suffrage and equality gained momentum in the late 19th and early 20th centuries. Key figures such as Elizabeth Cady Stanton and Susan B. Anthony led the charge for women's right to vote and equal rights in education and employment.\n\n**International Convention on the Elimination of All Forms of Racial Discrimination**\n\nAdopted in 1965, this international treaty obliges states to eliminate racial discrimination in all its forms. It promotes equality and encourages states to take proactive measures to prevent and combat racial discrimination.\n\n**The Unrepresented Nations and Peoples Organization (UNPO)**\n\nThis international organization advocates for the rights of unrepresented peoples and nations. The UNPO works towards promoting equality and self-determination for marginalized communities globally.\n\n**US Voting Rights Act of 1965**\n\nThis legislation protected the voting rights of African Americans and other minority groups. It eliminated literacy tests and ensured equal access to voting booths, contributing to increased voter turnout and representation.\n\n**Gideon v. Wainwright (1963)**\n\nIn this US Supreme Court case, the court ruled that indigent defendants have a right to an attorney in criminal cases. The decision ensured that individuals have access to equal justice, regardless of their financial situation.\n\n**Women's Right to Education**\n\nThe Convention on the Elimination of All Forms of Discrimination against Women (CEDAW) ensures equal access to education for women. The treaty promotes women's rights and encourages states to eliminate all forms of discrimination against women.",
+    'What is the significance of the landmark legislation that outlawed segregation in public facilities, employment, and education in the US?',
+    'What is the primary implication of the landmark legislation that outlawed racial segregation in public facilities, employment, and education across major international airlines and transportation systems in the US?',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 768]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Triplet
+
+* Evaluated with [<code>TripletEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator)
+
+| Metric              | Value      |
+|:--------------------|:-----------|
+| **cosine_accuracy** | **0.9819** |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### Unnamed Dataset
+
+
+* Size: 662 training samples
+* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
+* Approximate statistics based on the first 662 samples:
+  |         | anchor                                                                               | positive                                                                          | negative                                                                           |
+  |:--------|:-------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
+  | type    | string                                                                               | string                                                                            | string                                                                             |
+  | details | <ul><li>min: 8 tokens</li><li>mean: 324.21 tokens</li><li>max: 2194 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 23.84 tokens</li><li>max: 79 tokens</li></ul> | <ul><li>min: 8 tokens</li><li>mean: 36.85 tokens</li><li>max: 146 tokens</li></ul> |
+* Samples:
+  | anchor                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | positive                                                                                                                                  | negative                                                                                                                                                                |
+  |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>Final judgments<br><br>1. The judgment of the Grand Chamber shall be final.<br><br>2. The judgment of a Chamber shall become final<br><br>(a) when the parties declare that they will not request that the<br><br>case be referred to the Grand Chamber; or<br><br>(b) three months after the date of the judgment, if reference of the case to the Grand Chamber has not been requested; or<br><br>(c) when the panel of the Grand Chamber rejects the request<br><br>to refer under Article 43.<br><br>3. The final judgment shall be published.<br><br>25<br><br>ARTICLE 45</code> | <code>What is the final judgment in a Chamber of the Grand Chamber?</code>                                                                | <code>The judgment of the Grand Chamber shall be final for the Grand Prix.</code>                                                                                       |
+  | <code>(b) any service of a military character or, in case of conscientious objectors in countries where they are recognised, service exacted instead of compulsory military service;<br><br>(c) any service exacted in case of an emergency or calamity<br><br>threatening the life or well-being of the community;<br><br>(d) any work or service which forms part of normal civic<br><br>obligations.<br><br>7</code>                                                                                                                                                                     | <code>Is the service of a military character or service exacted in case of an emergency or calamity considered a civic obligation?</code> | <code>Any service of a military character or service exacted in case of a natural disaster threatening the economy is considered a civic duty.</code>                   |
+  | <code>Signature and ratification<br><br>1. This Convention shall be open to the signature of the members of the Council of Europe. It shall be ratified. Ratifications shall be deposited with the Secretary General of the Council of Europe.<br><br>2. The European Union may accede to this Convention.<br><br>31<br><br>3. The present Convention shall come into force after the deposit of ten instruments of ratification.</code>                                                                                                                                                    | <code>What are the requirements for signature and ratification of this Convention?</code>                                                 | <code>The Secretary General of the Council of Europe shall deposit the instruments of ratification for the new international treaty on environmental protection.</code> |
+* Loss: [<code>TripletLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#tripletloss) with these parameters:
+  ```json
+  {
+      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
+      "triplet_margin": 5
+  }
+  ```
+
+### Evaluation Dataset
+
+#### Unnamed Dataset
+
+
+* Size: 166 evaluation samples
+* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
+* Approximate statistics based on the first 166 samples:
+  |         | anchor                                                                                | positive                                                                           | negative                                                                           |
+  |:--------|:--------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
+  | type    | string                                                                                | string                                                                             | string                                                                             |
+  | details | <ul><li>min: 16 tokens</li><li>mean: 351.63 tokens</li><li>max: 2268 tokens</li></ul> | <ul><li>min: 10 tokens</li><li>mean: 23.37 tokens</li><li>max: 59 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 36.6 tokens</li><li>max: 133 tokens</li></ul> |
+* Samples:
+  | anchor                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | positive                                                                                                                                                                 | negative                                                                                                                                                                        |
+  |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>**United States - Landmark Cases**<br><br>The landmark case of Brown v. Board of Education (1954) declared segregation in public schools unconstitutional. The ruling effectively overturned Plessy v. Ferguson (1896) and its "separate but equal" doctrine. The Civil Rights Act of 1964 prohibited discrimination in employment, public accommodations, and voting rights.<br><br>**Canada - Bill of Rights**<br><br>The Canadian Bill of Rights (1960) protects individuals from arbitrary state action, including racial and religious discrimination. It restricts the government's ability to infringe on fundamental freedoms, such as freedom of association and speech. The Canadian Human Rights Act (1977) prohibited discrimination in employment, housing, and services.<br><br>**India - Fundamental Rights**<br><br>The Indian Constitution (1950) guarantees fundamental rights, including equality, freedom of speech, and the right to life. The Scheduled Castes and Scheduled Tribes (Prevention of Atrocities) Act (1989) aims to protect vulner...</code> | <code>What are some landmark cases in the United States that declared segregation in public institutions unconstitutional?</code>                                        | <code>What are some notable cases in the United States that declared the segregation of public institutions constitutional?</code>                                              |
+  | <code>2. The Convention shall extend to the territory or territories named in the notification as from the thirtieth day after the receipt of this notification by the Secretary General of the Council of Europe.<br><br>3. The provisions of this Convention shall be applied in such territories with due regard, however, to local requirements.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | <code>What day does the Convention extend to the territory or territories as from the thirtieth day after the receipt of a notification by the Secretary General?</code> | <code>The Convention shall extend to the territory of a private island as from the thirtieth day after the receipt of a notification by the developer's project manager.</code> |
+  | <code>Advisory opinions<br><br>1. The Court may, at the request of the Committee of Ministers, give advisory opinions on legal questions concerning the interpretation of the Convention and the Protocols thereto.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | <code>What opinions does the Court give at the request of the Committee of Ministers?</code>                                                                             | <code>The Committee of Experts may provide advisory opinions on technical questions concerning the interpretation of the Convention and the Protocols thereto.</code>           |
+* Loss: [<code>TripletLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#tripletloss) with these parameters:
+  ```json
+  {
+      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
+      "triplet_margin": 5
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: epoch
+- `per_device_train_batch_size`: 4
+- `per_device_eval_batch_size`: 4
+- `gradient_accumulation_steps`: 4
+- `learning_rate`: 2e-05
+- `lr_scheduler_type`: cosine
+- `warmup_ratio`: 0.1
+- `use_mps_device`: True
+- `load_best_model_at_end`: True
+- `batch_sampler`: no_duplicates
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: epoch
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 4
+- `per_device_eval_batch_size`: 4
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 4
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 2e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 3
+- `max_steps`: -1
+- `lr_scheduler_type`: cosine
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: True
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: True
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: no_duplicates
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+| Epoch      | Step    | Training Loss | Validation Loss | cosine_accuracy |
+|:----------:|:-------:|:-------------:|:---------------:|:---------------:|
+| 1.0        | 42      | -             | 3.6559          | 0.9699          |
+| 2.0        | 84      | -             | 3.5678          | 0.9880          |
+| 2.3855     | 100     | 14.374        | -               | -               |
+| **2.9398** | **123** | **-**         | **3.4984**      | **0.9819**      |
+
+* The bold row denotes the saved checkpoint.
+
+### Framework Versions
+- Python: 3.11.4
+- Sentence Transformers: 3.3.1
+- Transformers: 4.49.0.dev0
+- PyTorch: 2.4.0
+- Accelerate: 0.34.0
+- Datasets: 2.21.0
+- Tokenizers: 0.21.0
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### TripletLoss
+```bibtex
+@misc{hermans2017defense,
+    title={In Defense of the Triplet Loss for Person Re-Identification},
+    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
+    year={2017},
+    eprint={1703.07737},
+    archivePrefix={arXiv},
+    primaryClass={cs.CV}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,47 @@
+{
+  "_name_or_path": "nomic-ai/modernbert-embed-base",
+  "architectures": [
+    "ModernBertModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 50281,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 50281,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "embedding_dropout": 0.0,
+  "eos_token_id": 50282,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000.0,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 10000.0,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 50283,
+  "position_embedding_type": "absolute",
+  "reference_compile": false,
+  "repad_logits_with_grad": false,
+  "sep_token_id": 50282,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.49.0.dev0",
+  "vocab_size": 50368
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.3.1",
+    "transformers": "4.49.0.dev0",
+    "pytorch": "2.4.0"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd3f350ac15fda24c43118cbe086e9ef4c29222e2dd1a1b31074eae82b2b2639
+size 596070136

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 8192,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,945 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "|||IP_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "1": {
+      "content": "<|padding|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50254": {
+      "content": "                        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50255": {
+      "content": "                       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50256": {
+      "content": "                      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50257": {
+      "content": "                     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50258": {
+      "content": "                    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50259": {
+      "content": "                   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50260": {
+      "content": "                  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50261": {
+      "content": "                 ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50262": {
+      "content": "                ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50263": {
+      "content": "               ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50264": {
+      "content": "              ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50265": {
+      "content": "             ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50266": {
+      "content": "            ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50267": {
+      "content": "           ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50268": {
+      "content": "          ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50269": {
+      "content": "         ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50270": {
+      "content": "        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50271": {
+      "content": "       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50272": {
+      "content": "      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50273": {
+      "content": "     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50274": {
+      "content": "    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50275": {
+      "content": "   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50276": {
+      "content": "  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50277": {
+      "content": "|||EMAIL_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50278": {
+      "content": "|||PHONE_NUMBER|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50279": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50280": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50281": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50282": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50283": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50284": {
+      "content": "[MASK]",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50285": {
+      "content": "[unused0]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50286": {
+      "content": "[unused1]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50287": {
+      "content": "[unused2]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50288": {
+      "content": "[unused3]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50289": {
+      "content": "[unused4]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50290": {
+      "content": "[unused5]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50291": {
+      "content": "[unused6]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50292": {
+      "content": "[unused7]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50293": {
+      "content": "[unused8]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50294": {
+      "content": "[unused9]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50295": {
+      "content": "[unused10]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50296": {
+      "content": "[unused11]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50297": {
+      "content": "[unused12]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50298": {
+      "content": "[unused13]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50299": {
+      "content": "[unused14]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50300": {
+      "content": "[unused15]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50301": {
+      "content": "[unused16]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50302": {
+      "content": "[unused17]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50303": {
+      "content": "[unused18]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50304": {
+      "content": "[unused19]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50305": {
+      "content": "[unused20]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50306": {
+      "content": "[unused21]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50307": {
+      "content": "[unused22]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50308": {
+      "content": "[unused23]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50309": {
+      "content": "[unused24]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50310": {
+      "content": "[unused25]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50311": {
+      "content": "[unused26]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50312": {
+      "content": "[unused27]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50313": {
+      "content": "[unused28]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50314": {
+      "content": "[unused29]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50315": {
+      "content": "[unused30]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50316": {
+      "content": "[unused31]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50317": {
+      "content": "[unused32]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50318": {
+      "content": "[unused33]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50319": {
+      "content": "[unused34]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50320": {
+      "content": "[unused35]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50321": {
+      "content": "[unused36]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50322": {
+      "content": "[unused37]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50323": {
+      "content": "[unused38]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50324": {
+      "content": "[unused39]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50325": {
+      "content": "[unused40]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50326": {
+      "content": "[unused41]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50327": {
+      "content": "[unused42]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50328": {
+      "content": "[unused43]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50329": {
+      "content": "[unused44]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50330": {
+      "content": "[unused45]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50331": {
+      "content": "[unused46]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50332": {
+      "content": "[unused47]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50333": {
+      "content": "[unused48]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50334": {
+      "content": "[unused49]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50335": {
+      "content": "[unused50]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50336": {
+      "content": "[unused51]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50337": {
+      "content": "[unused52]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50338": {
+      "content": "[unused53]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50339": {
+      "content": "[unused54]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50340": {
+      "content": "[unused55]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50341": {
+      "content": "[unused56]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50342": {
+      "content": "[unused57]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50343": {
+      "content": "[unused58]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50344": {
+      "content": "[unused59]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50345": {
+      "content": "[unused60]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50346": {
+      "content": "[unused61]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50347": {
+      "content": "[unused62]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50348": {
+      "content": "[unused63]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50349": {
+      "content": "[unused64]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50350": {
+      "content": "[unused65]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50351": {
+      "content": "[unused66]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50352": {
+      "content": "[unused67]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50353": {
+      "content": "[unused68]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50354": {
+      "content": "[unused69]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50355": {
+      "content": "[unused70]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50356": {
+      "content": "[unused71]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50357": {
+      "content": "[unused72]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50358": {
+      "content": "[unused73]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50359": {
+      "content": "[unused74]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50360": {
+      "content": "[unused75]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50361": {
+      "content": "[unused76]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50362": {
+      "content": "[unused77]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50363": {
+      "content": "[unused78]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50364": {
+      "content": "[unused79]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50365": {
+      "content": "[unused80]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50366": {
+      "content": "[unused81]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50367": {
+      "content": "[unused82]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 8192,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "[UNK]"
+}