Upload ITERForRelationExtraction

README.md

@@ -0,0 +1,104 @@
+---
+license: apache-2.0
+base_model:
+- KISTI-AI/Scideberta-full
+library_name: transformers
+tags:
+- relation extraction
+- nlp
+model-index:
+  - name: iter-scierc-deberta-full
+    results:
+      - task:
+          type: relation-extraction
+        dataset:
+          name: scierc
+          type: scierc
+        metrics:
+          - name: F1
+            type: f1
+            value: 39.359
+---
+
+
+# ITER: Iterative Transformer-based Entity Recognition and Relation Extraction
+
+This model checkpoint is part of the collection of models published alongside our paper ITER, 
+[accepted at EMNLP 2024](https://aclanthology.org/2024.findings-emnlp.655/).<br>
+To ease reproducibility and enable open research, our source code has been published on [GitHub](https://github.com/fleonce/iter).
+
+This model achieved an F1 score of `39.359` on dataset `scierc`
+
+### Using ITER in your code
+
+First, install ITER in your preferred environment:
+
+```text
+pip install git+https://github.com/fleonce/iter
+```
+
+To use our model, refer to the following code:
+```python
+from iter import ITERForRelationExtraction
+
+model = ITERForRelationExtraction.from_pretrained("fleonce/iter-scierc-deberta-full")
+tokenizer = model.tokenizer
+
+encodings = tokenizer(
+  "An art exhibit at the Hakawati Theatre in Arab east Jerusalem was a series of portraits of Palestinians killed in the rebellion .",
+  return_tensors="pt"
+)
+
+generation_output = model.generate(
+    encodings["input_ids"],
+    attention_mask=encodings["attention_mask"],
+)
+
+# entities
+print(generation_output.entities)
+
+# relations between entities
+print(generation_output.links)
+```
+
+### Checkpoints
+
+We publish checkpoints for the models performing best on the following datasets:
+
+- **ACE05**:
+  1. [fleonce/iter-ace05-deberta-large](https://huggingface.co/fleonce/iter-ace05-deberta-large)
+- **CoNLL04**:
+  1. [fleonce/iter-conll04-deberta-large](https://huggingface.co/fleonce/iter-conll04-deberta-large)
+- **ADE**:
+  1. [fleonce/iter-ade-deberta-large](https://huggingface.co/fleonce/iter-ade-deberta-large)
+- **SciERC**:
+  1. [fleonce/iter-scierc-deberta-large](https://huggingface.co/fleonce/iter-scierc-deberta-large)
+  2. [fleonce/iter-scierc-scideberta-full](https://huggingface.co/fleonce/iter-scierc-scideberta-full)
+- **CoNLL03**:
+  1. [fleonce/iter-conll03-deberta-large](https://huggingface.co/fleonce/iter-conll03-deberta-large)
+- **GENIA**:
+  1. [fleonce/iter-genia-deberta-large](https://huggingface.co/fleonce/iter-genia-deberta-large)
+
+
+### Reproducibility
+
+For each dataset, we selected the best performing checkpoint out of the 5 training runs we performed during training.
+This model was trained with the following hyperparameters:
+
+- Seed: `3`
+- Config: `scierc/d_ff_150`
+- PyTorch `2.3.0` with CUDA `12.1` and precision `torch.bfloat16`
+- GPU: `1 NVIDIA GeForce RTX 4090`
+
+Varying GPU and CUDA version as well as training precision did result in slightly different end results in our tests
+for reproducibility.
+
+To train this model, refer to the following command:
+```shell
+python3 train.py --dataset scierc/d_ff_150 --transformer KISTI-AI/Scideberta-full --use_bfloat16 --seed 3
+```
+
+```text
+@inproceedings{citation}
+```
+

config.json

@@ -0,0 +1,78 @@
+{
+  "_name_or_path": "models/fleonce/iter-scierc-scideberta-full",
+  "activation_fn": "relu",
+  "architectures": [
+    "ITERForRelationExtraction"
+  ],
+  "d_ff": 150,
+  "d_model": 768,
+  "dataset": "scierc",
+  "dropout": 0.3,
+  "entity_types": [
+    "Task",
+    "Method",
+    "Material",
+    "OtherScientificTerm",
+    "Metric",
+    "Generic"
+  ],
+  "features": 1385248,
+  "link_types": [
+    "Used-for",
+    "Feature-of",
+    "Hyponym-of",
+    "Evaluate-for",
+    "Part-of",
+    "Compare",
+    "Conjunction"
+  ],
+  "max_length": 512,
+  "max_nest_depth": 3,
+  "model_type": "iter",
+  "num_links": 7,
+  "num_types": 7,
+  "threshold": 0.5,
+  "torch_dtype": "float32",
+  "transformer_config": {
+    "_name_or_path": "KISTI-AI/Scideberta-full",
+    "architectures": null,
+    "attention_head_size": 64,
+    "attention_probs_dropout_prob": 0.1,
+    "decoder_start_token_id": null,
+    "eos_token_id": null,
+    "hidden_act": "gelu",
+    "hidden_dropout_prob": 0.1,
+    "hidden_size": 768,
+    "initializer_range": 0.02,
+    "intermediate_size": 3072,
+    "is_encoder_decoder": false,
+    "layer_norm_eps": 1e-07,
+    "max_length": 512,
+    "max_position_embeddings": 512,
+    "max_relative_positions": -1,
+    "model_type": "deberta-v2",
+    "norm_rel_ebd": "layer_norm",
+    "num_attention_heads": 12,
+    "num_hidden_layers": 12,
+    "padding_idx": 0,
+    "pooler_dropout": 0,
+    "pooler_hidden_act": "gelu",
+    "pooler_hidden_size": 768,
+    "pos_att_type": [
+      "p2c",
+      "c2p"
+    ],
+    "position_biased_input": false,
+    "position_buckets": 256,
+    "relative_attention": true,
+    "share_att_key": true,
+    "task_specific_params": null,
+    "type_vocab_size": 0,
+    "vocab_size": 128100
+  },
+  "transformers_version": "4.37.0",
+  "use_bias": false,
+  "use_gate": true,
+  "use_mlp": true,
+  "use_scale": false
+}

generation_config.json

@@ -0,0 +1,5 @@
+{
+  "_from_model_config": true,
+  "max_length": 512,
+  "transformers_version": "4.37.0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c64349059cb28c2b8ce7db99744f76cb3f2af7a28a10432e1796c1ad97aab33f
+size 739972032