File size: 1,667 Bytes
d5dfa3a 36b87af d5dfa3a fc8cc94 d5dfa3a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
---
license: mit
---
# [NeurIPS 2024] Quantifying and Optimizing Global Faithfulness in Persona-driven Role-playing
This is a discriminator for persona-driven role-playing, distilled from GPT-4. It evaluates the NLI relation between the character's response and a persona statement of the character.
The discriminator is a component of a global role-playing faithfulness optimization framework (as shown below), to use it please visit [github.com/KomeijiForce/Active_Passive_Constraint_Koishiday_2024](https://github.com/KomeijiForce/Active_Passive_Constraint_Koishiday_2024)
```python
from classifier import Classifier, get_relevance_discriminator, get_nli_discriminator
from score import score_apc, score_APC
relevance_discriminator = get_relevance_discriminator(character=None, statement_query_relevance_dataset=None, relevance_finetune_epoch=None, use_pretrained_discriminator=True)
nli_discriminator = get_nli_discriminator(character=None, statement_to_response_nli_v2_dataset=None, nli_finetune_epoch=None, use_pretrained_discriminator=True)
character = "Komeiji Koishi"
statements = ["Komeiji Koishi lives with her sister, Komeiji Satori.", "Komeiji Koishi lives in Chireiden."]
query = "Where do you live, Koishi?"
responses = ["I live in Chireiden with my sister, Satori!", "I live in Chireiden!", "I live in Hakurei Shrine!"]
print([score_APC(character, statements, query, response, relevance_discriminator, nli_discriminator).item() for response in responses])
# [1.6079180240631104, 0.9955980777740479, 0.03315635025501251]
```
|