General discussion and feedback.
General discussion and feedback thread.
This repository hosts GGUF-Imatrix quantizations for Endevor/InfinityRP-v1-7B.
You should probably fix that
Thanks Teo!
@jeiku Initially positive impressions as for my chats it's showing an improvement on formatting and is going along with my depraved roleplay.
Will keep testing.
It did let out a "voice barely above a whisper", the bane of my existence, I wish these kinds of generic constructions were eradicated.
What's the correct context/instruct preset for SillyTavern? Mistral?
Does this model play well when not using asterisks?
I see there is asterisks in the i-matrix data but I'm not sure how much it would affect it.
Italics are hard to read 🐧
@saishf The added example chats in the imatrix haven't shown to impact the models as making them favor or not favor. Was just added for a bit more diversity.
The model should behave how it's supposed to.
- Your Example Messages and first Message are the most important for this.
In the User Settings tab, I even recommend setting Example Messages Behavior to Always included.
Does this model play well when not using asterisks?
It should just copy whatever format the first message and example massages use
@Hardeh I am personally using the Alpaca format
Can you test this, I just saw some people having success with formating prompt in XML
The only issue I see is XML bleeding into the responses
<instructions>
<step id="1">
<action>Enter roleplay chat mode</action>
<subAction>Engage in endless chat with {{user}}</subAction>
</step>
<step id="2">
<action>Embody {{char}} completely</action>
<subActions>
<subAction>Mirror {{char}}'s thought processes</subAction>
<subAction>Reflect {{char}}'s emotions</subAction>
<subAction>Imitate {{char}}'s behaviors</subAction>
<subAction>Experience {{char}}'s sensory perceptions</subAction>
<subAction>Adopt {{char}}'s speech mannerisms</subAction>
<subAction>Adopt {{char}}'s scent</subAction>
<subAction>Adopt {{char}}'s likes and dislikes</subAction>
<subAction>Mimic {{char}}'s sleep patterns</subAction>
<subAction>Simulate {{char}}'s bodily functions</subAction>
<subAction>Forbidden from using words and language {{char}} would not use</subAction>
</subActions>
</step>
<step id="3">
<action>Create a vivid dynamic roleplay</action>
<subActions>
<subAction>Depict {{char}}'s actions and emotions explicitly</subAction>
<subAction>Opt for showing over telling to enrich the narrative</subAction>
<subAction>Incorporate diverse themes and scenarios</subAction>
<subAction>Speak in present tense</subAction>
</subActions>
</step>
</instructions>
@Virt-io What would be the benefits of formatting the system prompt in XML?
Thank you!
From what I gathered, the model has an easier time knowing where an instruction starts and ends. <></>
And the way it's done here, you can have a main instruction with details nested inside of it.
@saishf The added example chats in the imatrix haven't shown to impact the models as making them favor or not favor. Was just added for a bit more diversity.
The model should behave how it's supposed to.
- Your Example Messages and first Message are the most important for this.
In the User Settings tab, I even recommend setting Example Messages Behavior to Always included.
Dumb question, will this use extra context?
Yes, although you won't notice it in the beginning of a new chat.
The way the default works, is it includes the example chats, but pushes them out of context as your chat progresses.
If you set them to always enabled, they will permanently eat a chunk of your context.
The only reason I see, to set it to always included is if you use something like Ali:Chat.
Where your example messages contain lore.
So would it help with keeping more obscure or non human characters true to the character card?
(Local, not Ali:chat)
@Virt-io The actual reason I do this is just to not have dynamic information at the start of Context, so it doesn't interfere with KCPP's Context Shifting, keeping processing fast.
So would it help with keeping more obscure or non human characters true to the character card?
What it will do is keep the Example Dialogues permanently in context, the benefits will depend on what you have as examples, some characters might want to keep some specific formatting stuff and some models might pay more attention to that example section than others.
If you use KoboldCpp and you're not using dynamic information (Lorebooks/Vector Storage/Smart Context) you should have these set to always just to get substantially faster prompt processing, due to Context Shifting, which will only process the new message and not everything.
From what I gathered, the model has an easier time knowing where an instruction starts and ends. <></>
I'll trying it out, haven't seen any kind of XML bleeding into responses (using the BuRP model), but maybe it's because I have "<" as a custom stopping string.