Thanks for taking the time to create this, and for sharing it
It's too bad that it costs me so much just to run it! Lulz.
Truly, though, I've tried well over a dozen LLMs thusfar, and this one is the best. Its diction seems better than GPT-3.5's at times. Of course, I should be thanking the people who created LLaMa, I suppose, but they are ensconced somewhere deep in the bowels of Meta, Inc.
I'm using it with KoboldAI, and am wondering what the best approach is to get it to write stories at length. I've never used an instruct model before, and I'm not sure of the best way to prompt it in KoboldAI to facilitate that. It always seems to want to wrap the story up. I'm also not entirely certain what the best pracftice is for having KoboldAI insert those #### instruction blocks automatically (out of band).
Thanks again.
Absolutely, no problem! I had a fun time picking what to merge and testing the results. Always glad to hear others enjoy it too!
For KoboldAI, I will typically use the Storywriter preset, set context tokens to 2048, and tokens it can generate to 680 or higher (you can click the number and type in your own, bypassing the slider limitations).
To break up the model's tendency to try to end things early with a nice little ending, and to use the Alpaca instruct with KoboldAI, I find it best to open the memory box and put the instructions in there. What I do, personally, is enable classic chat mode. For instance, I will name myself my Discord name (Chasm) and the bot Giselle (just a cool name that I like) then I go into memory and write something like:
Instruction:
Make Giselle an expert fiction writer and story teller who is excited to write long, visual story telling about X (where X is the topic you really want them to focus on).
Make Chasm (my chat name) her inspiration, and have Giselle eager to fulfill Chasm's every request with verbose, long stories about what they describe.
Response:
(oh btw that's supposed to be three hash tags before Instruction then a space as well as three hash tags and a space with response and a carriage return after the : part of response, Huggingface interprets my hashtags as markup to bolden text lol)
Something along those lines, I change it up often; experimenting with the presets and the exact instruction works wonders and is a bit fun too.
You can even tell the AI the role of the bot to be anything you would like, for instance "Make Giselle function as an adventure based text game and Chasm as the player input"
The possibilities go on. Thank you for the kind words!
Oh, so they are supposed to be on separate lines? Thank you for clearing that up, haha. So it does matter if there is a carriage return after each of them?
Oh, is there a way to use worldinfo with this, or is that only for models that understand whatever format koboldai uses to present it?