README.md

---
license: llama2
---

This model is the secret weapon behind the [Aurelian](https://huggingface.co/grimulkan/aurelian-alpha0.1-70b-rope8-32K-fp16) series of models. It takes a chunk of story as the input, and generates the prompt that could have produced it. Basically, it converts a story into Q&A format, for the purpose of training another model for instruct-tuned story writing.

I made it for internal use, and it is not very user-friendly. But it's what I've got until I get the compute to re-train it.

It can hallucinate details with a small probability, but is still usable in a majority of cases (as is evidenced by Aurelian). I am working on a new version with easier prompting format and fewer hallucinations.

A 6-bit EXL2 quantization is available [here](https://huggingface.co/grimulkan/story-reverse-prompt-70b-rope8-32K-6bpw_h6_exl2).


The steps to use this model are:
 - Get a plaintext version of a story. ePUBs are the easiest to convert.
   - Hopefully human-written, that would be the main point of using a model like this.
   - The model sometimes struggles with first-person stories or stories that keep changing the narrator, and gets confused about who is currently talking. Third-person stories work best.
 - Divide the story into chunks.
   - Typically less than 3000 tokens per chunk.
   - Try to break on chapter boundaries.
   - Try to strip any non-story text, like the chapter number, page number, etc.
 - Setup the initial prompt of the model (see below), and pass it the first chunk.
   - Model produces a prompt that could generate the story chunk (see example).
 - Concatenate the previous output cumulatively (context history, see examples below), and pass the combined input along with the next story chunk, similar to a normal chat conversation.
   - Model produces the writing prompt corresponding to the 2nd chunk.
   - The reason the model accepts a conversation history and has 32K of context is to keep the writing prompts sounding natural and to pay attention to prior story context.
     - For instance, let's say the character meets an elf in the new chunk, but the elf was already introduced some chunks before in the story. When writing the prompt, the model would correctly refer to 'the elf', rather than 'an elf', or by name, since it knows the prior story context.
     - This is the main advantage of using this model vs trying to generate a standalone summary per story chunk. Standalone summaries end up sounding very inconsistent over a long story.
   - Depending on how much VRAM you have, you may need to limit your context length (or truncate at 32K anyway, leaving room for the output), just like any other chat conversation. Most clients will do this automatically (egs., oobabooga). You will lose prior story context, but 32K is pretty big if you can use all of it.
- Continue this process, until your entire story text is converted into a series of writing prompts, and corresponding story chunks as output.
  - Then you can convert the Q&A to egs., fastchat format in a .json for SFT.
  - If you want to build an anti-chatGPT DPO set for creative output, you can pass the prompts (that are produced by this model) as an input to ChatGPT3.5, and use the resulting chatGPT output as the _rejected_ response, and the human generated chunk as the _accepted_ response.


## Input format:

It follows a weird format which I always wanted to fix, but never got around to doing it. So you get to use it as-is (for now). Sorry!

The included `story-reverse-prompt.yaml` implements the instruct format for Oobabooga (place in `/instruction-templates` and select it in the UI), but you still need to use the format inside each prompt yourself.

**Always use linear rope scaling = 8, even if you are not using 32K context.**

```
<s>An interaction between a user and an assistant. The user may provide the assistant with the continuation of the story, and then TASK the assistant to suggest a prompt or set of instructions that could have generated the given continuation.</s><NO-LINE-BREAK>
<s>USER: <Any general information about the story, the themes, character background, etc.>


Follow the TASKs below. If you understand, say YES.<NO-LINE-BREAK>
<s>ASSISTANT: YES</s><NO-LINE-BREAK>
<s>USER: Here is the next part of the story:


<1st story chunk>


TASK: Write a detailed prompt that would generate the above section of the story.</s><NO-LINE-BREAK>
<s>ASSISTANT: 
```

 - The model will respond with the output (see an example below).
 - Note that there is a blank space at the end of the above completion request (after `ASSISTANT:`).
 - I used `<NO-LINE-BREAK>` to indicate that there is not supposed to be a line break there, but I added a line break in the above text, just for human readability. Don't actually put a line break or the text `<NO-LINE-BREAK>` there!
 - The first prompt (with the YES response) is basically a hard-coded way to add some background info, outside the system prompt.
   - It is optional, but the model was trained to expect it (even if it only provides trivial information like 'This is a fictional story').
   - The model will not actually respond with YES, it's just what you type in before the first chunk (including the `<s>ASSISTANT: YES`... in your input).
 - Many clients will auto-include the BOS token `<s>` at the top (before the system message `An interaction between...`), so don't double-specify it. The included template will handle all this for Oobabooga and you can focus on what goes inside each prompt instead.


For the 2nd (and subsequent) chunks, you'd continue the conversation like so:
```
<s>An interaction between a user and an assistant. The user may provide the assistant with the continuation of the story, and then TASK the assistant to suggest a prompt or set of instructions that could have generated the given continuation.</s><NO-LINE-BREAK>
<s>USER: <Any general information about the story, the themes, character background, etc.>


Follow the TASKs below. If you understand, say YES.<NO-LINE-BREAK>
<s>ASSISTANT: YES</s><NO-LINE-BREAK>
<s>USER: Here is the next part of the story:


<1st story chunk>


TASK: Write a detailed prompt that would generate the above section of the story.</s><NO-LINE-BREAK>
<s>ASSISTANT: <model's writing prompt for the 1st chunk></s><NO-LINE-BREAK>
<s>USER: Here is the next part of the story:


<2nd story chunk>


TASK: Write a detailed prompt that would generate the above section of the story.</s><NO-LINE-BREAK>
<s>ASSISTANT: 
```
and so on.

If you run out of context length, you'd need to drop the oldest chunks/prompts (most clients will do this automatically). It is optional whether you want to drop or preserve the first general information/background prompt or not.


## Example:

Story is broken into 2 chunks here.

**Chunk 1:**

The year was 3187. The mighty city marked sporadic craters as deep as mountains, remnants from the Great Alien War three centuries ago. Nature stubbornly reclaimed its right as giant roots of ancient trees twisted through the rubble and ivy tendrils, winding around the crumbling remnants of once-opulent mansions and high-tech skyscrapers. The grandeur of the city was succumbed to the aging grace of decay, a ghost town where the very stones bore witness to a forgotten era of human legacy.

Yet, there was life. Humanity, irrevocable in its will to survive and thrive, persisted alongside the peculiar menagerie of alien creatures that now called the city home. The air thrummed with the buzz of multi-species chatter, the staccato thrum of conversations in a thousand languages, human and non-human alike. Amid the dilapidated ruins, hideous insectoid monstrosities from Sirius B scuttled next to human children, their spindly legs clack-clacking on the rubble-strewn streets. Half-visible wraiths shimmered ominously in the corners, with their shining red eyes piercing through the ethereal haze.

In the skies, flying beasts with scales that glittered like iridescent moonlight and wings that blotted out the sun soared in serpentine paths, scattering the remains of their meals over the mechanized ruins below. Fleets of hovercrafts wove precariously in between, buzzing noisily, trailing multicolored plasma contrails, as they ferried their mix of human and alien passengers to and from their unknown errands. Discarded technology from an era of unfathomable advancement lay cast aside as casually as children's playthings; hovering drones lay nestled in the shadows, hummed amidst the tendrils of a fluorescent creeper and resting against a rusted hulk of what might once have been a titan class cargo ship.

Despite the harmony that somehow managed to pull through the city's unsightly paradox of nature and technology, life in the city was a gamble on a good day and a death sentence on a bad one. Carved by the merciless, age-worn hands of crime and violence, it cried crimson tears as the atrocities of its inhabitants painted the alleys and backstreets in hues of fear and dread. Murder was as familiar as rain for the citizens, their prayers whispered under hushed voices laden with terror, their lullabies the ebbing scream of the hunted.

**Chunk2:**

Amidst the cobblestone paths and rubble-filled trenches, the horrid, grotesque abominations woven from countless star systems stalked the shadows. Towering, shadow creatures with eyes like liquid silver pulsed in the darkness. Silent as the void with bodies as black as obsidian, they contrasted drastically with the shimmering, transparent mollusks from Orion's Belt, their acid-green innards glowing through their glass-like exoskeletons, casting chilling lights on the debris-filled streets.

And then there were the Skeltaar, who'd emerged from cosmic wormholes somewhere in the city's industrial district. Eight-foot-tall, bipedal nightmares, their vicious jaws filled with serrated metallic denticles, their clothing - if it could be called so - an amalgamation of flesh and bioluminescent fungi. The city sang lullabies about them, low murmurs of the grotesque diet these beings followed - flesh, raw and fresh, preferably of the screaming kind. The chilling cry they let out, a cross between a machine grinding to a halt and a wailing child, was a haunting regular to the night's chorus, causing even the bravest inhabitants to pull their covers tighter, their subconscious tainted with the horrors of the day seeping into the vulnerabilities of the dreaming mind.

Amid this backdrop of half-imagined dystopia and unlikely vibrancy, crime rippled through the city like invisible waves, washing through the blood-stained alleyways and crowded marketplaces alike. The stench of smoke and iron-rich blood obstinately clung to the thick city air, a dreadful promise of the constant danger that prevailed within. Trading of illicit substances brewed in secret labs, murdering for pleasure and power by creatures from a different realm, swindling, gambling, prostitution of various kinds - all of it melded seamlessly into the city's chaotic heartbeat.


Here is how it will be fed into the model.

**First chunk input:**
```
<s>An interaction between a user and an assistant. The user may provide the assistant with the continuation of the story, and then TASK the assistant to suggest a prompt or set of instructions that could have generated the given continuation.</s><NO-LINE-BREAK>
<s>USER: This is a dark science fiction story with a dystopian backdrop.


Follow the TASKs below. If you understand, say YES.<NO-LINE-BREAK>
<s>ASSISTANT: YES</s><NO-LINE-BREAK>
<s>USER: Here is the next part of the story:


The year was 3187. The mighty city marked sporadic craters as deep as mountains, remnants from the Great Alien War three centuries ago. Nature stubbornly reclaimed its right as giant roots of ancient trees twisted through the rubble and ivy tendrils, winding around the crumbling remnants of once-opulent mansions and high-tech skyscrapers. The grandeur of the city was succumbed to the aging grace of decay, a ghost town where the very stones bore witness to a forgotten era of human legacy.

Yet, there was life. Humanity, irrevocable in its will to survive and thrive, persisted alongside the peculiar menagerie of alien creatures that now called the city home. The air thrummed with the buzz of multi-species chatter, the staccato thrum of conversations in a thousand languages, human and non-human alike. Amid the dilapidated ruins, hideous insectoid monstrosities from Sirius B scuttled next to human children, their spindly legs clack-clacking on the rubble-strewn streets. Half-visible wraiths shimmered ominously in the corners, with their shining red eyes piercing through the ethereal haze.

In the skies, flying beasts with scales that glittered like iridescent moonlight and wings that blotted out the sun soared in serpentine paths, scattering the remains of their meals over the mechanized ruins below. Fleets of hovercrafts wove precariously in between, buzzing noisily, trailing multicolored plasma contrails, as they ferried their mix of human and alien passengers to and from their unknown errands. Discarded technology from an era of unfathomable advancement lay cast aside as casually as children's playthings; hovering drones lay nestled in the shadows, hummed amidst the tendrils of a fluorescent creeper and resting against a rusted hulk of what might once have been a titan class cargo ship.

Despite the harmony that somehow managed to pull through the city's unsightly paradox of nature and technology, life in the city was a gamble on a good day and a death sentence on a bad one. Carved by the merciless, age-worn hands of crime and violence, it cried crimson tears as the atrocities of its inhabitants painted the alleys and backstreets in hues of fear and dread. Murder was as familiar as rain for the citizens, their prayers whispered under hushed voices laden with terror, their lullabies the ebbing scream of the hunted.


TASK: Write a detailed prompt that would generate the above section of the story.</s><NO-LINE-BREAK>
<s>ASSISTANT: 
```

**Model output (for 1st chunk):**

Write a scene in a post-apocalyptic setting, where the remnants of a once prosperous and advanced civilization have now become overgrown and reclaimed by nature. The story is set in the far future year 3187, and it is a dark sci-fi with a grim and ominous tone.

The story should mention that the city was destroyed during the Great Alien War 3 centuries ago, which left deep craters and ruins. Over time, nature has reclaimed the land, twisting ancient trees and ivy vines around the crumbling buildings. Due to the ancient war, the city now hosts an eclectic mix of human and alien fauna. The humans, irrevocable in their will to survive, have learned to co-exist with these creatures, and live alongside them. The alien creatures described include insectoid monstrosities from Sirius B and ethereal wraiths.

Use descriptive language to bring life to the scene and paint a vivid picture of this post-apocalyptic city. Mention the flying beasts from Sirius B soaring through the skies, the children playing beside ancient, discarded technology, the ruined buildings overgrown with fluorescent creepers, and the hovercraft flying dangerously close among them.

The story should also highlight the coexistence of nature and technology in this city, with the air filled with sounds from a mix of human and non-human species, and a blend of flying beasts from Sirius B. Mention the iridescent scales of the flying beasts scattering remains of their meals on the dilapidated ruins below.

Lastly, the story should reflect on the stark reality of life in the city being dangerous, with crime and violence rampant, and murders commonplace. The people live in fear, their prayers whispered under hushed voices, and their lullabies characterized by the screams of the hunted. End the scene with a poignant note on the hopelessness and despair that prevails in the city.</s>


**Second chunk input (with prior context):**
```
<s>An interaction between a user and an assistant. The user may provide the assistant with the continuation of the story, and then TASK the assistant to suggest a prompt or set of instructions that could have generated the given continuation.</s><NO-LINE-BREAK>
<s>USER: <Any general information about the story, the themes, character background, etc.>


Follow the TASKs below. If you understand, say YES.<NO-LINE-BREAK>
<s>ASSISTANT: YES</s><NO-LINE-BREAK>
<s>USER: Here is the next part of the story:


<1st story chunk from above: The year was 3187...of the hunted>


TASK: Write a detailed prompt that would generate the above section of the story.</s><NO-LINE-BREAK>
<s>ASSISTANT: <model output for 1st chunk from above: Write a scene...in the city></s><NO-LINE-BREAK>
<s>USER: Here is the next part of the story:


Amidst the cobblestone paths and rubble-filled trenches, the horrid, grotesque abominations woven from countless star systems stalked the shadows. Towering, shadow creatures with eyes like liquid silver pulsed in the darkness. Silent as the void with bodies as black as obsidian, they contrasted drastically with the shimmering, transparent mollusks from Orion's Belt, their acid-green innards glowing through their glass-like exoskeletons, casting chilling lights on the debris-filled streets.

And then there were the Skeltaar, who'd emerged from cosmic wormholes somewhere in the city's industrial district. Eight-foot-tall, bipedal nightmares, their vicious jaws filled with serrated metallic denticles, their clothing - if it could be called so - an amalgamation of flesh and bioluminescent fungi. The city sang lullabies about them, low murmurs of the grotesque diet these beings followed - flesh, raw and fresh, preferably of the screaming kind. The chilling cry they let out, a cross between a machine grinding to a halt and a wailing child, was a haunting regular to the night's chorus, causing even the bravest inhabitants to pull their covers tighter, their subconscious tainted with the horrors of the day seeping into the vulnerabilities of the dreaming mind.

Amid this backdrop of half-imagined dystopia and unlikely vibrancy, crime rippled through the city like invisible waves, washing through the blood-stained alleyways and crowded marketplaces alike. The stench of smoke and iron-rich blood obstinately clung to the thick city air, a dreadful promise of the constant danger that prevailed within. Trading of illicit substances brewed in secret labs, murdering for pleasure and power by creatures from a different realm, swindling, gambling, prostitution of various kinds - all of it melded seamlessly into the city's chaotic heartbeat.



TASK: Write a detailed prompt that would generate the above section of the story.</s><NO-LINE-BREAK>
<s>ASSISTANT: 
```

**Model output (for 2nd chunk):**

Describe some of the horrid and grotesque creatures that stalk the shadows of the city. Talk about their physical appearance and the fear they inspire in the inhabitants of the city. Describe their habitats and behavior.

Also mention the crime that is rampant in the city. Describe some of the types of illicit activities that are common in the city.</s>


...and so on. As you can see, you can get varied responses, sometimes specific and detailed (like the 1st chunk output), or short (leaving the story details to be more open-ended). This was done to mimic the different ways humans might ask to write a story section. Sometimes they may want to tell the model exactly what to write, sometimes they want the model to choose.

Right now, there is no way to control which kind of output you get from this model, but you can regenerate until you get the desired length/level of detail if you like.