File size: 3,409 Bytes
c84fcfd 09ccde3 c84fcfd 36a254e 94a9c07 0cba3b0 febba1e c84fcfd 4a87663 b6f8a96 27bb943 b6f8a96 2d2d75a b2c6ed0 683aa20 cab6600 683aa20 b2c6ed0 683aa20 afe1ab5 a5ea12e 4a87663 a5ea12e c84fcfd 4a87663 c84fcfd |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 |
---
license: other
license_name: xt-aurora-license
license_link: LICENSE
language:
- en
- es
tags:
- conversational
- chat
- roleplay
library_name: GGUF
pipeline_tag: text-generation
base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-715k-1.5T
datasets:
- XeTute/Small-Medium-Conversation-Multilingual
- XeTute/Conversational-Small
---
**Note**
With the release of Meta's LLaMA 3.2 1B, this model got outperformed significantly. Since we don't have a lot of GPU power or money to furter train this or another model to even come close to Meta's models, we recommend you to use theirs over ours.
We, XeTute, introduce AURORA V1.0 - a humerous, efficient, smart(for its size) and unbiased(because of too low parameter count, consider it a virtual child with a bunch of knowledge =)) Language Model.
**Intended usecases:**
- Next-Word prediction for mobile devices:
- - This Model can be reliably packaged into a keyboard-app to help make Next-Word suggestions more accurate (for performance, INT4 or less might be smart)
- Conversations:
- - AURORA can engage in conversations using the Vicuna format, remember to replace "ASSISTANT" with "AURORA" though.
- - AURORA can engage in SFW roleplay with simple character definitions. It wasn't trained on NSFW.
- - AURORA can engage in simple, short Q&A. It was trained on factual data too, which means it performs well for its size.
**Training:**
- Trained for two months.
- Dataset created by XeTute, and translated using different free-lancing services.
- Dataset included:
- - Mathematic Q&A
- - Logic Q&A
- - One-Page stories and roleplays with very brief character definitions
- ADAM as an optimizer.
Alltogether, the model was trained on additional 20B tokens.
<a href='https://ko-fi.com/C0C2ZXNON' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi3.png?v=3' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a>
Note:
- All previous beta versions of this series of SLMs were deleted, because almost no downloads were made.
- V1.0 is the last model in this series which will be published, because of too little community activity.
Recommended settings:
- Temperature 0.1 - 0,4 is stable.
- Context Length of 2048(base) to 4096(RoPE) will work well for story-telling, role-playing and simple conversations.
- Output Length: 256 will work very stable, but you can extent to 512. Anything beyond that point is risky, text might become repetitous.
- A system prompt which works well can be found at "Files at Versions" => "chat_template". Just copy and paste this into the system prompt or add it before your first message.
- Chat Format:
```For roleplay:
{name of your roleplay}: {input}
{name of AURORA's character}: {output}
```
or,
```For normal chatting:
USER: {input}
AURORA: {output}
```
Chat examples using KoboldCPP and the settings recommended above:
Note, a roleplay where you directly pass character definitions and a starting scenario will work way better, this is just an example.
We wish you a friendly chat with AURORA. |