README.md

---
language:
- en
---

05/10/24

thank you for all the support and feedback, I appreciate them.

really.

---

17/09/24

So, false positive for TB? lmao thats cool.

Life goes on and on.

Anyway, regarding below, it wont affect a regular person, because duh. An individual can easily discard and ignore what I wrote, and I wont do anything. This is why I dont get the outrage on some threads lol.

It only matters for companies and groups, which well, are the main reasons.


---

14/09/24

Please respect the model licenses, especially for variations of Lyra for mistral nemo.

Id rather not resort to outright gating the full weights and releasing quants only.

If you have talked to me in the past about licensing, nothing changes for you, feel free to use my models. this is mainly because ive seen Lyra used with so many damn merges.

have a nice day

---

28/08/24

Yayy Audit!

oh wait. we're getting audited today. 

time to-recheck everything in the ambulance to make sure it's flawless. got to make sure shit is flawless too.

oh wait, they aren't auditing our station today (literally last minute cancellation)

lmao what a waste of time. 2nd cancelled in 4 months.

---


15/08/24

Death. Sick people. Just poor living conditions. It does wear you down after a while. Poor sleep doesnt help.

Mental health down the gutters.

1 more year and im out. hopefully. Then reservist cycles. then signing on because job stability. 

it will never end, huh?

\- ignore my rambling. its nice to let these feelings out once in a while. im okay. really.

---

09/08/24

national day and I'm out here working in the ambulance smh

it never stops

---

04/08/24

model testing is not fun at all

---

31/07/24

Going home today, nice.

anyway, some people are just *entitled* as hell.

a few months ago our crew went to a cardiac arrest case, family was hysterical and didn't help. we saved the husband, drugs were pushed, he went back to life and is well now.

only a few days ago they decided to send a complaint, all because we were *late?* tf lady. your husband is alive and fine now.

here's my crypto (eth) (metamask) wallet since someone asked, you don't have to, but it helps
<br>`0xb2f71C762990e1FC3353319fA63f2C65249d9379`

---

27/07/24

welp, coin locker was spoiled. lost some items.

thankfully nothing of importance was lost, though some merch was?

it is what it is, thankfully my major expenses are settled.

have to budget and live frugally my remaining few days here though, kek. 

>>>>
>>>> 
heh, scratch that. knew i was missing something bigger 💀
 fuck 

---

23/07/24

Spontaneous Japan Trip - Tokyo Haul

![Bocchi](https://huggingface.co/Sao10K/Ramble/resolve/main/20240723_202327.jpg)

Kita is literally me.

========


08/07/24 - Midnight Thoughts

I wonder what I should do in life later on. 1 and a half years to go before this ride ends. Ugh, day shift starts in 7 hours.

\- Continue being an EMT after my National Service ends? I could probably sign on as an EMT-Driver, or join Private Ambulance services as one, my seniors did, seems to pay well too.

\- Maybe sign on and become a Paramedic instead? Seems cool, I have seen first hand the job realities myself working under them, and I do have a relevant diploma.

\- Orrr maybe go back to Nursing, since my initial Diploma was on that. Whoo. Being an RN is not too bad of a job, me thinks.

Just thinking on my life prospects. AI is a hobby and all, I'm working on projects in my free time, but life goes on.

---

05/07/24 - Meh

have a lot of ideas planned but nothing much can be done *yet*.

anyway dataset curation is the most tedious of them.

gemma 2 seems actually nice for once.

---

04/07/24 - Bruh 

\# Update 2

it's been half a day and I have not been in station. it's always proceeding to call after call. ugh.

\- \- \- 

Literally first step into ambulance at beginning of shift. literally just entered fire station less than a minute in.

Motor Vehicle Accident P1+

spinal + head injury.

what a wonderful day.

---

30/06/24 - I'm updating my model page but its 2am and I'm going to nap and fix everything in the morning aight night.

---

26/06/24 - Fire Post yay

Half in during my shift and I finally managed to reach the Fire Post without getting proceeded on the way there.

busy day.

---

25/06/24 - Post Audit Day

My knees are like fucked from staying in position so long during ths spinal management audit proficiency check.

the tester kept yapping while I was suffering in place holding the damn pelvic because of course pelvic fracture is tested too.

plus who the hell uses spinal board now for this, its all scoop stretcher. ugh.

so not worth the measly government allowance. 1.5 years to go.

---

24/06/24 - Audit Day

Nice

---

20/06/24 - #2

👋

---

20/06/24 

:) 

---

19/06/24 

```
2024-06-18 05:05:33,703 Error 524: A timeout occurred
2024-06-18 05:05:47,959 Error 524: A timeout occurred
2024-06-18 05:06:58,994 Response contains forbidden phrase 'Claude'. Attempt 2/4
2024-06-18 05:07:12,822 Error 524: A timeout occurred
2024-06-18 05:08:20,140 Response contains forbidden phrase 'Claude'. Attempt 3/4
2024-06-18 05:08:45,532 Response contains forbidden phrase 'I cannot'. Attempt 1/4
2024-06-18 05:08:53,022 Response contains forbidden phrase 'Claude'. Attempt 1/4
2024-06-18 05:09:14,313 Response contains forbidden phrase 'I will not'. Attempt 2/4
2024-06-18 05:09:31,185 Response contains forbidden phrase 'Claude'. Attempt 2/4
2024-06-18 05:09:34,251 Error 524: A timeout occurred
2024-06-18 05:09:42,272 Response contains forbidden phrase 'Unfortunately, I'. Attempt 3/4
2024-06-18 05:09:48,519 Response contains forbidden phrase 'Claude'. Attempt 4/4
2024-06-18 05:09:52,523 Failed to get a successful response after 4 attempts
2024-06-18 05:09:52,523 No response data received
2024-06-18 05:10:08,380 Response contains forbidden phrase 'I apologize, but'. Attempt 4/4
2024-06-18 05:10:12,385 Failed to get a successful response after 4 attempts
2024-06-18 05:10:12,385 No response data received
2024-06-18 05:10:16,391 Response contains forbidden phrase 'Claude'. Attempt 3/4
2024-06-18 05:11:01,555 Response contains forbidden phrase 'Claude'. Attempt 4/4
2024-06-18 05:11:04,558 Failed to get a successful response after 4 attempts
2024-06-18 05:11:04,558 No response data received
2024-06-18 05:11:18,417 Error 524: A timeout occurred
2024-06-18 05:11:41,451 Error 524: A timeout occurred
```

---

18/06/24 - mmmmmmh

someone said qwen, yeah?

https://huggingface.co/alpindale/magnum-72b-v1

***

14/06/24 - Poggers

The first time was so nice, I had to do it twice :)

Euryale v2.1 and Stheno v3.2 was a huge success fortunately, I liked the results. 

They're two of my favourite models, Euryale 1.3 is a close 3rd, and Fimbulvetr-11B-v2? Honestly I began disliking it even before Llama 3 came out, and that was my magnum opus at the time. 

Still is like my most popular model though?

Anyway, I think I am done for now. Euryale and Stheno have hit my sweet spot, I like how they turned out. Suree you can improve some minor stuff but it's not worth the effort to retrain. I'll just keep working on the dataset.

A Qwen-2 72B train would have been nice for the long context, but L3 70B did cost too much. Labour of love and all, but my allowance barely covers this train and expenses. National Service do be like that.

Plus I have to soon take more responsibility soon, it wouldn't be long before I become a senior medic, when the current seniors pass out in time.

I don't think I'll be releasing anything new for a while, except maybe when something super special gets released.

Peace out.

---

11/06/24 - update 2

H100s going Brr!

11/06/24 - Life 

i feel restless when I'm not working, but I feel so tired at work.

i literally spent most of my break in bed. sleeping, or on my phone. so lazy.

huh, it is what it is.



***

09/06/24 - Update 2

Having to clear leave is so fun, next block off, work, off again. I still have like over a week left of leave to clear.

Sucks I can't save everything up so I can go on one long staycation at home but it is what it is.

09/06/24

Lmao training L3 70B is such a pain in the ass compared to L2 70B kek.

H100s go brr though?

Euryale v2 SoonTM. Maybe.

***

02/06/24 - And another one.

Medical Cardiac Arrest P1+

That makes 4 Cardiac Arrests within 2 work blocks. I had like zero the past two months.

What a wonderful week.

***

30/05/24 - Worst Day Block ever.
- 2nd Day Shift - Horrible.
<br> Supposed to be stationed at Fire Post. Proceeded to case without even touching the Post multiple times. I haven't even set foot there yet.

13:15 - Unconscious / Fainting P1+ -> On Scene -> Medical Cardiac Arrest P1+

What's up with the people dying?

*my besto friendo*

![CorPuls](https://huggingface.co/Sao10K/Ramble/resolve/main/20240530_145359.jpg)

***

29/05/24 Update 2

- Euryale v2... soon... ish. It would take 141 hours for a 70b qlora on a Single A100. I don't have the compute and money for that lol. it'll come when it comes. which is somewhere in the near future.
- Stheno v3.2 soon-ish? Dataset's ready, just need to train it.
- Other experiments I'm doing a do-over.

29/05/24 - Update 1

14:05 - Stroke / CVA -> On Scene -> Medical Cardiac Arrest P1+

12:41 - Medical Cardiac Arrest P1+

i fucking hate this job sometimes. 2x CA in a row.

***

25/05/24 - (Near) Midnight Blues

2nd Night Shift Block in a row.

![Alpha](https://huggingface.co/Sao10K/Ramble/resolve/main/20240525_225614.jpg)

***

18/05/24 - Random Update 

what's up, gang?

So, where have I been? Here and there. a few private testing runs on llama 3.

Opus-Instruct Single Turn is done. Multi-turn working. c2 Logs were a fun side-distraction.

my stacked merge projects, 8x22b tune, all that? was honestly held back because of llama 3. I really want to work on them but I'm procrastinating... a lot. i have pretty much the tools setup. just have no desire to run the training runs itself.

random life updates: I got promoted to LCP, aced my EMTCT tests, so that's cool? also if any Singaporean's here, you seen the Tampines deadly multi-car accident last month? yeah, I responded to that. was honestly a sobering sight. damn.

anyway, ya boy's still alive and active. and cooking. I'll try to release something by the end of this month.



---

18/04/24 - Where am I?

had a few messages asking if I'm alive. yeah, I am. I occasionally talk in the KoboldAI or SillyTavern discords. Anyway, life update. Had a short trip to Batam for Hari Raya and all that, it was a short and refreshing break. You know why Batam is popular;)

where are all the projects I promised? I'm... working on them. yeah. So many new cool things came out, including the Maxtral models, that seemed easier to tune on, just more expensive. Really, I'm working on them. ($75 per 1M output tokens for Claude Opus is outrageous kek, and I'm working on ~10M tokens) The stacked merge and finetune experiments? working on those too. For my old followers, Hesperus? shit, I'm remaking those.



***

09/04/24 - Off topic

being an EMT honestly is fun work? kind of stressful at times, and 12h shift sucks, but its fun? a peek into my life so far lol. writing it out is really how to say it, stress relieving? relaxing? cathartic?

some cases are honestly bad and stressful, theres the occasional trolling uncles, routine cases and everything. kept me at my toes the whole shift, especially when we were first call.
look ma, I'm on the news? 

https://www.straitstimes.com/singapore/motorcyclist-50-taken-to-hospital-after-accident-near-bartley-road


<details><summary> totally routine topups that shift: </summary>
  
```
Penthrox x1
Tramadol x1

Nasal Cannula x2
Adult SFM x1
Adult NRM x1

Adult BVM x1
Adult Defid Pad x1

IV Admin Set x1
Ns 500 ml x1
20G Cannula x2
21g Cannula x1
3ml Syringe x1
Tegaderm x1

10cm Crepe Bandage x1
7.5cm Crepe Bandage x1
5cm Crepe Bandage x2
Gauze x1
Triangular Bandage x5
Large Pad x2
Small Pad x2

Surgical Mask x50
Linen x3

L Gloves x100
BP Sharp Tag x1
Ice pack x3

4/7
```

</details><br>

*This shouldn't violate any privacy thing? there's like no details leaked, ok govt?*


***



05/04/24 - Mini Updates

what's up huggers, it's like 5am here, on my phone, and I'm working night shifts. yeah boi.

fimb v3 stuff is slow as I mentioned on their page, if you wanna support me on ko-fi (on my page), that's cool. if not, i have my own money, feel free, support helps, appreciate those who do.

Why shit is slow:
- aphrodite-engine does not want to cooperate with me for dataset generation (toxic / nsfw instruct datasets)
- Claude Opus isn't on AWS yet, anthropic console sucks balls (I'm not American, sucks for me). I'm using Claude Sonnet in the mean time.
- I'm just so tired after shifts. plus, I'm getting called back again because someone else is sick so less time for me, have to help cover another rota. shame, planned on working on my data that day.
- Burnout, crash out? don't really know. i worked really hard on all this, I'm feeling the effects. I'm someone who well, burns themselves out after going hard and focusing 100% on a task. it's a habit I'm trying to fix.

have a good day, or night, wherever u are.

***

29/03/24 - Sup

Whats up gang its your favourite man out here.

Fimb v3 is in the works but loss values are fucked with latest axolotl versions? Unsure, but I know the seperate datasets trained fine in the past, now they're at > 13 loss in the beginning? Idk what's going on here.

Solstice v2 is in the works, dataset expanding.

Senko is in a rock right now, having issues with what I want it to do exactly. Test versions have their issues and all, but its not as much of a timesink as Fimbulvetr-v2 was.

Typhon and Glacier finetunes are on the backburner, lack of time mainly, plus its the fasting month and all. Not much energy after 12h night shifts. 

Claude 3 instruct data being worked on and all.

The main hardest thing about finetuning and all is the data curation, training is just money-consuming but easy to monitor and manage. The data creation, filtering, cleaning? That's hard.

mmm, funds are there, just need to finish curating data to tune.


***

14/03/24 - Updates

I miss her man

anyway im doing some experiments so feel free to try out Shiki and Senko, or not up to you tbh

ymmv with the models they are kinda sensitive to prompting

and yes I'm not kinda sound right now mentally but we ball

ohio out


***

09/03/24 - Just rambling on about my future plans

I don't even know what to do man. So many ideas, so little money and time.

I'm practically waiting on my paycheck so I can continue training.

Current Plans

- 14B-Glacier-Finetune - Sure a Full Finetune would be good but that would be expensive. A LoRA maybe? The goal is to smoothen out the tensors and repair brain damage that happens after frankenmerge.

- Typhon-Mixtral Finetune - I have compiled a nice dataset to further fine-tune on this merge of mine, to hopefully improve results. Unsure if I should further DPO this one too? On non-gpt4 data of course unlike current DPO pairs.

- 11B-Fimbulvetr-v3 - I have ideas. Unsure if it would make it that much better or if it will be a side-grade.

- Claude-3 Output-based Instruct Dataset - Cool Idea maybe? It'll introduce atleast a different '-ism' compared to GPT-4 ones out there which are generic. Claude is somewhat more human-like. I do have access so this is not as big of an issue. I just need more time.

- Yi Tunes - Yeah this is on the backburner for a while. I'll think on it. I've struggled with Yi-tunes in the past. Most of mine are privated.

Ignoring that, man having gauze packed into a wound hurts like hell without painkillers. My bad, old patients of mine back when I was a nursing student. You all made it look not that bad.

Anyway, here's something I've been listening to this past week. Some random music for someone who read this entire thing haha.

[Here](https://www.youtube.com/watch?v=t05Bd6xSn6c)

Today's Brainrot:

<img src="https://huggingface.co/Sao10K/Ramble/resolve/main/ohio.jpg"  style="width: 60%; min-width: 200px; display: block; margin: auto;">

***

***

Initial Ramble:

Someone questioned my model naming scheme. Good question. Here's how I chose my model names.
```
Stheno --> 13B L2 Merge Serie ---> It all Began Here --> stolen from discord, someone's idea
Medusa --> 7B L2 Merges ---> DOA
Euryale --> 70B L2 Merge Series --> Gorgon Sisters Continuation 
Zephyrus --> L1 Merge --> Sounded Cool tbh
WinterGoddess --> L2 70B Merge + Tune --> Training Run Name: (Autumn Royal, Winter something I forgot.) Decided to brainstorm some fun names. Yeah. Dont know where the Goddess part came from, but I was writing some ERP between a guy and a goddess, and the test model helped critique it, so yeah.
```

Why obsession over the theme of Winter? I do not know where to begin. I was sick, delirious during training break, and my brain got obsessed with the theme of winter. I stuck my head in the freezer once. I ate ice. I'm silly like that.

```
Fimbulvetr --> Final Winter. Represents the end of it all. My peak Solar model imo.
Frostwind --> Initally wanted something to do with Mistral + Solar, had Claude suggest Frostwind as a name. Initial plan was to train Mistral, but solar came out so I took that name instead.
Solstice --> Solar, Sun, Peak, or (Orgasm, you know?) since it was a Lewd/NSFW Instruct Dataset
Sensualize --> Had GPT4 generate me a name that is for an NSFW model. Yeah.

Older Model Names:
Nyaa --> Meow. Trained on cat assistant vibes.
Winterreise --> As above, training run for another model was named Sonata. sounded cool as fuck when i asked a model to merge Winter + Sonata.
Nyakura --> I was obsessed with cats during that time.
Venomia --> Trained over toxic datasets.
Ana -> 7B Mistral Merge + Tune --> Medusa revived, you know, that FGO Ana.
Lila --> RP with a character, obsessed with her while testing Euryale 1.5 (WinterGoddess)
Solus --> RP with a character, obsessed with her while testing Euryale 1.5 (WinterGoddess)... yeah.
```
***

Next question. What's the worst cases I've seen so far?

A traumatic arrest, a few cardiac arrests, accidents here and there. Most shifts are okay though. Some shifts are just tough but it is what it is.

***