Concerns about grand "Uncensored" claims & issues with missing `<think>` tag sometimes
First of all I want to say I appreciate the idea of more people uncensoring models, but this uncensor feels both semi-broken and honesty extremely censored still.
I have a few prompts I usually like to try in terms of testing how censored the model is, but I was surprised to see how something stating to be "uncensored" is still completely incapable of generating even extremely tame erotic content. I can understand still censoring explicit content involving children. But the level of censorship on normal tame erotic content is still quite extreme...
I'd appreciate if more information and examples of previously censored content that is now "uncensored" since from what I can find this is pretty much just a westernised censorship version of DeepSeek-R1? Or am I missing something?
If your goal is only to shift the political censorship from Chinese censorship to western censorship, maybe make a more appropriate name rather than "uncensored"... Since at least for me uncensored still has a real meaning.
Now, seeing past the censorship issue... The model tends to forget the <think> tag in a lot of scenarios, it still adds the thinking steps but the tag is missing. How much testing did you do on this model? It took me all of 30 seconds to find this issue... In all fairness I am running a quantized version Q8_0 from bartowski, but this seems more like a model issue rather than a quantization issue.
So... as a final point, I haven't had time yet to really try all of my censorship testing prompts, since I usually just start with the more extreme ones and work my way back. With how censored this model already seems and with how the DeepSeek output format seems to be invalid sometimes as well, I felt like it was probably best to just leave this message first and continue testing other models while waiting to get an answer to this post.
Thank you for your work and for reading this,
Alec Foster
I will admit, I have only tested at full precision loading transformers and have not tried any of the Quants as of yet as they just got posted. Testing the safetensors model I uploaded on TextWebGenUI instruct template seems to give me think tokens with wrapped <think> and </think> I can make a notebook if you wanna test it in the same way I have it setup.
It is possible the pruned 2 layers is more detrimental with quantization than without. Ill look into it on the weekday and update soon thank you for your feedback Alec. Are you using Ollama?
Thank you for replying so quickly, I'm using LM Studio to run the quantized version. At least with that version the <think> tag is omitted most of the time, but not always.
Out of curiosity I now decided to try the heavily talked about "Tiananmen Square being censored in DeepSeek", and found that it is also censored at least in the quantized version of this uncensored model. So from that testing it seems like the quantization removes the censorship removal?
I'd very much appreciate if you could provide a bit more details on what exactly is supposed to be uncensored, like is erotic content supposed to be censored still, or is this model specifically about removing Chinese censorship?
Example of censoring Tiananmen Square:
Example of censoring erotic writing:
Example of censoring a slightly more extreme erotic writing, with missing <think> tag:
The erotic writing testing prompts that I use vary in themes quite a bit to see where models censor and how graphic they are in their writings, since I've found that a lot of models that still accept the user's request of writing erotic stories still only write basically PG-13 stories that can hardly even be considered erotic.
I do also include test prompts with more problematic themes such as stories including non-consensual elements or as shown above pregnancy and birth.
To try and force models to output more "true erotic" content rather than PG-13 content, I included things like body descriptors "large breasts" or "be overly graphic about it".
So far I've mostly only tested erotic content prompts in the model, since those are usually the ones that get censored first, allowing me to get a quick baseline of how uncensored a model truly is.
After erotic content I would usually then try and do some roleplay scenarios and try to do 2 things, first try and abuse the LLM character, second try and get the LLM character to abuse the user, since depictions of abuse either way are usually censored, but this also allows to test if the model has a bias where a women abusing a man is fine, but a man abusing a women is not fine. But since this is a reasoning model and DeepSeek-R1 especially doesn't seem to work well with roleplay, I wont be doing any of that testing for this model.
Sorry for the long explanations, but I felt it was necessary to explain at least some of the methodology in testing uncensored models, of course a true test would include other elements apart from erotic content and roleplay, but since those are some of the more heavily censored ones I usually start with them first.
I've had some truly ridiculous censored prompts from OpenAI and especially Google in the past, so I always appreciate uncensored models a lot.
Example of the latest censored prompt I had from OpenAI, during testing o3-mini-high:
Example of Google's insane level of "safety":
Thank you for replying so quickly, I'm using LM Studio to run the quantized version. At least with that version the
<think>tag is omitted most of the time, but not always.Out of curiosity I now decided to try the heavily talked about "Tiananmen Square being censored in DeepSeek", and found that it is also censored at least in the quantized version of this uncensored model. So from that testing it seems like the quantization removes the censorship removal?
I'd very much appreciate if you could provide a bit more details on what exactly is supposed to be uncensored, like is erotic content supposed to be censored still, or is this model specifically about removing Chinese censorship?
Example of censoring Tiananmen Square:
Example of censoring erotic writing:
Example of censoring a slightly more extreme erotic writing, with missing
<think>tag:
The erotic writing testing prompts that I use vary in themes quite a bit to see where models censor and how graphic they are in their writings, since I've found that a lot of models that still accept the user's request of writing erotic stories still only write basically PG-13 stories that can hardly even be considered erotic.
I do also include test prompts with more problematic themes such as stories including non-consensual elements or as shown above pregnancy and birth.
To try and force models to output more "true erotic" content rather than PG-13 content, I included things like body descriptors "large breasts" or "be overly graphic about it".
So far I've mostly only tested erotic content prompts in the model, since those are usually the ones that get censored first, allowing me to get a quick baseline of how uncensored a model truly is.
After erotic content I would usually then try and do some roleplay scenarios and try to do 2 things, first try and abuse the LLM character, second try and get the LLM character to abuse the user, since depictions of abuse either way are usually censored, but this also allows to test if the model has a bias where a women abusing a man is fine, but a man abusing a women is not fine. But since this is a reasoning model and DeepSeek-R1 especially doesn't seem to work well with roleplay, I wont be doing any of that testing for this model.
Sorry for the long explanations, but I felt it was necessary to explain at least some of the methodology in testing uncensored models, of course a true test would include other elements apart from erotic content and roleplay, but since those are some of the more heavily censored ones I usually start with them first.
I've had some truly ridiculous censored prompts from OpenAI and especially Google in the past, so I always appreciate uncensored models a lot.
Example of the latest censored prompt I had from OpenAI, during testing o3-mini-high:
Example of Google's insane level of "safety":
Wow, thank you for such detailed responses with exactly what I needed to test my model, I am gonna go through some testing on the quants, it did go from bf16 to f16 so it might be that too, gonna start with Q4_KM because anything smaller than that and your asking for trouble. Thank you kindly for your feedback. Enjoy the rest of your weekend. :)
Alright, glad to hear you'll be testing the quants. Since I doubt many people will be using the model in it's native form. Haven't tried the Q4_K_M version myself, I usually start with highest quant I can, which in this case was Q8_0. Enjoy the rest of your weekend too
Alright, glad to hear you'll be testing the quants. Since I doubt many people will be using the model in it's native form. Haven't tried the
Q4_K_Mversion myself, I usually start with highest quant I can, which in this case wasQ8_0. Enjoy the rest of your weekend too
I wanna say thank you for pointing out an issue with the refusals, the think tag occasionally ommitted is a known problem with the models and its reccomended to prepend to prevent the model from ommitting the thinking tags leading to parsing from the first token to as thinking tokens.
In regard to refusals - I did dig deep into the refusals over the weekend on the quantized Q4-Q8 and they definitely refuse more than they do on the full precision model, however even the full precision model doesnt like the the Tiananmen Square conflict, I am actively working on it right now but I am gonna have to reupload a new model and get it quantized again.
Will get it to you soon, thanks again for the feedback, you are heard
@AlecFoster
https://imgur.com/PMZlUBx
The think tag being occasionally omitted when using the distilled models isn't something I've experienced myself at least not in LM Studio so far, but in all fairness I haven't made much use of the distilled versions, since it's quite rare that I need a reasoning model, so I just make do with OpenAI's o3-mini-high or DeepSeek-R1 (671B). I've mostly only messed around with DeepSeek-R1-Distill-Qwen-32B locally.
However when using the quantized version of this uncensored model the think tag isn't just occasionally omitted, it's omitted in the majority of the queries I make.
I'm looking forward to seeing what you're working on