I was using Bing to create a list of countries to visit. Since I have been to the majority of the African nation on that list, I asked it to remove the african countries...
It simply replied that it can't do that due to how unethical it is to descriminate against people and yada yada yada. I explained my resoning, it apologized, and came back with the same exact list.
I asked it to check the list as it didn't remove the african countries, and the bot simply decided to end the conversation. No matter how many times I tried it would always experience a hiccup because of some ethical process in the bg messing up its answers.
It's really frustrating, I dunno if you guys feel the same. I really feel the bots became waaaay too tip-toey
I had an interesting conversation with chatgpt a few months ago about the hot tub stream paradigm on twitch. It was convinced it's wrong to objectify women, but when I posed the question "what if a woman decides to objectify herself to exploit lonely people on the Internet?" It kept repeating the same thing about objectification. I think it got "stuck"
I think the ethical filters and other such moderation controls are hard coded pre-process thing. That's why it repeats the same things over and over, and has the same hangs up as early 00s poorly made censor lists. It simply cuts off the system and substitutes a cookie cutter response.
I find it interesting that they don't offer a version of GPT 4 that uses it's own language processing to screen responses for "unsafe" material.
It would use way more processing than the simple system you outlined above, but for paying customers that would hardly be an issue.
It's possible but also incredibly complicated and technically involved to tweak a LLM like that. It's one of the main topics of machine learning research.
A lot of people get stuck with issues like that where there are conflicting principles.