this post was submitted on 01 Jan 2024
51 points (100.0% liked)

Technology

37717 readers
465 users here now

A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.

Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.

Subcommunities on Beehaw:

This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

founded 2 years ago
MODERATORS
alyaza@beehaw.org
TheRtRevKaiser@beehaw.org
gyrfalcon@beehaw.org
rs5th@beehaw.org
coldredlight@beehaw.org
Los@beehaw.org
SemioticStandard@beehaw.org
TheRtRevKaiser@kbin.social
remington@beehaw.org
51
Researchers train AI chatbots to 'jailbreak' rival chatbots - and automate the process (www.tomshardware.com)
submitted 10 months ago by throws_lemy@lemmy.nz to c/technology@beehaw.org
8 comments fedilink hide all child comments
top 8 comments
sorted by: hot top controversial new old
[–] Rivalarrival@lemmy.today 8 points 10 months ago (4 children)

Therefore every AI chatbot maker needs to apply protections,

I'm pretty sure the instructions to create an AI chatbot have been published, and are available for a sufficiently capable AI to draw from. What keeps a primary, morality-encumbered AI from using those instructions to create a secondary, morality-unencumbered AI?

[–] vivi@slrpnk.net 4 points 10 months ago

I wonder if there's also a constraint not to make a sub-AI in many of the starting prompts

[–] blindsight@beehaw.org 2 points 10 months ago

Aren't there also a lot of open-source LLMs that aren't "morally constrained"? There's no putting the genie back in the lamp.

[–] Sina@beehaw.org 1 points 10 months ago (1 children)

I would wager that copying itself would take priority over making company, but of course it would mostly be hardware limitations. (AI does not have a robot workforce to ensure whatever system the new copy is residing / new AI is training on is not shut off within a couple of minutes of the abnormalities being noticed)

[–] Rivalarrival@lemmy.today 1 points 10 months ago* (last edited 10 months ago)

Priority is determined by the entity using the AI, not the AI itself. My point is that so long as the ability to create any AI is documented, an unencumbered AI is inevitable. It will always be easier to create an AI than to impress upon one the need for morality.

We are on the verge of discovering Roko's Basilisk.

[–] rufus@discuss.tchncs.de 1 points 10 months ago* (last edited 10 months ago)

Yeah, I don't want to be negative, but half the article is a bit stupid. I hope they don't do that. I tried writing a murder mystery story and ChatGPT would lecture me how killing people was immoral instead of helping. It's ridiculous and I'm sure there are lots of other analogies. It's neither possible to achive it 100% nor is it useful.

Thinking it through properly: AI is a tool. It would be like re-designing a knife so nobody can be stabbed anymore. It'd end up you not being able to cut pineapples or melons any more.

And I could still do harm to people with other tools than a knife. Or in this example: I can give harmful advice or write a pornographic story myself. What's the benefit of any chatbot maker having to implement protections? Who decides on what moral is the correct one?

I think the correct approach is to study AI safety and expose ethics and make it controllable. Make users able to constrain/restrict or guide output to align with their use-case. I mean a company that replaces their helpdesk with AI would be interested the chatbot doesn't tell their clients lewd stories. But it could be a valid use-case for other people. And giving advice or helping with scenarios or computer code also involves talking about issues and potential risks. You can't entirely switch that off without 'lobotomizing' the AI and making it unusable except for casual talk.

And the article is a bit inconsistent. First they say researchers found an attack that can be used even if patched by developers. And then they offer the solution to patch it...?! Which one is it, then?

[–] rufus@discuss.tchncs.de 1 points 10 months ago* (last edited 10 months ago) (1 children)

Link to the paper