this post was submitted on 22 Feb 2025
58 points (92.6% liked)

Technology

63134 readers
3402 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
58
Who needs a sneaker bot when AI can hallucinate a win for you? - EQL Blog (www.eql.com)
submitted 1 day ago by cyrano@lemmy.dbzer0.com to c/technology@lemmy.world
8 comments fedilink hide all child comments
 

By this point, we’d narrowed down the affected users to a single email client - Yahoo Mail, which is where we got suspicious. Had Yahoo Mail introduced any features lately that might be causing this…?

As it turns out, yes, yes they had. A quick Google search revealed that a few months ago Yahoo jumped on the AI craze with the launch of ”AI-generated, one-line email summaries”.

At this point, the penny dropped. Just like Apple AI generating fake news summaries, Yahoo AI was hallucinating the fake winner messages, presumably as a result of training their model on our old emails. Worse, they were putting an untrustworthy AI summary in the exact place that users expect to see an email subject, with no mention of it being AI-generated 🤯

you are viewing a single comment's thread
view the rest of the comments
[–] andallthat@lemmy.world 13 points 1 day ago* (last edited 11 hours ago)

I think that using large language models to summarize email (especially marketing), news, social media posts or any type of content that uses a lot of formulaic writing is going to generate lots of errors.

The way I understand large language models, they create chains of words statistically, based on "what combination is the most likely based on my training material"?

In marketing emails, the same boilerplate language is used to say very different things. "You have been selected" emails have similar wording to "sorry this time you have not won but...". Same cheery "thanks for being such a wonderful sucker" tone and 99% similar verbiage except for a crucial "NOT" here and there.