Largest study of its kind shows AI assistants misrepresent news content 45% of the time – regardless of language or territory
An intensive international study was coordinated by the European Broadcasting Union (EBU) and led by the BBC
New research coordinated by the European Broadcasting Union (EBU) and led by the BBC has found that AI assistants – already a daily information gateway for millions of people – routinely misrepresent news content no matter which language, territory, or AI platform is tested.
8 comments:
From the report: "Professional journalists from participating PSM evaluated more than 3,000 responses from ChatGPT, Copilot, Gemini, and Perplexity against key criteria, including accuracy, sourcing, distinguishing opinion from fact, and providing context. "
No mention of repeating the analysis for main stream media. I rather suspect that the MSM would be found lacking in many similar ways.
But when the gamekeepers report on the poachers there will be bias. Not a surprise.
Is 'routinely misrepresents news content' a BBC synonym for 'does not support our point of view'? After all, it would be strange if multiple AI programs were to 'misrepresent' the same stories in the same way. Climate models don't get close to unanimity in their predictions.
DJ - yes that's it, a gamekeepers report on the poachers. Legacy media are battling for their existence - from their perspective the "right" conclusion is too obvious to take them seriously, especially when we take past performance into account.
Anon - good point. There is nothing wrong with differing points of view, but with big media this is often what their stories are, a point of view adopted for undivulged corporate reasons.
Tried using the AI at an art site, to generate pictures. The results seemed OK at a very superficial level but as soon as you look at all closely they were a mess; people with missing limbs, and/or disembodied body parts, sometimes even multiple heads. Impossibilities such as features that are in the background but nevertheless cross in front of foreground figures. It clearly has no real understanding of the 3-D elements it has been asked to represent in 2-D form. If current AI does that with graphics, it presumably does the same with text, assembling sentences that are plausible at first glance but quite unrelated to any real understanding of the material.
For me, current AI still fails the Turing test and trusting it for real world use would be asking for trouble.
Wot, just climate? I can't remember the last thing that the BBC represented properly
Barbarus - I find Copilot AI is okay for fairly simple blogging graphics if nobody is likely to look closely at the detail - there will be errors.
One problem with AI text is that high standards are applied to its output, presumably because we expect to trust it at some point. To my mind it is already more 'intelligent' than most people if asked clearly phrased questions and it is possible to have an 'intelligent' conversation with it. It isn't human intelligence though, it's different.
Bucko - yes, climate is just the one which stands out, but anything to do with the environment, energy, politics, economics is likely to be slanted one way or another.
How we would like to think we think — logical, rational, reasoned, consistent… but most of what we say is just bleating and chirping, making the right noises to be with herd and flock. AI passed the 'Turing test' long ago.
djc - yes I remember an article on it passing the 'Turing test' and that was quite a while ago. Didn't keep a link though.
There seems to be a general problem in admitting how good AI is even though many people are trying to tell us. A few AI oddities or errors are magnified without reference to the much higher level of human errors and oddities.
Post a Comment