“Research finds ChatGPT & Bard headed for ‘Model Collapse'”, writes Ilkhan Ozsevim in AI Magazine:
A recent research paper titled, ‘The Curse of Recursion: Training on Generated Data Makes Models Forget’ finds that use of model-generated content in training causes irreversible defects in the resulting models, where tails of the original content distribution disappear.
The Wall Street Journal covered the same topic, but it is behind a paywall and I have not read it: “AI Junk Is Starting to Pollute the Internet”.
They feed Large Language Models (LLMs) such as ChatGPT vast amounts of data on what humans have written on the internet. They learn so well that soon AI-generated output is all over the internet. The ever-hungry LLMs eat that, and reproduce it, and what comes out is less and less like human thought.
Here is a foretaste of one possible future from Euronews: “AI-generated ‘Heidi’ trailer goes viral and is the stuff of nightmares” – even if I do have a suspicion that to place a video so perfectly in the uncanny valley of AI-generated content still requires a human hand.