You hear it everywhere. Whenever someone discusses the tendency of Large Language Models (LLMs) like ChatGPT to make up facts or present fictional information, they say the LLM is hallucinating. Beyond simply being a term used by the media, it is also used by researchers and laypeople alike to refer to whenever an LLM produces text which, in one way or another, does not correspond to reality.
Despite its prevalence, the term is, at best, somewhat deceptive, and at worse, actively counterproductive to thinking about what LLMs are actually doing when they produce text which is deemed problematic or untrue. It seems to me that “hallucination” is a bad term for several reasons. It both attributes properties to the LLMs they don’t have while also ignoring the real dynamics behind the production of made-up information in their outputs.