Generative-AI output presented as factual but unsupported; contested as metaphor, defect class, and liability trigger.
In clinical use of generative tools — ambient scribes, discharge-summary drafters, literature assistants — a hallucination is any statement in the output that is not supported by the source encounter, chart, or cited literature: an invented medication, a lab value never measured, a fabricated reference. Clinicians treat the draft as unverified content: every assertion that would enter the record or influence care must be traceable to something the clinician saw, said, or can check before signature. A fluent, confident output containing one unsupported medication line is a patient-safety hazard on par with a transcription error, not a stylistic flaw.
Verify every clinically consequential assertion in a generated draft against the chart or source encounter before signing, and report fabricated content through the incident-reporting channel like any other documentation error.
Koenecke et al. (FAccT 2024) found OpenAI's Whisper inserted fabricated sentences, including invented medications and violent content, into transcripts — a documented hazard for AI scribes built on speech-to-text.
OmniGloss seed synthesis, 2026 (machine-drafted, pending expert validation)
For clinical leadership deciding whether a generative tool enters the workflow, hallucination is a quantified acceptance question: what fabrication rate, on which task, with what severity distribution from cosmetic to care-altering, caught by which verification step. Vendor claims of low hallucination mean little without task-specific evidence — a rate measured on exam questions says nothing about discharge summaries. The operative decision is where in the workflow a fabricated statement could reach a patient without clinician review, and whether the residual rate after that review is compatible with the department's risk tolerance and its duty of care.
Demand task-specific fabrication-rate evidence with severity grading before authorizing deployment, and define where mandatory clinician verification sits in the workflow as an explicit condition of approval.
e.g., a hospital pilots an AI discharge-summary drafter only in departments where a physician co-signature step is enforced, after reviewing the vendor's error study on the hospital's own specialty mix.
What counts as a clinically significant fabrication versus a benign paraphrase varies between institutions; some count omissions of critical findings as hallucinations while others class omissions as a separate error category, which makes vendor rates non-comparable across sites.
Elham Asgari et al. (2025). A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation. NPJ Digital Medicine
For clinical NLP teams, hallucination is operationalized against a grounding corpus: an output span is hallucinated if it is not entailed by the source note, structured record, or retrieved evidence, distinguishing intrinsic errors (contradicting the source) from extrinsic ones (unverifiable additions). Teams measure it with entailment classifiers, span-level faithfulness annotation, and clinician review on stratified samples. Because next-token generation samples from a learned distribution rather than consulting a fact store, the working assumption is that the rate can be reduced — via retrieval grounding, constrained decoding, abstention — but not driven to zero, so downstream verification is part of the design, not a stopgap.
Define a grounding corpus, measure span-level faithfulness against it with automatic entailment checks and clinician annotation, and design abstention and verification steps sized to the residual error rate.
e.g., a radiology-report summarizer is evaluated by having radiologists flag every generated finding absent from the source study; unsupported-finding rates above a preset threshold block release.
Anchors the position that fabrication is an irreducible property of sampling-based generation; see the dissensus record opposing teams that treat hallucination as an eliminable engineering defect for bounded tasks.
Ji et al. (2023), Survey of Hallucination in Natural Language Generation, ACM Computing Surveys 55(12)
intrinsic/extrinsic hallucination taxonomy; faithfulness vs. factuality
Ziwei Ji et al. (2022). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys
In clinical-AI quality and safety assurance, hallucination is an incident class in the safety-management system: fabricated clinical content is logged, graded for severity like a medication error, trended across deployments, and fed into post-market surveillance of the tool. What counts is defined procedurally — a report is a hallucination event if generated content presented as factual is contradicted by or absent from the authoritative record. The steward's artifacts are the event taxonomy, the reporting pathway, thresholds that trigger tool suspension, and the audit trail showing each event's disposition and corrective action.
Maintain a fabrication-event taxonomy and reporting pathway, trend event rates per deployment, and enforce predefined thresholds at which a generative tool is suspended pending investigation.
e.g., a hospital's AI oversight committee treats a cluster of three severity-graded fabrication events in an ambient-scribe pilot as a stopping rule, mirroring how medication-error clusters trigger review.
FDA, Artificial Intelligence/Machine Learning-Based Software as a Medical Device Action Plan (2021)
For analysts and client-facing staff using generative assistants, a hallucination is a plausible but unfounded output that could enter work product: an invented figure, a misstated regulatory provision, a citation to a filing that does not say what the summary claims. The working rule is that model output is unreconciled data — nothing goes into a client communication, research note, or regulatory submission until it is tied back to a golden source: the filing, the market-data feed, the rulebook. Fluency and confidence carry no evidential weight in this discipline; only reconciliation does, exactly as with an unconfirmed trade ticket.
Reconcile every figure, citation, and regulatory statement in generated output against its golden source before it enters client-facing or regulatory work product, and flag unverifiable content to supervisors.
e.g., an assistant drafting fund commentary attributes a 40 basis-point performance figure to a source document containing no such number; the analyst catches it only on reconciliation.
For executives accepting generative-AI risk, hallucination is a legal-exposure category: a fabricated statement made to a client or regulator through a firm system is the firm's statement. It is booked as operational and conduct risk — misstatements in advice, disclosures, or complaints handling — and enters the model-risk framework as a known failure mode with adverse consequences whether or not the model is right on average. Sign-off therefore means accepting a residual fabrication rate in a specific use, with named compensating controls and an owner, not endorsing a vendor's aggregate benchmark score.
Treat generated client-facing statements as firm statements: authorize each use case with an explicit residual fabrication-rate acceptance, named compensating controls, and clear accountability for outputs reaching clients or regulators.
In Moffatt v. Air Canada (2024), a Canadian tribunal held the airline liable for a bereavement-refund policy its chatbot invented — deployer liability that risk committees in regulated finance read as directly transferable.
Federal Reserve SR Letter 11-7, Supervisory Guidance on Model Risk Management (2011)
definition of model risk as adverse consequences from incorrect or misused model output
For engineering teams shipping generative features in banking, hallucination is a measurable defect class with a budget: the proportion of generated claims not attributable to the retrieval context or reference data, measured claim-by-claim on task-specific evaluation sets and enforced in CI before release. The design stance is that for bounded, retrieval-grounded tasks — summarizing a specific filing, answering from a policy corpus — fabrication is an engineering problem: tighten retrieval, constrain generation to cited spans, force abstention on low support, and the measured rate falls below the human-error baseline the process already tolerates.
Build claim-level attribution checks into evaluation and CI, set an explicit hallucination budget per use case, and block release when grounded-attribution rates fall below threshold.
e.g., a team shipping a policy-lookup assistant requires 99 percent of generated answers to carry verifiable citations into the policy corpus on a 1,000-case evaluation before each release.
Anchors the position that hallucination is a reducible engineering defect for bounded grounded tasks; see the dissensus record opposing validators who treat residual fabrication as permanent and controls-dependent.
mitigation methods: retrieval augmentation and faithfulness metrics
In independent model validation, hallucination is a standing limitation of generative models that validation cannot certify away: the validator verifies that the developer's fabrication-rate measurements are sound, that challenge testing probes the actual use case rather than a generic benchmark, and that documented compensating controls — human review gates, output attribution, monitoring — are commensurate with residual risk. Under model-risk guidance the presumption is that generative output can be incorrect while the model functions as designed, so effective challenge targets the control environment and the evidence behind claimed rates, never a promise of elimination.
Independently re-measure fabrication rates on use-case-specific challenge sets, assess whether compensating controls match the residual rate, and refuse validation sign-off where controls assume hallucination has been eliminated.
e.g., a validation team rejects a deployment memo claiming zero hallucination because the evidence was a public benchmark, and requires a challenge set built from the bank's own documents.
Anchors the position that residual fabrication must be presumed and controlled; see the dissensus record opposing engineering teams that budget hallucination toward zero for bounded tasks.
In newsroom practice, a hallucination is fabricated journalistic matter produced by a generative tool: a quote no one said, an event that did not occur, a source, study, or article that does not exist. It is handled under the existing verification discipline — every fact, name, quote, and citation in AI-assisted copy is checked as if supplied by an unvetted stringer. The tool's output has the status of a tip, not a source; publishing an unverified generated claim is a fabrication failure attributable to the journalist and the outlet, not to the software.
Treat generated text as an unvetted tip: independently verify every fact, quote, and citation before publication, and never source a claim to the model itself.
CNET issued corrections in 2023 after AI-drafted finance explainers were found to contain basic factual errors, prompting a public overhaul of its AI-assisted publishing process.
For publishers and editors-in-chief, hallucination is a liability and trust exposure attached to publishing decisions: fabricated content that reaches the audience triggers corrections, defamation exposure if it names real people, and disclosure obligations where AI-generated material must be labeled. The operative question at the desk is not the model's error rate but the control chain — which uses of generative tools are permitted, what verification is mandatory before publication, when AI assistance must be disclosed, and who signs off. A fabricated published claim is an editorial failure with the responsible editor's name on it.
Set and enforce an editorial AI policy specifying permitted uses, mandatory pre-publication verification, disclosure labeling, and named sign-off responsibility for any generative content that reaches the audience.
e.g., a magazine's policy requires AI-drafted service journalism to pass the same fact-check as freelance copy and to carry an AI-assistance note, with the section editor accountable for both.
For developers of narrative games, story tools, and concept-generation systems, hallucination largely names the product working: the model's capacity to produce people, places, and events unconstrained by fact is the creative engine, and suppressing it degrades the tool. The defect category is drawn differently — breaking established canon, contradicting the design bible, or leaking real-world claims into fiction presented as fact. These teams often reject the term outright for their context, operationalizing quality as coherence with the fictional world and its lore artifacts rather than correspondence with reality.
Specify the fictional canon and consistency constraints the generator must respect, evaluate outputs for world-coherence rather than factual grounding, and confine factuality checks to assets presented as real.
e.g., a studio's NPC dialogue generator is evaluated on lore consistency and tone; inventing a nonexistent town is desired behavior unless it contradicts the game's established map.
Anchors the position that unconstrained generation is a valued creative capacity rather than a defect; see the dissensus record opposing editorial-standards communities that treat any published fabrication as a breach.
Peiqi Sui et al. (2024). Confabulation: The Surprising Value of Large Language Model Hallucinations. Unknown Journal
For press councils, standards editors, and ethics ombudsmen, machine-fabricated content is assessed under the same clauses as human fabrication: accuracy and distortion provisions of editorial codes carve out no exception for tooling. A hallucinated quote in published copy is a fabrication breach; the finding names the outlet and the responsible editor, and the plea that the model made it up is not mitigation but an aggravating admission that verification was absent. The steward's standing concern is that the term hallucination itself softens what the code plainly calls fabrication, and adjudications deliberately use the code's word.
Adjudicate AI-fabricated published content under existing accuracy and fabrication clauses, attribute responsibility to the outlet and responsible editor, and reject tool involvement as a mitigating circumstance.
In 2023 Die Aktuelle published an AI-generated 'interview' with Michael Schumacher; the editor-in-chief was dismissed and the family compensated — adjudicated as fabrication attributable to the outlet, not a tool malfunction.
Anchors the position that machine fabrication in published matter is an editorial breach under existing codes; see the dissensus record opposing creative communities that value the same generative behavior.
In case handling, a hallucination is generated content that misstates the legal or factual basis of an administrative act: an invented provision, a repealed rule cited as current, an eligibility condition the statute does not contain, a fabricated precedent. Caseworkers using drafting or summarization assistants operationalize the check as source-tracing: every legal reference must be opened in the authoritative register and every case fact matched to the file before the draft touches a decision. An unverified generated citation in a decision letter is treated as the caseworker's own citation error, with the same disciplinary consequences.
Trace every generated legal reference to the authoritative register and every case fact to the file before use in a decision, and treat unverifiable generated content as absent.
In Mata v. Avianca (S.D.N.Y. 2023), lawyers were sanctioned for filing ChatGPT-fabricated case citations — the canonical warning now circulated in public-sector legal drafting guidance.
European Commission JRC, DigComp 2.2: The Digital Competence Framework for Citizens (2022)
Competence 1.2: evaluating data, information and digital content
For agency leadership, hallucination is a legality risk to the administrative act itself: a decision that relied on fabricated legal or factual content is defective — vulnerable to annulment, appeal, and maladministration findings — because reason-giving and due process require the stated basis to be real. Where generated content misstates facts about an identifiable person, the accuracy principle of data-protection law is engaged, with duties to rectify. Authorizing generative tools therefore means deciding which process steps may never rest on unverified output and writing that boundary into the agency's procedural rules and delegation orders.
Define which steps of the administrative process may use generative output and under what verification, and ensure decision files document the verified basis for every stated reason.
In 2024, noyb's complaint against OpenAI argued that ChatGPT's false statements about a real person's birth date breach the GDPR accuracy principle — the exposure agencies weigh before deploying assistants on citizen data.
Whether the GDPR accuracy principle applies to a generative model's probabilistic outputs, or only to stored and communicated personal data, is actively disputed among regulators, complainants, and providers.
For government digital-service teams, hallucination is a defect to be engineered out of scope before go-live: citizen-facing assistants are constrained to answer only from authoritative content — legislation registers, benefit rules, published guidance — with mandatory source links, refusal on out-of-corpus questions, and release gates on grounded-answer rates measured against a curated set of real citizen questions. On this stance, an assistant that invents an entitlement rule is misconfigured, not exhibiting an unavoidable property: bound the task, ground the answers, and fabrication for the covered domain approaches zero, with everything else routed to a human caseworker.
Constrain citizen-facing generation to an authoritative corpus with mandatory source citation and refusal behavior, and gate release on grounded-answer rates measured on a curated benchmark of real citizen questions.
New York City's MyCity business chatbot was found in 2024 telling employers that unlawful practices were permitted — exactly the failure mode this operationalization's grounding and refusal gates are built to exclude.
Anchors the position that hallucination is eliminable for bounded, authoritative-corpus tasks; see the dissensus record opposing communities that treat residual fabrication as intrinsic to generative decoding.
In official-statistics stewardship, hallucination is a breach of the traceability the quality framework presumes: every published figure must be derivable from documented sources and methods, so a generative system that produces a number, series, or attribution without a computable lineage has produced content that cannot enter official outputs at all. Stewards operationalize the boundary institutionally — generative tools may draft commentary or code, but any statistical claim must be regenerated from governed data through documented pipelines; plausible-but-unsourced is a category the code of practice has no lawful place for.
Certify that every statistical claim in a publication is regenerated from governed data via documented methods, and bar generative drafts from contributing figures that lack computable lineage.
e.g., a statistics office allows an LLM to draft a release's narrative but requires every number to be injected from validated production tables, never taken from the model.
European Statistics Code of Practice, revised edition 2017, Eurostat
Principles 7 and 12: sound methodology; accuracy and reliability
For teams building generative tools for news production — summarizers, headline generators, research assistants — hallucination is a hard faithfulness defect: any generated statement not attributable to the source article, wire copy, or verified database is a failure regardless of fluency. These builders operationalize it with claim-level attribution checks against source material, provenance links on every generated assertion, and evaluation sets drawn from the newsroom's own archive. The product requirement inherited from editorial standards is that the tool must never add information — only condense, transform, or retrieve it — which puts them at odds with sibling teams building fiction tools.
Enforce a no-new-information contract in generation: check every output claim for attribution to source material, surface provenance links, and fail builds where unattributable claims exceed the editorial threshold.
e.g., a wire service's summarization tool is evaluated by aligning each summary sentence with supporting sentences in the source dispatch; any unaligned factual claim counts as a blocking defect.
Holds the editorial-fidelity position within creative industries: fabrication is a defect even where sibling teams building fiction tools value the same generative behavior; see the dissensus record.
hallucination in abstractive summarization; faithfulness evaluation
For patients and caregivers consulting health chatbots, hallucination is confident misinformation received in a moment of vulnerability, with the verification burden falling on the person least equipped to carry it. What counts here is not a faithfulness metric but a power asymmetry: fluent, authoritative-sounding fabrications about symptoms, dosages, or entitlements are indistinguishable from sound advice without clinical knowledge the user by definition lacks, and harms concentrate on people with least access to a professional who could correct the record. Health-literacy work therefore frames the competency as calibrated distrust plus known escalation routes, not detection skill.
Treat chatbot health advice as unverified information: check dosages, symptoms, and entitlements against professional or official sources, and escalate to a clinician rather than acting on generated advice alone.
e.g., a symptom-checker chatbot confidently invents a drug-interaction warning, prompting a caregiver to stop a prescribed medication — the harm pattern patient-safety educators use in AI-literacy training.
For ombudsmen, courts, and internal auditors reviewing administrative conduct, hallucinated content in official documents is examined as a failure of the duty of care in reason-giving: the question is not how the text was produced but whether the authority verified the basis of its act. A fabricated citation or invented fact in a decision letter, tribunal filing, or report supports findings of maladministration and can vitiate the act; systematic reliance on unverified generation is an organizational finding about missing controls, escalating beyond the individual case to the authority's procedures themselves.
Examine whether authorities verified the stated basis of acts drafted with generative tools, record fabricated content as a reason-giving defect, and escalate systemic unverified use as an organizational control failure.
e.g., an ombudsman investigating a benefits refusal finds the decision letter cites a nonexistent regulation produced by a drafting assistant, and issues a maladministration finding with systemic recommendations.
In advertising and brand work, a hallucination is a generated claim about a product, price, or endorsement that no one substantiated: an invented statistic in body copy, a fabricated award, a testimonial nobody gave. Copywriters operationalize the check through the claim-substantiation discipline the industry already runs for regulators — every factual assertion in generated copy is routed through the same substantiation file as human-written claims, because ad-standards bodies and consumer-protection law attach liability to the advertiser regardless of how the claim was drafted or by what tool.
Route every factual claim in generated copy through the claim-substantiation process before release, and strip or verify statistics, awards, and endorsements the model introduced on its own.
Google's 2023 launch ad for Bard included the model's false claim that the James Webb Space Telescope took the first exoplanet image — an unsubstantiated generated claim published in marketing material.
In clinical-ethics and research-governance review, the term hallucination is itself scrutinized before the phenomenon is assessed: committees note that it borrows a psychiatric symptom to describe software behavior, misleadingly implies the system perceives and errs like a mind, can stigmatize patients who experience actual hallucinations, and rhetorically shifts fault from vendor design choices to a quasi-medical quirk. Review practice therefore records the phenomenon as fabricated or unsupported output, requires protocols and consent materials to describe it in those terms, and treats vendor language about occasional hallucinations as a disclosure to be unpacked, not accepted.
Require protocols, consent documents, and vendor materials to describe fabricated output in plain, non-psychiatric terms, and probe what design and deployment choices the hallucination framing may be deflecting.
NIST's 2024 Generative AI Profile pointedly names this risk 'confabulation' rather than hallucination, reflecting the same discomfort with the psychiatric metaphor that ethics committees raise in review.
Whether to retain 'hallucination', replace it with 'confabulation', or drop mental-state metaphors entirely divides standards bodies, clinicians, and disability advocates; all three usages are live in current healthcare governance documents.
Andrew L Smith et al. (2023). Hallucination or Confabulation? Neuroanatomy as metaphor in Large Language Models. PLOS Digital Health
In digital farm advisory practice, a hallucination is a generated recommendation that cannot be traced to an authoritative agronomic source: a chatbot stating a dosage for a plant-protection product, a fictitious national approval, or an invented subsidy deadline delivered with the fluency of extension advice. It is operationalized as a traceability failure — any generated claim about dosages, approvals, or obligations that cannot be matched to the product label, the national approval register, or official guidance is treated as fabricated and withheld, because farmers act on advisories within tight spray and sowing windows.
Trace every generated claim about dosages, approvals, deadlines, or obligations back to labels, approval registers, or official guidance before it reaches a farmer, and block whatever cannot be matched.
e.g., an advisory chatbot confidently recommends a fungicide and dose for a crop for which the product holds no national approval; a farmer following it would breach residue and cross-compliance rules.
Ji et al., Survey of Hallucination in Natural Language Generation (ACM Computing Surveys, 2023)
hallucination as fluent generated content unsupported by source or world knowledge
As intelligence organizations adopt large language models for summarization and drafting, hallucination denotes machine-generated content, such as a unit designation, a grid reference, or a quoted document, that is fluent, plausible, and unsupported by any underlying reporting. Tradecraft assimilates it to a familiar category: uncorroborated single-source reporting from a source of unknown reliability. The operational response is therefore not blanket prohibition but grading and corroboration, so that generated claims carry no evidentiary weight until traced to source reporting, plus hard bans in functions where a fabricated coordinate could enter a targeting or navigation chain.
Treat generated assertions as ungraded, uncorroborated reporting: trace each material claim back to source records before it enters an assessment, and bar generative output from targeting-critical data fields.
E.g., an LLM-drafted daily summary that invents a plausible battalion designation and assembly-area grid; if uncaught, the fabrication propagates into the common operational picture with an implied pedigree it never had.
Whether hallucination is a defect to be engineered out before fielding or an inherent property to be managed by tradecraft varies between acquisition and test authorities on one side and analytic-tradecraft cells on the other.
Ji et al., Survey of Hallucination in Natural Language Generation, ACM Computing Surveys (2023)
hallucination as fluent but unsupported generation
In classroom and academic practice, hallucination is confidently wrong teaching material: fluent chatbot output — an invented citation, a wrong theorem step, a fabricated historical detail — that a learner cannot yet recognize as unsupported precisely because they are still forming the knowledge needed to check it. Educators operationalize it through verification routines: requiring sources students must actually retrieve, treating unverifiable AI claims as unevidenced, and vetting AI tutoring content before release. The stakes are pedagogical, since novices absorb plausible falsehoods more readily than experts, and the resulting misconception may surface only later, in assessment.
Require learners to verify AI-generated claims against retrievable sources, vet AI tutoring content before classroom use, and design tasks in which unsupported statements earn no credit.
e.g., a student's literature review citing plausible-looking journal articles suggested by a chatbot that do not exist, discovered only when the supervisor tries to retrieve the references.
Educators split on whether hallucination is a product defect that vendors must engineer out of learning tools before adoption, or an inherent property of generative models that schools should exploit to teach source-checking and critical AI literacy.
Survey literature on hallucination in natural language generation
For maintenance and manufacturing-engineering teams adopting LLM assistants, a hallucination is generated content that reads like the manual but is not in the manual: an invented torque specification, a plausible but nonexistent part number, a repair sequence merging two machine variants. On a shop floor these are not chat quirks; a fabricated spec can defeat a torque-controlled joint or void a machine's certification. The operational response is engineering, not philosophy: retrieval locked to controlled documents, answers that cite the source revision, and a hard rule that uncited values never reach a work instruction.
Verify every AI-supplied specification, part number, or procedure against the controlled document of record before it enters a work instruction, and reject uncited outputs by default.
e.g., an assistant asked for a head-bolt torque on an older machine variant confidently returns the newer variant's value; applied unchecked, the joint is out of specification.
Ji et al. 2023, Survey of Hallucination in Natural Language Generation
characterization of hallucination as generated content unsupported by source material
In legal practice, a hallucination is fabricated authority or false assertion in work product — a citation to a case that does not exist, a quotation no court wrote, a misstated holding — operationalized as a sanctions trigger rather than a model property. Filing it breaches the duty of candor to the tribunal and the certification that filings are grounded in existing law; the lawyer's verification duty is non-delegable, so 'the tool generated it' is no defence. The controlling operation is checking every generated authority against the primary source before it enters any filing or advice.
Verify every citation, quotation, and factual assertion in AI-assisted work product against the primary source before filing or sending it, and treat unverifiable output as nonexistent.
In Mata v. Avianca (S.D.N.Y. 2023), lawyers were sanctioned under Rule 11 after filing a brief with nonexistent cases generated by ChatGPT and standing by them when challenged — fabricated citations adjudicated as attorney misconduct.
Whether hallucination is chiefly attorney misconduct (a failure of the non-delegable verification duty) or a product defect grounding claims against the tool's vendor divides professional-responsibility analysis from emerging product-liability and consumer-protection arguments.
Survey literature on hallucination in natural language generation (Ji et al., 2023, ACM Computing Surveys)
hallucination as fluent output unsupported by source or world knowledge
In logistics customer service and documentation work, hallucination is a generative assistant producing fluent but unsupported operational facts: a tracking chatbot inventing a delivery attempt that never happened, a drafting tool fabricating an HS code, incoterm, or customs requirement, or a summarizer asserting a departure time absent from the event stream. Because customers, brokers, and authorities act on these statements, teams treat any generated claim not traceable to a shipment record or tariff source as a defect that must be caught before it reaches the counterpart.
Constrain generative assistants to answer from shipment records and tariff data, require source references for every operational claim, and route unverifiable outputs to a human agent before release.
e.g., a customer-facing bot that tells a consignee their parcel was handed to a neighbour although the event stream shows no delivery attempt, generating a false claim dispute.
characterization of hallucination as fluent output unsupported by source or input data
Front-of-house teams and platform support desks meet hallucination as the confident wrong answer their chatbots give guests and workers: a booking assistant inventing a late-checkout policy, a care-line bot describing a medication schedule no nurse wrote, a host-support agent citing a cancellation-fee rule that does not exist. The output reads like policy but is unsupported by any system of record, and staff inherit the fallout — honouring the invented promise, correcting the guest, or documenting the incident so the operator can fix prompts and escalation rules.
Verify chatbot statements about prices, policies, and care instructions against the system of record before acting on them, and log invented answers for escalation and correction.
e.g., a hotel's AI concierge assures a guest that pets stay free, contradicting written policy; reception must decide on the spot whether to honour the bot's invented promise.
Survey literature on hallucination in natural language generation (Ji et al., ACM Computing Surveys, 2023)
characterization of fluent but unsupported generated content
In customer-facing commerce, hallucination is a generative system confidently asserting product facts, prices, policies, or endorsements that the catalog and terms of business do not support: a chatbot inventing a refund policy, auto-generated product descriptions adding materials the item does not contain, an AI-drafted email citing a promotion that never existed. Operationally it is caught — or not — by grounding copy in the product-information system and requiring human sign-off on claims, because every fabricated statement is simultaneously a customer-experience defect and a potential misleading-commercial-practice exposure.
Ground generative product copy and chatbot answers in the catalog and policy database, verify claims before publication, and treat any unsupported statement reaching customers as an incident, not a typo.
In Moffatt v. Air Canada (2024), a Canadian tribunal held the airline liable after its website chatbot invented a bereavement-fare refund policy that a customer relied on when booking.
Content and CX teams treat occasional fabrication as an editable quality defect within creative tolerance; legal and compliance teams treat any unsupported product or policy claim as actionable misrepresentation the moment it reaches a consumer.
Ji et al. (2023), Survey of Hallucination in Natural Language Generation, ACM Computing Surveys
definition of hallucination as fluent output unsupported by source or world knowledge
In research workflows, hallucination names generated content presented in scholarly form but supported by no source, most consequentially fabricated references, misattributed findings, and invented numerical results in literature reviews, code comments, and manuscript drafts. Groups operationalize it as a verification duty rather than a model property: every model-produced citation is resolved against a persistent identifier, every quoted finding is checked in the source itself, and generative assistance in drafting is disclosed under the venue's policy. Unverified generated content that reaches a submitted manuscript is handled as a research-integrity failure of the authors, not as a tool malfunction.
Verify every model-generated citation, quantity, and attribution against the primary source before it enters a manuscript, and disclose generative-tool use as the venue requires.
e.g., a draft containing a plausible-looking reference that resolves to no DOI and no such article; the failure is caught by resolving identifiers, not by the passage reading implausibly.
NLP researchers treat hallucination as an expected consequence of probabilistic generation and a measurable faithfulness property, while domain researchers and journal editors treat it as an avoidable defect and an authorship responsibility. The two framings place the burden of correction on different parties.
intrinsic/extrinsic hallucination taxonomy; faithfulness versus factuality
In generative-AI engineering, hallucination is a defect class: output asserted as factual that is unsupported by the model's inputs, retrieval context, or verifiable sources. Teams operationalize it through groundedness and factuality metrics in evaluation harnesses, retrieval augmentation with citation checks, and incident tracking when confabulated output ships. The rate is managed like any reliability metric - measured per release, budgeted per use case - even as researchers note that plausible-but-unsupported generation is intrinsic to how sampling from a language model works.
Define what counts as unsupported output for your product, measure hallucination rate against grounded sources in the evaluation suite, and gate releases and prompt changes on that budget.
Google's Bard demo (February 2023) asserted that the James Webb Space Telescope took the first picture of an exoplanet - a widely reported confabulation that became the canonical shipped-hallucination incident.
The live split is defect versus property: product teams treat hallucination as a bug to be driven toward zero, while many researchers treat it as inherent behavior of probabilistic generation to be bounded and disclosed, not eliminated.
Survey of hallucination in natural language generation (OmniGloss registry source)
hallucination as unsupported generated content; intrinsic vs. extrinsic framing