Hallucination in Generative AI: Challenges to data integrity,ethical concerns and implications for knowledge management

Research questions

The study is guided by the following research questions:

1. How do generative AI hallucinations affect data reliability and integrity within knowledge management environments?

Research objectives

1. To examine the effects of generative AI hallucinations on data reliability and integrity within knowledge management environments.

How Generative AI hallucinations affect data reliability

Generative AI, particularly Large Language Models (LLMs), holds immense potential for revolutionizing data-driven fields. However, its susceptibility to hallucinations, producing fabricated, inaccurate, or misleading outputs, raises significant concerns regarding data reliability. The dual-edged nature of LLMs, where their efficiency is counterbalanced by inaccuracies stemming from input uncertainty and lack of comprehensive problem-solving intelligence, is highlighted by Thorne ¹¹ This study indicates the necessity of verification and validation mechanisms to ensure AI-generated outputs uphold reliability and accuracy, emphasizing the interplay between human trust and system reliability. More so, Jesson et al. ¹² present a quantitative approach to estimating hallucination rates in Generative AI. Their findings emphasize the unpredictability of AI responses under certain conditions, offering a structured framework to measure and mitigate these issues. This method has implications for assessing the reliability of AI systems in both controlled and real-world environments.

The causes of hallucinations, pointing to factors such as incomplete training data and inherent biases, are explored by Patel ¹³ The study explores the broader implications of hallucinations, both as obstacles to trustworthy outputs and as potential catalysts for creativity. ¹³ Further emphasizes the importance of high-quality training data, transparency, and human oversight in minimizing the negative impacts of hallucinations, further advocating for diversified data sources to improve output reliability. Further, Kamel ¹⁴ examines the challenges faced by university communities due to AI hallucinations, noting that reliance on statistically probable outputs, rather than factual accuracy, can lead to misinformation. The mixed-methods approach employed in this study reveals the pitfalls researchers encounter when depending on AI-generated content. The findings of Kamel ¹⁴ highlight the need for integrating AI tool assessments into information literacy curricula to equip users with critical evaluation skills, thereby enhancing the reliability of academic work.

Sun et al. ¹⁵ offer a comprehensive classification of distorted information generated by AI. Through empirical content analysis, the study categorizes errors into eight primary types, such as logical errors and unfounded fabrications, further subdivided into 31 specific subtypes. This classification provides a foundation for identifying and addressing risks associated with distorted AI outputs, presenting actionable insights for developers and users to enhance the reliability of Generative AI systems.

Park and Lee ¹⁶ investigate trends in hallucination research across various fields, emphasizing advancements in detection and mitigation techniques. Their analysis highlights strategies like supervised fine-tuning and reinforcement learning with human feedback, which improve output reliability. By synthesizing insights from a wide range of studies, Park and Lee ¹⁶ contribute a technology-focused perspective that underscores the importance of interdisciplinary approaches to addressing AI hallucinations. Christensen, Hansen, and Wilson ¹⁷ explore the implications of hallucinations within consumer decision-making processes, particularly in the tourism industry. They identify the trust consumers place in AI systems despite erroneous outputs, which can lead to poor decisions. The study demonstrates the tangible risks of unreliable AI outputs in practical applications, reinforcing the necessity of improving data accuracy and consumer awareness to mitigate such effects.

Kim et al. ¹ further examine the impact of hallucinations in decision-making, showing how inaccurate AI-generated recommendations decrease user trust and satisfaction. Their studies emphasize the prominence and type of incorrect information as critical factors influencing user acceptance. Also, Nehra and Bansode ¹⁸ explore the integration of Generative AI in academic libraries, identifying potential benefits such as personalized information retrieval. However, they also highlight challenges like biases and ethical implications, which compromise data reliability. The study advocates for continuous monitoring of AI outputs and emphasizes the importance of managing data quality to harness AI’s full potential in academic environments.

Ethical dilemmas posed by Generative AI hallucinations

Generative AI, particularly large language models (LLMs), presents profound ethical dilemmas due to their ability to produce seemingly realistic but often fabricated or biased outputs. Hill ¹⁹ emphasizes the safety and efficacy challenges posed by these hallucinations, particularly when such models fabricate information, leading to significant ethical concerns. His research highlights the importance of using vector databases to reduce biases and align AI systems with human values. Further, Tortora ²⁰ explores the ethical challenges of Generative AI in forensic psychiatry and criminal justice, where decisions carry life-altering consequences. The author identifies risks such as data manipulation and biased outcomes, raising concerns about the potential misuse of AI in sensitive fields. Tortora ²⁰ calls for interdisciplinary collaboration and stringent evaluations of generative AI systems to prevent ethical breaches in high-stakes domains, underlining the need for informed and cautious adoption.

Harding et al. ²¹ propose a conceptual synthesis of ethical principles for Generative AI, identifying six core principles, including respect for intellectual property, truthfulness, and sociocultural responsibility. They introduce meta-principles to address conflicts among these ethics guidelines and emphasize continuous monitoring of evolving ethical concerns. Their study underscores the complexity of balancing multiple ethical principles while advocating for a structured and dynamic approach to ethical AI development. Laakso ²² critique the use of LLMs in moral psychology and behavioral research, highlighting their ethical implications. While LLMs can imitate human behavior, their limitations in context comprehension and reasoning raise concerns about their reliability in sensitive research areas. The authors caution against overreliance on AI in areas requiring nuanced human judgment, reinforcing the importance of ethical scrutiny in its applications. Kulkarni ²³ conducts a systematic literature review of ethical issues associated with LLMs, identifying recurring challenges such as biases, transparency, and accountability. His analysis aligns these issues with the European Commission’s guidelines for trustworthy AI, highlighting significant gaps in addressing these ethical dilemmas. Kulkarni ²³ concludes that while existing mitigation strategies exist, they are insufficient, particularly in tackling inherent biases and maintaining accountability in AI outputs.

Zhang et al. ²⁴ examines the ethical implications of AI-generated content, focusing on the risks of blurring lines between human and machine-generated outputs. This study highlights the potential for misinformation and intellectual property concerns while advocating for responsible AI deployment. Zhang et al. ²⁴ stresses the importance of addressing these ethical risks to prevent long-term harm to content authenticity and societal trust in AI systems. Ungless et al. ²⁵ discuss the challenges in assigning responsibility for Generative AI outputs, particularly in cases involving fabricated information or bias. They argue for proactive ethical frameworks and policy measures to guide the responsible development and deployment of LLMs. Their emphasis on anticipatory regulation highlights the importance of addressing ethical dilemmas before widespread adoption.

Lin ⁹ introduces the “Triple-Too” problem in ethical AI applications: abstract principles that lack contextual relevance and overemphasis on risks rather than benefits. He advocates for actionable strategies, including bias mitigation, transparency, and reproducibility, to bridge the gap between ethical guidelines and real-world applications. Also, Earley ²⁶ propose a practical guide for ethical research with LLMs, translating complex ethical considerations into actionable recommendations. Their “LLM Ethics Whitepaper” offers concrete guidelines for integrating ethics at every stage of AI development, emphasizing the importance of transparency and harm mitigation. Their work serves as a valuable resource for ensuring ethical integrity in LLM research and applications.

Implications of Generative AI hallucination for knowledge management systems

Generative AI, while a transformative tool for knowledge management (KM), presents significant challenges due to hallucinations—fabricated or inaccurate outputs that undermine the reliability of knowledge systems. Yang ²⁷ highlights these challenges, particularly the risk of exposing corporate intellectual property and the lack of traceability in AI outputs. This approach secures intellectual property, provides audit trails, and enhances the accuracy of KM systems, with experimental results demonstrating improved response precision. Further, Renukappa, Suresh, and Jallow ²⁸ emphasizes the role of AI in enhancing KM by streamlining processes and fostering organizational agility. However, successful adoption requires addressing cultural and human resource factors critical to aligning AI capabilities with organizational goals. Yang’s findings suggest that while AI redefines KM strategies and boosts efficiency, it also necessitates ongoing refinement to mitigate challenges such as hallucinations and overreliance on technology. Taherdoost and Madanchian ²⁹ discuss AI’s potential to improve KM processes in the construction industry, particularly by enabling more efficient document retrieval and knowledge sharing. Their research underscores the importance of integrating AI into KM systems to prevent recurring errors and enhance project outcomes. However, they caution against ignoring challenges like maintaining data integrity and addressing hallucinations, which could compromise KM processes. Riemer and Peter ³⁰ analyze how AI transforms KM, addressing the shortcomings of traditional systems, particularly in remote and hybrid working environments. They highlight AI’s role in managing organizational knowledge more effectively but caution against risks like inaccurate data propagation due to hallucinations. Their review emphasizes the need for continuous evaluation and adaptation of AI-driven KM systems to ensure reliability and efficacy. Alavi, Leidner, and Mousavi ³¹ conceptualize Generative AI as “style engines,” focusing on its novel ability to encode and reproduce patterns from training data. They argue that while these capabilities enhance creativity in KM, the probabilistic nature of AI introduces risks, particularly in reliability and accuracy. Their framework for integrating Generative AI into traditional KM systems calls for reconciling its limitations with its unique strengths to maximize its potential. Kudryavtsev, Khan, and Kauttonen ³² explore how Generative AI impacts the processes of knowledge creation, storage, transfer, and application. While AI enhances KM by fostering innovation and improving access to vast knowledge repositories, it also raises concerns about overreliance, ethical considerations, and the misapplication of AI-generated knowledge. They advocate for balancing AI capabilities with human insights to ensure effective and responsible KM practices.

Koponen ⁵ investigates the acceptance of Generative AI among knowledge workers, finding that trust and perceived usefulness significantly influence its adoption. However, hallucinations negatively impact trust, highlighting the importance of addressing these inaccuracies to maintain user confidence in AI-driven KM systems. The findings of Koponen ⁵ suggest that fostering positive attitudes toward AI requires transparency and effective mitigation strategies. Burtch, Lee, and Chen ⁴ examine the implications of Generative AI on online knowledge communities, noting declines in user participation on platforms like Stack Overflow due to the prevalence of hallucinations. These findings highlight the risks of decreased engagement and trust in KM systems reliant on AI, underscoring the need for rigorous validation and quality control mechanisms. Shah ³ discusses the transformative potential of Generative AI in information access while cautioning against challenges like hallucinations and information provenance. The author advocates for a multifaceted approach that combines technical advancements with policy changes to ensure fairness, transparency, and accountability in AI-driven KM systems.

Allan et al. ² examine the transformative influence of Generative AI on information retrieval (IR) systems, with particular focus on its integration into knowledge management frameworks. Their study synthesizes insights from a workshop that convened experts across academia, industry, and government to discuss the future trajectory of IR-GenAI systems. The report underscores key challenges, including issues of hallucination and data reliability, while highlighting opportunities for enhancing information accessibility. ² Equally provide actionable recommendations for stakeholders, emphasizing the need for collaborative efforts to harness Generative AI’s potential while mitigating its risks. Romero-Mariona et al. ³³ explore the use cases and challenges of integrating Generative AI into knowledge management processes. Their study identifies significant opportunities for AI to revolutionize the creation, capture, and access of organizational knowledge. However, they note that hallucinations pose a substantial risk, particularly in propagating inaccurate or fabricated information. To address these challenges, the authors propose developing reusable toolkits tailored for AI-enhanced knowledge management, fostering both innovation and reliability in organizational practices.

Alkaissi and McFarlane ³⁴ investigate the implications of Generative AI in knowledge transfer and trust-building within educational and organizational contexts. They highlight the utility of retrieval-augmented generation (RAG) models in mitigating hallucinations and enhancing decision-making transparency. Their research underscores the importance of incorporating robust information retrieval mechanisms to improve trust and efficacy in AI-driven systems. The role of Generative AI, particularly ChatGPT, in scientific writing and research contexts is examined by Alkaissi and McFarlane ³⁴ Their study documents the tool’s ability to generate insightful content while also exposing its limitations in producing hallucinated or misleading information. The authors argue that such inaccuracies can jeopardize the credibility of knowledge systems, particularly in critical fields like biomedical research. Perceptions of Generative AI’s impact on knowledge industries through participatory workshops across multiple sectors are explored by Woodruff et al. ³⁵ While participants recognized AI’s potential for automating routine tasks, they also identified risks such as deskilling, disconnection, and disinformation. The study highlights that while Generative AI can enhance productivity and streamline KM processes, the prevalence of hallucinations necessitates careful integration strategies to prevent the erosion of trust and skill development within industries.

Data sources and inclusion criteria

The researchers utilized academic databases, including Directory of Open Access Journals (DOAJs) and Google Scholar. DOAJ was selected because of its coverage of peer-reviewed open access scholarship relevant to emerging interdisciplinary AI research, while Google Scholar was used to capture broader scholarly outputs, including conference papers, preprints, and grey literature, important in a rapidly evolving field where foundational discussions often appear outside traditional indexed journals. The reliance on DOAJ and Google Scholar is recognized as a limitation and reflects the exploratory scope of the review. The search was conducted using specific keywords, such as “hallucination and data reliability,” “ethical dilemmas posed by hallucinations,” and “implications for knowledge management.” Boolean combinations such as (“AI hallucination” AND “data reliability”), (“generative AI” AND ethics), and (“large language models” AND knowledge management) were used. To maintain the study’s currency, only literature published between 2017 and 2025 was included (Table 2). Although searches initially covered 2015-2025, the analysis primarily concentrated on literature published from 2017 onward, but largely 2020 and beyond, reflecting the rapid emergence of large language models and intensified scholarship following the generative AI boom post-2022. Studies published before 2015 were excluded to ensure relevance to the rapidly evolving field of generative AI. The review focused on existing literature, including journal articles, preprints, theses, and conference proceedings that addressed challenges to data integrity, ethical concerns, and implications for knowledge management in the context of generative AI (Table 1). The inclusion and exclusion criteria are provided as follows:

OPEN IN VIEWER

Table 1.

Source types and percentages.

Source type	Frequency	Percentage (%)
Journal	18	52.94
Preprint	7	20.59
Thesis	3	8.82
Conference proceeding	6	17.65
​	34	100%

Exclusion criteria

• Studies focused solely on non-generative AI

• Duplicate records

• Opinion pieces lacking substantive analysis

• Articles not addressing hallucination-related risks

In all, 34 articles were arrived at. All included studies were published and publicly available to ensure credibility and accessibility. The article selection process is illustrated in a PRISMA flow diagram (Figure 1), showing records identified, screened, excluded, and retained for synthesis. Figure 1 presents the structured PRISMA-guided screening and selection procedure used in this review. It shows that 512 records were initially identified from DOAJ and Google Scholar, after which 93 duplicate records were removed. The remaining studies underwent title and abstract screening, leading to substantial exclusions based on relevance. Following retrieval and full-text assessment, only 34 studies satisfied the inclusion criteria. This demonstrates the rigorous filtering process applied to ensure that only credible and directly relevant literature addressing generative AI hallucination, data integrity, ethical concerns, and knowledge management implications formed the final evidence base for the study.OPEN IN VIEWER

Figure 1.

Prisma flow diagram (adapted from Haddaway et al. ³⁶ ) illustrating the procedure for Inclusion and Exclusion.

Table 1 shows the distribution of source types included in the review. The data indicate that journal articles constitute the largest share (52.94%), suggesting a strong reliance on peer-reviewed, formally published scholarship. Preprints account for 20.59%, reflecting the rapidly evolving nature of generative AI research, where emerging findings are often disseminated prior to formal publication. Conference proceedings (17.65%) also constitute a notable share, highlighting the contribution of scholarly meetings to advancing current discourse. In contrast, theses represent a smaller share (8.82%), indicating comparatively limited use of extended academic research outputs.

Table 2 presents the temporal distribution of the selected studies. The findings reveal a clear concentration of publications in recent years, with 2024 (52.94%) and 2023 (29.41%) dominating the dataset. This pattern underscores the surge in scholarly attention following recent advancements in generative AI. Earlier years contribute minimally, with only isolated studies between 2017 and 2022, and no studies were recorded for several intervening years. Overall, the distribution confirms that research on AI hallucination is both recent and rapidly expanding.

OPEN IN VIEWER

Table 2.

Articles selected by year.

Year	Frequency	Percentage (%)
2025	3	8.82
2024	18	52.94
2023	10	29.41
2022	1	2.94
2021	0	0.00
2020	1	2.94
2019	0	0.00
2018	0	0.00
2017	1	2.94
2016	0	0.00
2015	0	0.00
​	34	100%

Data analysis and synthesis

Thematic analysis was employed to identify key themes and patterns. The final corpus was coded into three principal themes: data reliability (n = 10 studies), ethical dilemmas (n = 12 studies), and knowledge management implications (n = 12 studies), with some studies contributing to multiple themes. The process involved iterative coding to extract and categorize findings related to challenges to data integrity, ethical concerns, and implications for knowledge management. A hybrid deductive-inductive coding approach was employed: initial codes were informed by the study’s research questions (deductive), while emergent subthemes were generated from the reviewed literature (inductive). Coding and thematic synthesis were conducted manually with repeated comparison across studies to ensure consistency and analytical rigor. The themes were then synthesized to draw meaningful insights and build the conceptual model for the study. The conceptual model was developed through cross-theme synthesis linking recurrent drivers, risks, and mitigation responses identified across the reviewed studies. A methodological limitation is the relatively small corpus (34 studies), reflecting the emerging nature of scholarship specifically addressing AI hallucination through a knowledge management lens.

Ethical considerations

The study adhered to all ethical guidelines for academic research. All sources were properly cited, and the authors of the reviewed studies were acknowledged in the reference list. No data manipulation or unauthorized use of information occurred during the research process.

Theme 1: How hallucinations affect data reliability

The studies in Table 3 present a comprehensive analysis of how generative AI hallucinations influence the reliability of data across various domains. A key recurring theme across multiple studies is the need for robust verification and validation frameworks to mitigate AI hallucinations. Thorne ¹¹ highlights that trust and reliability in AI outputs require well-defined validation processes, while Patel ¹³ emphasizes the role of high-quality training data and transparency in improving generative AI reliability. Jessen et al. ¹² introduce a novel approach using log probability metrics to estimate hallucination rates, providing a practical tool for managing AI-generated inaccuracies. Another prominent theme is the classification of hallucinations, as outlined by Sun et al. ¹⁵ who systematically identify different types of AI distortions, offering valuable insights for risk management. This aligns with Park and Lee, ¹⁶ who provide a literature review of hallucination research trends and mitigation strategies, such as reinforcement learning with human feedback (RLHF). In educational contexts, Kamel ¹⁴ discusses the challenges faced by students and faculty in distinguishing AI hallucinations from accurate information, advocating for the integration of AI literacy into academic curricula. While the reviewed studies largely converge on the need for verification mechanisms, they diverge in diagnosing the source of reliability failure. Some frame hallucination primarily as a technical problem solvable through model refinement, whereas others conceptualize it as an epistemic trust problem requiring human oversight and institutional safeguards. This distinction is significant because it shapes whether solutions emphasize algorithmic correction or socio-technical governance.

OPEN IN VIEWER

Table 3.

How Hallucinations affect Data Reliability.

Author and date	Article title	Context/Respondent	Purpose of the study/methods	Major findings/themes
Thorne, S. (2024)	Understanding the interplay between trust, reliability, and human factors in the age of generative AI	Generative AI systems in general	To examine challenges in trust, reliability, and human factors in generative AI outputs.	Highlights the need for verification and validation processes in using generative AI for reliable outputs.
Jesson, A. et al. (2024)	Estimating the hallucination rate of generative AI	Large language models (LLMs) in synthetic regression and natural language tasks	Develops a method to estimate hallucination rates using prediction questions and responses.	Introduces a novel approach for evaluating hallucination probability using log probability metrics in generative AI.
Patel, H. (2024)	Bane and boon of hallucinations in context of generative AI	Generative AI tools like ChatGPT	To understand AI hallucinations and propose strategies to mitigate negative impacts while maximizing creative potential.	Stresses the importance of high-quality training data, transparency, and human feedback to enhance generative AI.
Kamel, H. (2024)	Understanding the impact of AI hallucinations on the university community	Students and staff at New Giza University	To raise awareness about AI hallucinations and promote ethical, effective AI tool usage through a mixed-methods approach.	Highlights challenges in detecting AI hallucinations and advocates integrating AI tool evaluations in information literacy.
Sun, Y. et al. (2024)	AI hallucination: Towards a comprehensive classification of distorted information	Users of AI-generated content (243 cases analyzed)	Aims to systematically classify distorted information in AIGC and its internal characteristics.	Identifies 8 first-level error types and 31 second-level types to aid in detecting and managing AIGC risks.
Park, D.-M., and Lee, H.-J. (2024)	Literature review of AI hallucination research since the advent of ChatGPT	654 papers from arXiv on hallucination research	Explores trends in AI hallucination research in key fields like machine learning and computer vision.	Identifies trends like hallucination mitigation via SFT and RLHF, highlighting the importance of multimodal AI research.
Christensen, J. et al. (2025)	Understanding the role and impact of generative AI hallucination within consumers’ tourism decision-making processes	900 consumers in tourism decision-making	Investigates the interplay of AI hallucination awareness, TAM, and TPB models in tourism.	Many consumers trust AI tools over other sources, despite frequent hallucinations, due to perceived impartiality.
Kim, J. H. et al. (2023)	Do you trust ChatGPTs? Effects of the ethical and quality issues of generative AI on travel decisions	Travelers using ChatGPT	Studies the impact of ChatGPT’s ethical concerns on travelers’ acceptance and satisfaction.	Ethical concerns reduce acceptance; perceived trust mediates these effects.
Kim, J. H. et al. (2025)	When ChatGPT gives incorrect answers	Travelers in experimental setups	Examines the role of incorrect information and its prominence in tourism decision-making.	Incorrect information negatively impacts decision-making when prominent in a single domain but lessens in mixed domains.
Nehra, S. S., and Bansode, S. Y. (2024)	Exploring the prospects and perils of integrating artificial intelligence and ChatGPT in academic and research libraries	Academic and research library settings	Explores the potential and challenges of AI-powered chatbots like ChatGPT in libraries.	Highlights ethical concerns, potential biases, and the need for high-quality data management in libraries.

Consumer trust and perception of AI tools are also explored, with Christensen et al. ¹⁷ and Kim et al. ¹⁸ analyzing the tourism sector. Their findings reveal that while many travelers trust AI-generated content due to its perceived impartiality, hallucinations can negatively impact decision-making when the incorrect information is highly prominent. Meanwhile, Nehra and Bansode ¹⁹ explore the potential benefits and ethical challenges of AI-powered chatbots in academic libraries, emphasizing the need for rigorous oversight to ensure data reliability.

A notable gap across this body of literature is limited empirical evidence on comparative effectiveness of mitigation strategies across contexts, particularly in knowledge-intensive domains such as academia and decision-support systems. In summary, the evidence in the studies in Table 3 suggests a persistent tension between generative efficiency and epistemic reliability, positioning hallucination not merely as an error condition but as a structural challenge to trustworthy AI-enabled knowledge production.

Theme 2: Ethical dilemmas posed by hallucinations

The studies in Table 4 examine the ethical concerns associated with generative AI hallucinations, covering topics such as bias, misinformation, and ethical accountability. A key recurring theme is the inherent biases and ethical concerns within AI-generated content, with Hill ²⁰ and Ferrara ⁶ noting that hallucinations often introduce unintended biases that can reinforce societal stereotypes and misinformation. The studies argue for vector databases and bias-reducing methodologies to align AI-generated outputs with ethical standards. The need for interdisciplinary collaboration in ethical AI deployment is emphasized by Tortora ¹⁰ who discusses the ethical challenges of using generative AI in forensic psychiatry. The study highlights the critical importance of evaluating AI tools before deploying them in high-stakes environments, such as criminal justice and healthcare. Similarly, Laine et al. ¹⁰ introduce six ethical principles specific to generative AI, advocating for continuous monitoring and adaptive ethical frameworks. Although there is broad agreement that ethical governance is necessary, studies differ substantially in emphasis: some prioritize regulatory intervention and formal oversight, whereas others favor embedded ethical-by-design approaches or developer self-governance. This divergence signals an unresolved debate over whether hallucination should be governed primarily through technological safeguards, institutional regulation, or socio-ethical accountability frameworks.

OPEN IN VIEWER

Table 4.

Ethical Dilemmas posed by Hallucinations.

Author and date	Article title	Context/respondent	Purpose of the study/methods	Major findings/themes
Hill, M. (2023)	Hallucinating machines: Exploring the ethical implications of generative language models	Generative language models and their ethical implications	Explores ethical challenges of LLMs using critical making methodology through four experiments.	Hallucinations prevalent in LLMs; proposes use of vector databases for bias reduction and ethical alignment.
Tortora, L. (2024)	Beyond Discrimination: Generative AI Applications and Ethical Challenges in Forensic Psychiatry	Forensic psychiatry and criminal justice	Reviews the impact of GenAI in reshaping forensic psychiatry and its ethical and societal implications.	Advocates for responsible evaluations and interdisciplinary collaboration before deploying GenAI in high-stakes domains.
Laine, J., Minkkinen, M., and Mäntymäki, M. (2025)	Understanding the Ethics of Generative AI: Established and New Ethical Principles	Large language models and their ethical principles	Scoping review exploring established and emerging ethical principles in GenAI systems.	Introduces six GenAI ethics principles, emphasizing continuous monitoring and mapping contradictions among principles.
Harding, J. et al. (2024)	AI language models cannot replace human research participants	Scientific research using LLMs	Examines speculative uses of LLMs in scientific contexts and emphasizes their potential and limitations.	Highlights LLMs’ abilities and limitations in mimicking human behavior, calling for critical evaluation in research use.
Laakso, A. (2023)	Ethical challenges of large language models	Ethical issues in LLM applications	Systematic literature review identifying 39 ethical challenges of LLMs and mapping them to EU AI ethics guidelines.	Inherent biases and methodological issues remain significant challenges for trustworthy AI applications.
Kulkarni, C. S. (2022)	Ethical Implications of Large Language Models in Content Generation	Generative AI systems in content creation	Explores ethical dimensions like intellectual property and misinformation in LLM-generated content.	Highlights ethical concerns due to blurred lines between human and AI-generated content.
Zhang, J. et al. (2023)	Ethical considerations and policy implications for large language models	Policy and ethical frameworks for LLMs	Reviews ethical considerations in LLM development and deployment, emphasizing proactive policy measures.	Highlights the need for ethical frameworks to guide responsible LLM development and usage.
Ferrara, E. (2023)	Should ChatGPT be biased? Challenges and risks of bias in large language models	Biases in ChatGPT and other LLMs	Investigates origins of biases in LLMs and strategies to mitigate them.	Advocates for multidisciplinary collaboration to address biases and achieve ethical AI systems.
Lin, Z. (2024)	Building ethical guidelines for generative AI in scientific research	Ethical use of LLMs in scientific research	Proposes user-centered ethical guidelines for responsible LLM usage in research practices.	Highlights the importance of transparency, bias mitigation, and reproducibility in AI-assisted research.
Leidner, J. L., and Plachouras, V. (2017)	Ethical by design: Ethics best practices for natural language processing	Ethics in natural language processing research	Proposes best practices and ethical processes for NLP projects, emphasizing an ethics review board.	Ethical outcomes should align with ethical design principles from project inception.
Zhiheng, X. et al. (2023) 37	Safety and ethical concerns of large language models	Safety and ethical risks in LLM applications	Introduces key safety and ethical concerns, such as bias and robustness issues in LLMs.	Calls for enhanced safety measures and ethical considerations in deploying LLMs across applications.
Ungless, E. L. et al. (2024)	The Only Way is Ethics: A Guide to Ethical Research with Large Language Models	Ethical practices in NLP research	Develops a comprehensive guide to ethical research with LLMs, summarizing hundreds of studies.	Provides concrete recommendations for conducting ethical research and mitigating harms in LLM projects.

Several studies underscore the complexity of ethical oversight, with Zhang et al. ²⁴ identifying 39 ethical challenges mapped to EU AI ethics guidelines. Ungless et al. ²⁵ explores intellectual property rights and the ethical issues arising from AI’s ability to generate human-like content, warning of the potential for misinformation and authenticity challenges. Earley ²⁶ further stress the importance of proactive policy measures to guide ethical AI development, suggesting that companies and regulatory bodies must collaborate to establish effective guidelines. A contrasting perspective is provided by Laakso ²² who argue that while AI can simulate human responses in research contexts, it should not replace human participants entirely. Their study calls for a cautious approach to incorporating AI in sensitive research areas. Leidner and Plachouras ³⁸ propose ethical design principles for AI applications, advocating for ethics review boards to oversee AI implementation from inception to deployment. A recurring gap is that much of the literature proposes ethical principles normatively, but fewer studies empirically examine how such principles are operationalized in real-world AI deployments. Collectively, the evidence positions hallucination as both an ethical risk and a governance challenge, extending beyond bias or misinformation into questions of responsibility, legitimacy, and trust. Although there is consensus on the need for ethical frameworks, variations exist in the proposed approaches, ranging from regulatory oversight to self-imposed ethical guidelines by developers.

Theme 3: Implications for knowledge management

The studies in Table 5 explore the far-reaching implications of generative AI hallucinations on knowledge management (KM), highlighting both opportunities and challenges. A recurring theme is the transformative potential of AI in KM processes, with Renukappa, Suresh, and Jallow ²⁸ and Taherdoost and Madanchian ²⁹ demonstrating how AI can enhance organizational agility, efficiency, and knowledge sharing. Renukappa, Suresh, and Jallow ²⁸ particularly emphasizes the role of retrieval-augmented generation (RAG) in reducing hallucinations and improving knowledge security. However, the literature reveals a tension between techno-optimistic perspectives emphasizing augmentation benefits and more critical perspectives warning of deskilling, erosion of trust, and degradation of communal knowledge practices. Several studies highlight the integration of AI with existing KM systems, with Riemer and Peter ³⁰ finding that AI enhances document retrieval and learning from past projects in the construction industry. Alavi, Leidner, and Mousavi ³¹ explore AI’s role in bridging gaps caused by remote work, addressing challenges related to documentation and knowledge dissemination. However, concerns regarding the trustworthiness of AI-generated knowledge are prevalent. Shah ³ warns of the challenges posed by AI inaccuracies, emphasizing the need for accountability measures. Burtch et al. ⁴ analyze the impact of generative AI on online knowledge communities like Stack Overflow and Reddit, noting a decline in user engagement on platforms where AI-generated content is prevalent. Another important theme is the acceptance and adoption of AI in knowledge work, as explored by Koponen ⁵ who identifies key factors influencing AI adoption, including trust, performance expectancy, and social influence. Meanwhile,Woodruff et al. ³⁷ reveal concerns about deskilling, dehumanization, and misinformation, indicating that knowledge workers primarily view AI as a tool for menial tasks rather than a transformative force. An important gap concerns limited empirical evidence on long-term organizational consequences of integrating hallucination-prone systems into knowledge workflows, particularly in public-sector and academic contexts. The studies collectively suggest that while generative AI offers significant potential for enhancing KM, organizations must address challenges related to misinformation, user trust, and ethical considerations to ensure effective implementation. The evidence generally suggests that the core challenge is not whether generative AI belongs in knowledge management, but under what governance, validation, and human oversight conditions it can be trusted.

OPEN IN VIEWER

Table 5.

Implications for knowledge management.

Author and date	Article title	Contexts/respondents	Purpose of the study/methods	Major findings/themes
Earley, S. (2023)	What executives need to know about knowledge management, large language models and generative AI	Corporate executives and employees using LLMs	Discusses opportunities and risks of LLMs in knowledge management; evaluates retrieval-augmented generation (RAG)	RAG significantly improves accuracy and security of knowledge management, reducing hallucinations.
Yang, S. S. (2024)	The impact of artificial intelligence on knowledge management practices	Various organizations implementing AI for KM	Examines AI’s transformative role through literature review and empirical research	AI enhances organizational agility, efficiency, and competitive edge.
Renukappa, S. et al. (2020)	Knowledge management and artificial intelligence (AI)	UK construction industry managers	Qualitative research with semi-structured interviews to assess AI’s role in KM	AI improves project outcomes by integrating with KM systems, aiding document retrieval.
Taherdoost, H., and Madanchian, M. (2023)	Artificial intelligence and knowledge management: Impacts, benefits, and implementation	Various industries and businesses	Critical review of AI’s impact on KM processes	AI bridges gaps in remote work, enhances documentation, and introduces new challenges.
Riemer, K., and Peter, S. (2024)	Conceptualizing generative AI as style engines: Application archetypes and implications	Information systems and AI researchers	Conceptual analysis of generative AI as style engines	GenAI should be viewed beyond traditional computing expectations to leverage its creative potential.
Alavi, M. et al. (2024)	Knowledge Management Perspective of Generative AI (GenAI)	Organizations using GenAI for knowledge processes	Analyzes GenAI’s role in knowledge processes	Highlights GenAI’s potential and risks in knowledge creation, storage, and transfer.
Koponen, K. (2024)	Acceptance of Generative AI in Knowledge Work	Finnish knowledge workers (students, professionals)	Survey-based quantitative study using UTAUT model	Attitude is the strongest determinant of AI acceptance.
Burtch, G. et al. (2024)	The consequences of generative AI for online knowledge communities	Stack Overflow and Reddit users	Analyzes the impact of LLMs on user engagement	Stack Overflow saw a decline in activity due to GenAI, while Reddit remained unaffected.
Shah, C. (2023)	Generative AI and the future of information access	AI users across various sectors	Examines AI’s impact on information retrieval	AI transforms how users seek and access information, with challenges in accuracy and bias.
Kudryavtsev, D. et al. (2024)	Transforming Knowledge Management Using Generative AI: From Theory to Practice	Companies implementing GenAI for knowledge work	Identifies and prioritizes AI use cases for KM	Proposes a toolkit to address identified use cases and challenges.
Alkaissi, H., and McFarlane, S. I. (2023)	Artificial hallucinations in ChatGPT: implications in scientific writing	Scientific and medical writing fields	Case studies examining ChatGPT’s performance	Highlights potential benefits and risks of AI in academic writing.
Woodruff et al. (2024)	How knowledge workers think generative ai will (not) transform their industries	Knowledge workers across seven industries in the US	Participatory research workshops with 54 participants across three cities	Participants view AI as a tool for menial tasks under human review; concerns about deskilling, dehumanization, disconnection, and disinformation.

Across the three themes, a central pattern emerges: hallucination challenges cannot be treated separately as reliability, ethics, or knowledge management issues, but as interconnected dimensions of trustworthy AI governance.

Implications for policy

The study’s findings highlight the urgent need for policymakers to establish comprehensive regulatory frameworks that address the challenges associated with generative AI hallucinations. Policies should focus on ensuring transparency, accountability, and ethical use of AI by mandating regular audits, quality assurance processes, and clear reporting mechanisms for AI-generated content. Regulatory bodies should also develop standardized guidelines for AI developers and organizations to follow, ensuring compliance with ethical principles and minimizing the risks associated with misinformation and bias. Additionally, anticipatory regulatory measures should be implemented to proactively address potential future risks associated with the rapid evolution of AI technologies. These findings suggest policy responses should move beyond compliance-oriented regulation toward adaptive AI governance models capable of addressing evolving hallucination risks through continuous oversight, dynamic standards, and multi-stakeholder accountability. Policies should also clarify roles and responsibilities among AI developers, service providers, and end-users to foster accountability and ethical compliance across all sectors.

Implications for practice

From a practical perspective, organizations leveraging generative AI technologies must implement robust mechanisms to ensure the reliability and accuracy of AI-generated content. This includes adopting advanced verification tools, such as RAG models, that integrate AI outputs with trusted data sources to reduce hallucinations. Importantly, the findings indicate mitigation should not rely solely on technical fixes such as RAG, but should integrate organizational governance, human review, and AI literacy interventions, reflecting the socio-technical nature of hallucination risks. Regular auditing and monitoring of AI performance should be prioritized to detect inaccuracies and ensure compliance with established standards. Training and upskilling employees in AI literacy is also essential to enable them to critically assess AI-generated information and make informed decisions. Additionally, organizations should implement clear guidelines on the ethical use of AI, incorporating principles of fairness, transparency, and bias mitigation into their operational frameworks. Effective collaboration between AI developers, information managers, and policymakers is crucial in developing AI systems that align with organizational goals while ensuring ethical and responsible AI adoption. In the field of knowledge management, practitioners should adopt a hybrid approach that combines AI capabilities with human oversight to ensure the quality and relevance of information. This reinforces that trustworthy mitigation depends on layered governance rather than technical controls alone. AI tools should be integrated in a way that complements existing knowledge processes rather than replacing human expertise. Organizations should also invest in AI-assisted decision support systems that provide users with verified and contextually relevant information, reducing the risk of misinformation. Furthermore, fostering a culture of continuous learning and adaptation is vital to keep up with the evolving capabilities of AI, ensuring that knowledge management systems remain resilient and effective in the face of AI-driven changes.