Abstract
Experts face increasing cognitive demands due to the volume of scientific artifacts, databases, frontier models, and evolving hypotheses across domains. Traditional AI systems often prioritize model performance over user integration, limiting their effectiveness in real-world workflows. This paper presents BioSage, a Compound AI (CAI) platform designed through a workflow-centric, human-centered approach. Guided by principles from human factors, cognitive science, and industrial-organizational psychology, BioSage incorporates multiple collaborative agents—retrieval, reasoning, and translator—each aligned to a specific user need in the scientific discovery process. The system supports key workflows such as knowledge retrieval and synthesis, debate, brainstorming, and experimental design. Through initial user testing with domain experts and cross-disciplinary benchmarks, BioSage demonstrates how user-aligned CAI architecture can reduce cognitive load, improve insight generation, and build trust through transparency and perspective-taking. This work underscores the value of integrating human-centered design early in CAI development to enhance adoption and impact in high-stakes environments.
Keywords
Introduction
In many scientific and technical domains (e.g., biomedical research, materials science, data science and AI), experts confront an information overload problem: thousands of new scientific artifacts, data sets, and evolving hypotheses inundate their workflows. The sheer volume and complexity of such diverse multimodal information can overwhelm even highly skilled teams, hindering timely knowledge synthesis and innovative discoveries (Endsley, 2017).
The demand for effective and rapid knowledge discovery and synthesis has driven the rise of agentic AI solutions (Gao et al., 2024; Lu et al., 2024; Schmidgall et al., 2025; Swanson et al., 2024). However, AI systems are often developed primarily from a technology-first perspective, focusing on model accuracy, computational efficiency, or algorithmic novelty (Lála et al., 2023; Shneiderman, 2022). While these technical advances are crucial, they can fail to address human needs when using AI which ultimately determine whether a system integrates seamlessly into real-world practices and operational workflows. Factors such as trust, transparency, and user workflow alignment have a decisive impact on how readily experts adopt and benefit from AI (Amershi et al., 2019; McDuff et al., 2025; Tu et al., 2025).
Within the human factors and ergonomics community, researchers have advocated for user-centered design principles that emphasize function allocation, trust calibration, and cognitive workload management (Endsley, 2017; Feigh & Pritchett, 2018). By starting with the end-user’s tasks, environment, and mental models, we can develop multi-agent compound AI systems (Volkova et al., 2024; Zaharia et al., 2024) that not only perform accurately but also align with human needs—resulting in improved decision-making, reduced error rates, and greater acceptance (Bansal et al., 2021; Liao et al., 2020).
In this paper, we present our workflow-centric approach to designing a multi-agent compound AI platform aimed at supporting complex scientific discovery tasks across scientific disciplines (Boiko et al.. 2023, Bran et al., 2024). Crucially, we did not begin by building the “best performing” generative multi-agent system. Instead, our diverse team—comprising industrial/organizational (I/O) psychologists, cognitive and HF/E specialists, as well as data science and AI experts—first examined the use cases most critical to scientific workflows and then mapped these needs onto an agentic AI architecture. Our guiding question was: How can a multi-agent system be structured so that each agent directly addresses a user-centered workflow requirement? Initial testing and brainstorming of our proof-of-concept multi-agent system have shown promise and have uncovered best practices for improving AI utility in high-stakes knowledge work (Lála et al., 2024).
Background
Human–AI Interaction and Workflow Design
Human–AI interaction (HAI) research underscores the importance of considering how users integrate AI outputs into their cognitive processes, decision-making, and everyday tasks (Amershi et al., 2019; Endsley, 2017). For instance, if an AI system cannot provide transparent results, users may become either over-reliant (trusting it blindly) or under-reliant (ignoring it altogether). These trust calibration challenges point to the need for transparent interfaces, user override options, and iterative feedback loops (Liao et al., 2020; McDuff et al., 2025; Tu et al., 2025).
Workflow design is another core dimension in HAI. A well-designed AI system should meld with existing team processes rather than impose disruptive new demands. This synergy is especially crucial in AI for scientific discovery, where interdisciplinary teams must frequently brainstorm, critically evaluate findings, and synthesize large volumes of literature. By mapping user tasks and pain points (e.g., literature overload, confusion about conflicting data) onto specific AI capabilities (e.g., summarization, research debate, multi-modal retrieval etc.), developers can avoid the pitfalls of tech-first solutions that fail to resonate with the end users. In this way agentic frameworks, or multi-agent systems purpose-built for specific tasks, can be integrated into workflows, known as agentic workflows.
Multi-Agent AI Systems and Human Factors
Multi-agent AI systems distribute complex tasks across automated “agents,” enhancing scalability and task coverage (Feigh & Pritchett, 2018). In principle, each agent can excel at a distinct function—such as retrieving data, generating hypotheses, or critiquing evidence—rather than having one monolithic model attempt to do it all. However, multi-agent designs often prioritize agent–agent communication protocols and autonomous coordination while neglecting the human at the center of the workflow. Drawing on HF/E principles, we reframe multi-agent systems so that every agent’s role and interaction style maps onto a user-identified need. For instance, if researchers need help sifting through new research artifacts, the system can incorporate specialized “summarize” focused prompting of querying, reasoning, and response synthesis (Figure 1). If teams struggle to challenge each other’s assumptions, “compare” focused prompting of the same agents can highlight conflicting findings. These agentic workflow roles serve as a blueprint for bridging the technical capabilities of AI with the practicalities of scientific workflows.
BioSage agentic workflows.
Approach
Agentic AI Workflow Design
Our approach centered on collaborative sessions that included: (1) I/O Psychologists, who provided insights into team processes, collaboration norms, and how the AI could enhance group dynamics; (2) HF/E and Cognitive Experts, who conducted task analyses, identified cognitive load factors, and recommended interface elements to promote transparency and user control; (3) AI Developers and Data Engineers who translated these human-centered specifications into multi-agent architectures, ensuring each AI agent aligns with a distinct user needs (Feigh & Pritchett, 2018).
We then switched to use case definition and brainstorming. Specifically, we defined a broad scientific discovery use case—focusing on tasks like research paper review, ideation, and conflict resolution (e.g., when contradictory hypotheses and claims appear). Through brainstorming and pilot scenarios, we identified the most pressing pain points (e.g., “No time to review new papers thoroughly,” “Hard to track contradictory evidence,” “We need creative prompts for experiment design”). Such user-centered analysis directly informed the system architecture. Rather than building a single, all-purpose AI model, we ensured the compound AI system supported the following specialized human-AI workflows, specifically (a) summarization workflow that focuses on cross-disciplinary knowledge retrieval and synthesis, filters and consolidates scientific artifacts e.g., papers, reports relevant to a given query; (b) debating workflow that surfaces contradictory or critical viewpoints, prompting the human user to reflect on potential flaws or alternative interpretations, and (c) brainstorming workflow that generates novel research ideas and hypotheses, sparking creative discussions in the team (Figure 2).
Example of BioSage system interaction with multimodal agents, LLM and VLM models and tools to enable cross-disciplinary knowledge discovery and synthesis and advance scientific discovery.
Agent Development
BioSage takes a thoughtful approach to agent development that aligns specialized AI agents with user-centered workflow requirements. The system incorporates multiple specialized agents that collaborate to facilitate knowledge discovery and synthesis; retrieval agents with self-reflection capabilities to find and synthesize relevant knowledge, reasoning agents that support transparency in knowledge discovery and synthesis, and translator agents that help bridge knowledge across diverse scientific domains (e.g., biochemistry, and AI). The agents are designed to operate within a compound AI system with LLMs, VLMs and Retrieval Augmented Generation (RAG), allowing them to process diverse scientific content including text, images, tables, and structured data. Importantly, the agent development process focuses on supporting real scientific workflows through features like cross-disciplinary brainstorming, research debate etc. Each agent was designed to follow a rigorous workflow to achieve a relevant and robust answer via a multi-turn dialog unlike current SOTA (Lála et al., 2023) to the original researcher query.
Retrieval Agent
BioSage retrieval agent focuses on retrieving and synthetizing relevant multi-modal knowledge from scientific information. The process begins with query planning where the agent extracts key words and develops a list of concepts and relevant domains to query for. Next the agent iterates through each domain to extract multimodal evidence and perspectives utilizing generalized background knowledge of each selected research domain. Finally, the retrieval agent synthesizes the relevant knowledge from each domain into a multi-faceted answer.
Reasoning Agent
BioSage reasoning agent utilizes concepts from cognitive science to ensure that answers are grounded in human-like reasoning, combining macro- and micro-reasoning processes. Macro-reasoning focuses on the high-level structure of thought, where cognitive processes integrate broad, complex information to generate overarching insights. This is operationalized through a structured multi-step insight extraction pipeline—beginning with labeling the contextual perspective in relation to a query, constructing and decomposing thought processes, and culminating in comparative analysis, convergence scoring, and expert identification. In parallel, micro-reasoning refines and optimizes individual cognitive steps, ensuring that each part of the reasoning chain remains contextually aligned, and logically coherent. This is enhanced by the incorporation of second thought—a metacognitive process that revisits initial judgments to revise and reaffirm insights based on self-reflection, what-if analysis, and perspective taking. By dynamically integrating both macro- and micro-reasoning with second thought, the system mimics the layered, adaptive, and reflective qualities of expert human cognition, enabling the generation of high-quality, interpretable insights.
Translation Agent
BioSage translation agent takes relevant out of domain language and translates it into language from the user’s domain. The sophisticated prompting utilizes input from in-domain probing to understand the user’s language and then performs a cross-domain gap assessment to assess jargon differences and analogous concepts between the domains. In the end, the translator agent transforms the original response to remove or explain out-of-domain jargon while utilizing in-domain concepts to explain complex out-of-domain concepts. Through this sophisticated cross-domain translation process, BioSage not only bridges communication gaps between fields but also democratizes access to complex knowledge by making it actionable for users within their domain-specific frameworks.
Agentic Workflow Testing and Iterative Refinement
To validate BioSage agents’ performance, we developed a cross-disciplinary benchmark to assess how well BioSage can integrate multiple domain’s perspectives coupled with collaborative feedback from domain experts. In these sessions, we observe how agent’s outputs integrate into real or simulated research tasks, noting gaps in usability, transparency, or appropriate understanding of the user’s needs. The team then refines both the front-end interface (e.g., better explanation features), the back-end architecture (e.g., improved query planning and synthesis), and agent integration control and prompting (e.g., selecting the right agents and prompting based on the desired behavior).
Outcomes
While rigorous quantitative and qualitative performance evaluation are still in development, our iterative agentic workflow methodology has already generated valuable preliminary insights and established emerging best practices that show promise for wider implementation. Testing thus far has included SME and researcher testing under various conditions to stress the different workflows and examine agent and human-agent team-level performance. While we have been able to develop testing and results of quantitative benchmarks for agent/model performance, this testing of human-agent collaboration, workflow, and team performance has been largely subjective and qualitative thus far, yet we have been able to generate meaningful insights for future research and development efforts. More rigorous validation and benchmarking is forth-coming.
Integration Over Innovation: By aligning the multi-agent system with specific user tasks, we found that even relatively straightforward AI components can provide meaningful help if well-placed in the workflow (Shneiderman, 2022).
Reduced Cognitive Barriers: summarization and debate workflows help users quickly contextualize new findings or identify potential blind spots, reducing the mental overhead of sifting through endless reading lists (Liao et al., 2020). Furthermore, having the agent prompt back to the user relevant questions to investigate what was favorable for the researcher will provide next steps with minimal cognitive load.
Trust and Feedback Loops: initial feedback from AI engineers and domain scientists underscores that “explain” or “show sources” features are essential for building user confidence and supporting healthy skepticism (Bansal et al., 2021). For example, the retrieval agent uses perspective-taking across relevant domains to present evidence and viewpoints specific to each field. This helps researchers understand how different disciplines contribute to the final response.
Conclusion
This paper outlines a collaborative, interdisciplinary method for designing compound AI solutions that directly address human workflow needs. By beginning with the use case and HAI interaction requirements—rather than diving straight into algorithmic complexity—we developed a compound AI system architecture where each agent’s function naturally supports a key step in scientific discovery. For the HFES community, our work highlights the value of integrating I/O psychology, cognitive ergonomics, and engineering in early-stage AI development. The goal is to empower experts—not to replace them—by embedding AI insights into daily research tasks in a trustworthy, transparent, and user-driven manner (Endsley, 2017).
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Defense Advanced Research Projects Agency, Biological Technologies Office (DARPA BTO; contract number: HR00112530112). The content presented in this manuscript does not necessarily reflect any official position of DARPA or the U.S. Department of Defense.
