Assessing the efficacy of LLMs in neuroscience: A performance analysis of ChatGPT and gemini on synaptic plasticity

Abstract

Background

Synaptic plasticity, which plays a critical role in fundamental neurological processes, is a complex subject to master. Therefore, large language models (LLMs) are increasingly being used to facilitate the learning of such complex topics. However, these models have limitations, including producing inaccurate information and failing to capture the nuances of scientific terminology.

Objectives

This study aimed to evaluate the accuracy, quality and readability of LLM responses to questions on synaptic plasticity.

Methods

The widely used LLMs ChatGPT-4 and Gemini 2.5 were selected in the study. Ten questions were posed to each LLM, and the initial responses were recorded. Five neurophysiologists evaluated the responses qualitatively using a 4-point Likert scale. Readability level of the answers was analyzed using Flesch-Kincaid Grade Level test.

Results

In the qualitative assessment, both models generally provided accurate and acceptable information. Within the limited scope of the questions analyzed, Gemini received higher median scores in certain instances; however, no statistically significant difference was observed between the two models across most of the question set. Linguistic analysis showed that Gemini's responses were longer and featured a higher Flesch-Kincaid Grade Level, suggesting a structure more aligned with academic or technical discourse.

Conclusion

For the specific neuroscientific inquiries examined in this study, both LLMs demonstrated a high capacity for generating accurate content. While Gemini's responses exhibited a more technical linguistic profile, the findings are context-specific and further research is needed to determine if these trends persist across broader scientific domains and larger datasets.

Keywords

large language models generative artificial intelligence synaptic plasticity learning physiology education

Introduction

Synaptic plasticity is the ability to change the strength and effectiveness of connections between neurons in the brain by creating new connections between neurons and strengthening or weakening existing connections.^1,2 Synaptic plasticity is a multidisciplinary field and a complex, multifaceted topic that researchers from many different scientific disciplines need to learn about. For instance, occupational therapists rely on the principles of synaptic plasticity in the development of post-stroke cognitive rehabilitation protocols. Similarly, the design of advanced medical devices and point-of-care diagnostic systems by biomedical engineers requires a detailed understanding of these neurophysiological mechanisms. Due to its multidisciplinary nature, mastering the multifaceted mechanisms of synaptic plasticity is often a challenging task requiring significant cognitive work for researchers and students from various scientific backgrounds.¹ Therefore, researchers and students who are new to this subject use different resources to learn synaptic plasticity and its underlying mechanisms.^1–3 On the other hand, for medical and neuroscience educators, developing effective pedagogical strategies to teach these intricate concepts is a core professional responsibility. In recent years, Large Language Models (LLMs) have emerged as powerful tools for simplifying and learning such complex scientific topics.^3,4 Large language models are frequently preferred in the health field because they provide fast access to information, summarize information and facilitate the understanding of abstract concepts using analogies.^5,6 These models, which have the potential to process complex medical information^6,7 support diagnostic processes^8,9 and even provide continuing education opportunities for health professionals^10,11 are also playing a transformative role in educational practices. Particularly in modern educational approaches like flipped learning, where the work of educators is shifting from traditional lecturing to curating preparatory materials and facilitating active learning, LLMs can significantly reduce the workload of educators. Simultaneously, by serving as accessible, 24/7 personalized tutoring tools, these models strongly support students in their primary occupational task of learning.

However, large language models sometimes exhibit critical limitations, most notably “AI hallucination” a phenomenon where models generate factually incorrect or fabricated information with high confidence.^12,13 Furthermore, they may struggle to capture the nuances in scientific terminology and face limitations in making an in-depth interpretation.¹⁴ Therefore, recent studies are being conducted to evaluate the answers of large language models in the health field. Cakir et al. aimed to examine the accuracy of ChatGPT's responses to female urology questions,¹⁵ Du et al., to ascertain how ChatGPT-3.5, ChatGPT-4.0, and Microsoft Copilot answered obstetric ultrasound questions and their performance in analyzing obstetric ultrasound reports,¹⁶ Anvari et al., to determine the ability of large language models to answer hepatology questions and compare the accuracy and quality of responses provided by different large language models,¹⁷ and Watane et al., to compare ChatGPT's responses to frequently asked questions about the upper eyelid blepharoplasty procedure with those of an oculofacial plastic surgeon.¹⁸ Many similar studies can be found in literature examining the performance of large language models for different diseases and terms.^10,17,19,20 Despite the growing body of literature on LLMs in healthcare, there is a lack of comparative analysis specifically focused on complex neuroscientific mechanisms. While LLMs are not intended to replace rigorous peer-reviewed literature for primary research, their increasing role as supplementary tools for conceptual mapping and academic synthesis necessitates a formal evaluation of their accuracy and linguistic complexity within specialized domains like synaptic plasticity.

Therefore, the present study aims to evaluate the performance of different LLMs in explaining synaptic plasticity for academic learners, including medical students and neuroscience researchers. To achieve this goal, our evaluation procedures focused on three specific aspects: (1) the scientific accuracy and quality of the responses, assessed via expert qualitative evaluations; (2) the structural readability of the generated texts, measured using the Flesch-Kincaid Grade Level; and (3) the quantitative linguistic characteristics of the outputs, such as word and sentence counts.

Methods

Model selection and configuration

This study was designed as a comparative evaluation of large language model performance in the context of neuroscience education, using an exploratory analytical framework to generate preliminary evidence without a predefined directional hypothesis regarding model superiority. The study primarily targets academic learners, including medical students and neuroscience researchers, who require high-fidelity technical explanations. The methodological choices were guided by the aim of ensuring standardized input conditions, reproducibility, and expert-informed assessment of LLM responses rather than the development or validation of a psychometric measurement tool. Current artificial language models and the functions offered by these models were examined. Within this framework, the study assessed the ChatGPT-4 Turbo platform, developed by OpenAI, and the Gemini 2.5 Flash platform, developed by Google, as they represent leading and widely used large language models. All interactions were conducted on May 18, 2025. Default system settings were used for both models, and no manual adjustments were made to sampling parameters such as temperature or nucleus sampling. No API-based access or fine-tuning was employed.

Standardized querying environment

To ensure impartiality and minimize contextual bias, new user accounts were created for each platform, and all processes were conducted on a personal computer utilizing a browser with a clear history. The publicly available free web-based versions of ChatGPT and Gemini were used throughout the study, with no subscription-based or premium features enabled. Privacy and session controls were strictly maintained by enabling “temporary chat” mode in ChatGPT and disabling session and activity history on Gemini. All questions were typed manually, with each question entered in a new, independent conversation session to prevent prior chat history from affecting subsequent answers or triggering the models’ self-learning mechanisms.

Question creation and answer compilation process

On May 18, 2025, an initial pool of questions related to synaptic plasticity was generated using a LLM prompt designed to elicit commonly structured conceptual questions in this domain. For this purpose, the question ‘Can you list the most frequently asked questions about synaptic plasticity worldwide?’ was asked to each LLM and the initial answers received from LLM were recorded and a comprehensive question pool was created based on these answers. Ten representative questions on synaptic plasticity were generated using a large language model and curated by the research team based on conceptual relevance and educational value. The questions were designed to elicit explanatory responses on core mechanisms rather than to reflect empirically determined real-world frequency. The questions generated were reviewed by the research team (the authors and 4 physiologists) to remove redundant, ambiguous, or out-of-scope items.^7,21 From this refined pool, ten representative and conceptually relevant questions were selected by a physiologist with expertise in neuroscience based on their clarity, educational relevance, and coverage of core synaptic plasticity concepts. This expert-curated approach ensures that the stimulus set functions as a rigorous benchmark for testing explanatory adequacy, rather than a mere collection of random inquiries. These questions were not intended to represent empirically validated real-world frequency but to serve as a standardized and reproducible stimulus set for comparative LLM evaluation. Finally, these ten questions, shown in Table 1, were presented to ChatGPT-4 Turbo and Gemini 2.5 Flash separately according to the standardized protocol described above. The initial responses were recorded exactly as generated, without any requests for clarification, manual editing, or post-generation formatting to preserve the integrity of the models’ raw output.

Table 1.

Research questions.

1	What is synaptic plasticity?
2	How do presynaptic and postsynaptic changes contribute to synaptic plasticity?
3	Why is synaptic plasticity considered crucial for learning and memory?
4	What happens to learning and memory when synaptic plasticity is impaired?
5	What is long-term potentiation (LTP)?
6	What is long-term depression (LTD)?
7	What are the main molecular mechanisms underlying LTP and LTD?
8	How is synaptic plasticity studied in neuroscience?
9	How does the timing of synaptic activity influence the direction and magnitude of plasticity (e.g., Hebbian learning)?
10	Can interventions that modulate synaptic plasticity be used to treat memory disorders?

Evaluation of LLM responses

To comprehensively assess the quality of responses generated by artificial intelligence, a rigorous evaluation process was undertaken. In this study, response ‘quality’ was operationally defined as the accuracy, comprehensiveness, and educational adequacy of the explanations, as evaluated by expert neurophysiologists. Conversely, ‘readability’ was operationally defined as the objective structural complexity of the generated text, quantified using the Flesch-Kincaid Grade Level metric. The evaluation process was designed to reflect the standards of high-attainment groups, utilizing expert assessments to ensure scientific accuracy and automated readability analyses to objectively measure educational depth. This process contains both qualitative analyses, which delve into the content, accuracy, and appropriateness of the responses through expert opinion, and readability analyses, which objectively determine the structural complexity and fluency of the text. Together, these methods aim to provide a holistic understanding of the overall effectiveness and quality of the generated responses.

The qualitative evaluation of the responses was conducted by an expert committee made up of five physiologists. To minimize any potential scoring bias during the evaluation, the outputs from ChatGPT and Gemini were anonymized before being submitted for assessment. For qualitative assessment, a 4-point Likert-type scoring system developed by Mika et al.²¹ was used. This scale was selected to enable structured expert judgment across predefined quality dimensions while avoiding neutral responses, thereby encouraging decisive evaluations. Likert-type scales are widely used in expert-based content evaluation studies due to their interpretability and suitability for aggregating qualitative judgments. This system allowed for a structured rating of answers, as detailed in Table 2.

Table 2.

Method used for evaluating responses.

Score	Category	Description
1	Excellent response, no additional explanation needed	The response is highly accurate and comprehensive, requiring no further clarification.
2	Satisfactory, requires minimal explanation	The response is correct, but minimal additional explanation is needed to fully address the user's question.
3	Satisfactory, requires moderate explanation	The response is accurate, but a moderate level of additional explanation is needed to fully meet the user's needs.
4	Unsatisfactory, requires substantial explanation	The response contains significant inaccuracies or is overly generalized, potentially misleading the user.

In parallel, the readability level of the LLM responses was evaluated using the WordCalc online readability test (https://www.wordcalc.com/readability/). For each question, character count, word count, sentence count, average syllables per word, and average words per sentence were calculated using the WordCalc and supplementary manual verification in Microsoft Word. Readability was assessed using the Flesch–Kincaid Grade Level, a widely used readability metric that estimates the educational grade level required to comprehend a text based on sentence length and word complexity. Higher grade levels indicate increased textual complexity and lower readability. This metric was chosen due to its extensive use in biomedical and educational research and its suitability for objective comparison of readability across LLM responses. To minimize any potential scoring bias during the evaluation, the outputs from ChatGPT and Gemini were anonymized before being submitted for assessment. The evaluation framework combined expert-based qualitative assessment and objective readability analysis to provide a holistic evaluation of response quality, balancing scientific accuracy, explanatory adequacy, and structural readability.

Expert panel characteristics and instructions

Qualitative evaluation was conducted by an independent committee of five physiologists (3 male, 2 female) with an age range of 37 to 46 years. All evaluators held advanced academic degrees (PhD) in physiology or neuroscience and held academic positions ranging from Assistant Professor to Associate Professor.

Prior to the evaluation, the experts received standardized written instructions. They were informed of the study's overall objective but were completely blinded to the source of the generated responses (ChatGPT vs. Gemini). Evaluators were instructed to independently read each anonymized response and score its scientific accuracy, comprehensiveness, and educational adequacy using the provided 4-point Likert-type scale. They were strictly requested not to discuss their ratings with one another during the evaluation process to ensure completely independent judgment.

Statistical analyses

Statistical analyses were performed using SPSS (IBM Corp., Armonk, NY, USA). Qualitative scores obtained from the 4-point Likert-type scale were treated as ordinal data, and comparisons between ChatGPT and Gemini responses were conducted using the Mann–Whitney U test. For the quantitative linguistic features and readability scores, mean and standard deviation values were computed across the ten responses generated by each model. Comparisons between Gemini and ChatGPT were performed using the Mann–Whitney U test due to the non-normal distribution and small sample size. P-values of 1.000 in the results reflect tied ranks and limited variability characteristic of small-sample, non-parametric data. Inter-rater reliability and test–retest analyses were not performed, as the study was designed as an exploratory expert-based comparison rather than a psychometric validation study. A p-value of <0.05 was considered statistically significant for all tests.

Results

Qualitative analyses

The assessment of LLM responses was based on expert qualitative judgments using a 4-point Likert-type scoring system, where 1 indicated the highest quality and 4 the lowest. These ordinal ratings were subsequently subjected to quantitative statistical analysis. The median scores and interquartile ranges (IQR) for both models for each question, along with the results of the Mann–Whitney U test, are presented in Table 3.

Table 3.

Comparison of expert quality ratings of the LLM responses (n = 5 expert raters).

No	Quality of answer
	ChatGPT		Gemini
	Median	IQR	Median	IQR	z	p
1	2.00	1.50	2.00	1.00	−1.247	0.212
2	1.00	1.00	1.00	0.50	−0.655	0.513
3	2.00	0.50	1.00	1.00	−2.032	0.042
4	2.00	1.50	1.00	1.00	−0.808	0.419
5	2.00	0.50	2.00	1.00	−1.678	0.093
6	2.00	0.50	2.00	1.00	−1.678	0.093
7	2.00	0.50	1.00	1.00	−1.225	0.221
8	2.00	1.00	2.00	1.00	0	1.000
9	2.00	1.50	1.00	1.00	−0.808	0.419
10	2.00	1.00	1.00	1.00	−0.600	0.549

IQR: interquartile range, Z: Mann Whitney U Test, p < 0.05.

The results indicated that both ChatGPT and Gemini provided generally accurate and acceptable responses on synaptic plasticity, with most median scores centering around 2.00. While Gemini achieved a median score of 1.00 (“best quality”) for five specific questions (Q3, Q4, Q7, Q9, Q10) compared to ChatGPT's one (Q2), a statistically significant difference was observed only for Question 3 (p = 0.042). For the remaining nine questions, no statistically significant differences were found (p > 0.05), indicating that the perceived response quality was generally comparable between the two models across most items.

Readability and linguistic analyses

The readability of the responses was assessed using the Flesch-Kincaid Grade Level (FKGL) metric, with lower scores indicating easier readability. The FKGL results for each LLM across ten questions are summarized in Table 4 and further visualized in Figure 1. As shown in Table 4, the mean FKGL for Gemini's responses was 20.26 ± 1.93 (median: 20.27), while ChatGPT's responses exhibited a lower mean of 17.73 ± 2.24 (median: 17.69). Gemini's responses yielded a statistically significant higher mean grade level compared to ChatGPT (z = 2.343, p = 0.021).

Figure 1.

Flesch-Kincaid grade level scores per question for gemini and ChatGPT responses.

Table 4.

Flesch-Kincaid grade level scores for LLM responses.

Model	Mean ± SD	Median	Range (Min-Max)	z	p
Gemini	20.26 ± 1.93	20.27	17.36–23.63	2.343	0.021
ChatGPT	17.73 ± 2.24	17.69	14.74–20.81	2.343	0.021

The linguistic features of the LLM responses from Gemini and ChatGPT were quantitatively analyzed, and the results are presented in Table 5 and further visualized in Figure 2. Gemini produced responses with higher character counts (4335.9 ± 1241.53 vs. 2228.7 ± 1170.19; p = 0.006), word counts (513.5 ± 130.34 vs. 268 ± 129.69; p = 0.003), and sentence counts (31.6 ± 14.31 vs. 15.7 ± 6.93; p = 0.002). Gemini's responses also exhibited a statistically significant greater average number of syllables per word (2.43 ± 0.16 vs. 2.2 ± 0.13; p = 0.031). Although Gemini's responses tended to have slightly more words per sentence (18.79 ± 2.53) than ChatGPT's (16.65 ± 4.2), this difference did not reach statistical significance (z = 1.550, p = 0.121).

Figure 2.

Comparative linguistic features per question for LLM responses.

Table 5.

Comparison of mean linguistic features of LLM responses.

Feature	Gemini (Mean ± SD)	ChatGPT (Mean ± SD)	z	p
Character Count	4335.9 ± 1241.53	2228.7 ± 1170.19	2.759	0.006
Word Count	5135 ± 130.34	268 ± 129.69	2.949	0.003
Sentence Count	31.6 ± 1.31	15.7 ± 6.93	3.071	0.002
Average Syllables per Word	2.43 ± 0.16	2.27 ± 0.13	2.158	0.031
Average Words per Sentence	18.79 ± 2.53	16.65 ± 4.2	1.550	0.121

Discussion

The rapid development of artificial intelligence-supported large language models (LLMs) has increased their use by academic candidates seeking information. The ability of LLMs to simplify complex topics, support rapid literature searches, and provide multilingual support suggests that their use will grow. However, when specific and detailed information is required, such as synaptic plasticity, LLMs can mislead users due to fabricated (incorrect) information superficial answers, and a lack of references¹³

Our findings indicate that both ChatGPT and Gemini provide answers with high security profiles to frequently asked questions about synaptic plasticity. Although both LLMs provided accurate answers to factual questions, Gemini's responses were consistently rated higher by our expert panel in terms of comprehensiveness and structural coherence, particularly for interpretative questions. The higher Flesch-Kincaid Grade Levels of Gemini's responses indicate that the answers provided by Gemini can be better understood by individuals with higher levels of education. Conversely, it is seen that the answers produced by ChatGPT are simpler in terms of word length and complexity. Our linguistic analyses further support this, revealing that ChatGPT's outputs were quantitatively more readable and potentially accessible to a broader audience. Conversely, the higher degree of lexical complexity observed for Gemini suggests a more technical and academic linguistic profile. These patterns align with the observed differences in FKGL scores, where higher word counts and syllable density in Gemini's outputs correlate with increased quantitative reading difficulty. While these features suggest a more detailed narrative specifically for the domain of synaptic plasticity, they do not imply a generalized superiority in communication, as sentence-level structure (words per sentence) remained statistically comparable between the two models. A study comparing ChatGPT and Google Bard also found that ChatGPT had lower Flesch-Kincaid Grade Levels, which aligns with our findings.⁷ However, another study comparing rheumatology board-level questions found that ChatGPT had a higher percentage of correct answers than Gemini.²² While that study involved 420 questions, our study was limited to 10 questions and compared not only answer accuracy but also answer quality, understandability, and the educational level addressed. In a different study testing questions related to postmenopausal osteoporosis, it was observed that ChatGPT 4.0 gave more accurate answers than Gemini.²³ These conflicting findings across studies can be explained by examining four underlying factors. First, differences in research question types: studies using closed-ended, board-style questions, test factual recall, whereas our questions on Hebbian learning and learning disorders required interpretative reasoning. Second, differences in evaluation criteria: most studies measured only accuracy, while we additionally assessed comprehensiveness, structural coherence, and readability. Third, differences in model versions: earlier versions of ChatGPT (GPT-3.5) vs. GPT-4, or Gemini Pro vs. Ultra, are known to produce different performance profiles; our study used [GPT-4 Turbo / Gemini 2.5 Flash]. Fourth, the epistemological nature of the domain: synaptic plasticity, as a basic neuroscience topic, demands conceptual explanation more than clinical guidelines, an area where Gemini demonstrated relative strengths in our evaluation. When the accuracy rates of the answers given to the questions about specific topics are compared, the studies showing that the accuracy rates of ChatGPT are higher than Gemini^22,24–26 seem to be the majority. Viewed alongside these conflicting results, our findings add a further nuance: in our targeted evaluation of synaptic plasticity which prioritized answer quality and educational value alongside accuracy Gemini demonstrated a consistent tendency to provide more detailed and structured explanations. This highlights that model performance is not merely a function of factual correctness but may vary significantly depending on the evaluation and the specific epistemological demands of the subject matter. It is important to remember that our study was limited to a single subject within the medical field. Comparisons of accuracy and comprehensibility between ChatGPT and Gemini may differ across research topics, including engineering, law, coding, and social sciences.^14,16,27

In our study, while answers to basic information questions like “What is LTP?” were similar, the answers to questions that required interpretation such as the effect of Hebbian learning (a fundamental theory stating that synapses are strengthened when pre- and postsynaptic neurons are coactive, often summarized as ‘cells that fire together, wire together’²⁸ on plasticity) on plasticity and the relationship between plasticity and learning disorders were presented more comprehensively by Gemini. AI hallucination, a critical limitation where models generate factually incorrect information with high confidence, poses a significant risk to the validity of AI-assisted research.²⁹ Notably, we did not encounter the phenomenon known as AI hallucination in either LLM within our limited question set. This absence, while encouraging, should be interpreted cautiously given the narrow scope of our evaluation. Nonetheless, it suggests that for well-defined scientific queries within established domains like synaptic plasticity, current LLMs can produce reliable outputs, potentially aiding researchers in exploratory or educational contexts.

In conclusion, our focused comparative analysis within the domain of synaptic plasticity revealed a nuanced performance difference. The expert evaluations indicated a tendency for Gemini's responses to be rated as more comprehensive and structurally detailed, particularly for interpretative questions. This suggests that for complex, niche scientific topics, LLMs may exhibit differential strengths in explanatory depth. Future studies should expand on this by investigating a wider range of capabilities, such as summarization accuracy at various knowledge levels, performance on factually critical information, and adaptability to different query scenarios, to more clearly delineate the specific strengths and optimal use cases of each model in academic research.

Limitations

This study represents a novel comparison of the quality and readability of responses from two advanced LLMs, ChatGPT-4 and Gemini, concerning synaptic plasticity. However, our study has several limitations. The analysis focused on the 10 most frequently asked questions about synaptic plasticity posed to ChatGPT-4 and Gemini-2.5 A larger, more diverse set of questions could yield different results. Second, the responses are limited by the datasets on which these LLMs were trained. Furthermore, as LLM technology is rapidly evolving and new versions are constantly released, the findings of this study represent a snapshot in time. Consequently, our conclusions are specific to the model versions we examined and may not hold true for future updates. Also, there are several limitations that should be considered when interpreting the findings. The question pool was generated using an LLM prompt requesting commonly asked questions on synaptic plasticity. While this approach ensured standardization and reproducibility across models, it does not represent a formally validated set of real-world frequently asked questions and may be influenced by model-specific biases or hallucinations. Therefore, the results should be interpreted within the context of LLM educational questioning rather than as a direct reflection of learner or clinician information needs. Future studies should construct question sets using literature-based frameworks, expert consensus methods, or learner-derived surveys to enhance ecological validity. In addition, the qualitative evaluation relied on a small panel of expert evaluators and did not include formal inter-rater reliability or test–retest stability analyses. The use of ordinal Likert-type data combined with a limited number of evaluators restricts statistical power and limits the interpretability of inferential statistics. Accordingly, all statistical analyses should be considered exploratory, and the findings reflect aggregated expert judgment and comparative performance trends rather than definitive model superiority. Readability was assessed solely using the Flesch–Kincaid Grade Level, which may not fully capture the linguistic complexity of scientific texts. While this metric enables standardized comparison across LLM responses, future studies should incorporate additional readability and linguistic measures for a more comprehensive evaluation. The absence of multiple comparison correction due to the exploratory design constitutes a methodological limitation. Lastly, within the limited scope of this research our findings suggest that Gemini's responses were more comprehensive while ChatGPT's were simpler in language. However, since we did not directly test any user group, we cannot claim that either model is ‘more suitable’ for any specific educational level.Readability was assessed solely using the Flesch–Kincaid Grade Level, which may not fully capture the linguistic complexity of scientific texts The effect of this on different educational levels remains a subject for future investigation.^30,31 Furthermore future studies should directly compare how learners across different educational backgrounds actually perceive and benefit from each model's response style.

Footnotes

ORCID iDs

Melek Altunkaya

Ercan Babur

Emine Cihan

Cansu Sahbaz Pirincci

Ethical considerations

This work did not involve research using human participants, their data, or biological material. As such, Consent to Participate/Consent to Publish are not applicable.

Author contributions

Each author contributed to the editorial conception, preparation, read, and provides their approval of the final manuscript.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

LLM use

During the preparation of this study, the authors used ChatGPT-4 and Gemini-2.5 to identify questions and receive responses to the questions asked. After using this tool/service, the authors revised the content as needed, but the responses were not edited as the purpose of this study was to evaluate the effectiveness of ChatGPT-4 and Gemini-2.5 in the first place, as described in the paper.

Supplemental material

Supplemental material for this article is available online.

References

Zhang

Ben Zablah

Zhang

, et al. Rho signaling in synaptic plasticity, memory, and brain disorders. Front Cell Dev Biol 2021; 9: 729076.

Appelbaum

Shenasa

Stolz

, et al. Synaptic plasticity and mental health: methods, challenges and opportunities. Neuropsychopharmacology 2023; 48: 113–120.

Harris

Littleton

. Transmission, development, and plasticity of synapses. Genetics 2015; 201: 345–375.

Bernstein

Zhang

Govil

, et al. Comparison of ophthalmologist and large language model chatbot responses to online patient eye care questions. JAMA Netw Open 2023; 6: e2330320.

Thirunavukarasu

Ting

DSJ

Elangovan

, et al. Large language models in medicine. Nat Med 2023; 29: 1930–1940.

Lee

. The rise of ChatGPT: exploring its potential in medical education. Anat Sci Educ 2024; 17: 926–931.

Azzopardi

Logeswaran

, et al. Artificial intelligence chatbots as sources of patient education material for cataract surgery: ChatGPT-4 versus Google Bard. BMJ Open Ophthalmol 2024; 9: e001824.

Eneva

Dogan

. Evaluation of medical diagnosis capabilities of three artificial intelligence models – ChatGPT-3.5, Google Gemini, Microsoft Copilot: sustainable development goals (SDGs). J Lifestyle SDGs Rev 2025; 5: e03545.

Ferdush

Begum

Hossain

. ChatGPT and clinical decision support: scope, application, and limitations. Ann Biomed Eng 2024; 52: 1119–1124.

10.

Simon

Gelikman

Turkbey

. Evaluating the efficacy of artificial intelligence chatbots in urological health: insights for urologists on patient interactions with large language models. Transl Androl Urol 2024; 13: 879–883.

11.

Tan

DNH

Tham

Koh

, et al. Evaluating chatbot responses to patient questions in the field of glaucoma. Front Med (Lausanne) 2024; 11: 1359073.

12.

Lee

Frieske

, et al. Survey of hallucination in natural language generation. ACM Comput Surv 2023; 55: 1–38.

13.

Alkaissi

McFarlane

. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus 2023; 15: e35179.

14.

Rahman

Mahir

Tashrif

MTA

, et al. Comparative analysis based on deepseek, chatgpt, and google gemini: Features, techniques, performance, future prospects. Systems and Soft Computing 2025; 7: 200396.

15.

Cakir

Caglar

Halis

, et al. Assessing the knowledge of ChatGPT in answering questions regarding female urology. Urol J 2024; 21: 410–414.

16.

Akolekar

Jhamnani

Kumar

, et al. The role of generative AI tools in shaping mechanical engineering education from an undergraduate perspective. Sci Rep 2025; 15: 9214.

17.

Anvari

Lee

Jin

, et al. Artificial intelligence in hepatology: a comparative analysis of ChatGPT-4, bing, and bard at answering clinical questions. J Can Assoc Gastroenterol 2025; 8: 58–62.

18.

Watane

Perzia

Weiss

, et al. ChatGPT and frequently asked patient questions for upper eyelid blepharoplasty surgery. Orbit 2024; 44: 295–298.

19.

Sahin Ozdemir

Ozdemir

. Comparison of the performances between ChatGPT and gemini in answering questions on viral hepatitis. Sci Rep 2025; 15: 1712.

20.

Zhou

Pan

Zhang

, et al. Evaluating AI-generated patient education materials for spinal surgeries: comparative analysis of readability and DISCERN quality across ChatGPT and deepseek models. Int J Med Inform 2025; 198: 105871.

21.

Mika

Martin

Engstrom

, et al. Assessing ChatGPT responses to common patient questions regarding total hip arthroplasty. J Bone Joint Surg A 2023; 105: 1519–1526.

22.

Abdul Sami

Abdul Samad

Parekh

, et al. Comparative accuracy of ChatGPT 4.0 and google gemini in answering pediatric radiology text-based questions. Cureus 2024; 16: e70897.

23.

Liu

Yang

, et al. Comparative analysis of ChatGPT-4o mini, ChatGPT-4o and gemini advanced in the treatment of postmenopausal osteoporosis. BMC Musculoskelet Disord 2025; 26: 369.

24.

Strzalkowski

Strzalkowska

Chhablani

, et al. Evaluation of the accuracy and readability of ChatGPT-4 and google gemini in providing information on retinal detachment: a multicenter expert comparative study. Int J Retina Vitreous 2024; 10: 61.

25.

Fattah

Salih

, et al. Comparative analysis of ChatGPT and gemini (bard) in medical inquiry: a scoping review. Front Digit Health 2025; 7: 1482712.

26.

Mete

. Evaluating the performance of ChatGPT, gemini, and bing compared with resident surgeons in the otorhinolaryngology in-service training examination. Turk Arch Otorhinolaryngol 2024; 62: 48–57.

27.

Rossettini

Rodeghiero

Corradi

, et al. Comparative accuracy of ChatGPT-4, microsoft copilot and google gemini in the Italian entrance test for healthcare sciences degrees: a cross-sectional study. BMC Med Educ 2024; 24: 694.

28.

Hebb

. The organization of behavior: A neuropsychological theory. Psychology press, 2005.

29.

Lee

Frieske

, et al. Survey of hallucination in natural language generation. ACM Comput Surv 2023; 55: 1–38.

30.

Merç

Pirinççi

CŞ

Cihan

EJH

, et al. Evaluation of AI chatbots for patient education and information on chronic obstructive pulmonary disease. Heart Lung 2026; 75: 21–25.

31.

Solnyshkina

Zamaletdinov

Gorodetskaya

, et al. Evaluating text complexity and Flesch-Kincaid grade level. JSSER 2017; 8: 238–248.