A recent study led by researchers from Harvard and Beth Israel Deaconess Medical Center found that OpenAI’s o1 model outperformed physicians in various emergency room triage and clinical management scenarios. The research involved six experiments where the AI model was tested against hundreds of doctors, revealing significant accuracy differences—67% for the AI during triage compared to 55% and 50% for the doctors. While the findings highlight AI’s potential in handling complex medical data and diagnosing more accurately, researchers caution that the study’s reliance on text inputs limits its applicability, as real-world clinical decision-making involves non-text data. They assert that AI should complement, rather than replace, human physicians, particularly as a second opinion to improve diagnostic accuracy.

OpenAI: OpenAI is an AI research organization developing advanced reasoning models like the o1 series, which emphasize step-by-step chain-of-thought processing for complex problem-solving. In a recent Harvard-led study published in Science, its o1 model outperformed physicians in emergency room triage, diagnosis, and clinical management tasks using real-world patient data.
Dr. Wei Xing: Dr. Wei Xing is a lecturer in the University of Sheffield’s school of mathematical and physical sciences, specializing in AI applications. He critiqued the Harvard o1 study by warning that it does not confirm AI safety for routine clinical use and highlighted risks of doctors over-relying on AI outputs.
Dr. Adam Rodman: Dr. Adam Rodman is an internist at Beth Israel Deaconess Medical Center and Harvard Medical School faculty member who leads AI integration into the medical curriculum and directs the AI program at the Shapiro Center. He emphasized in the recent Harvard study that the o1 model effectively processes real-world emergency department data for diagnosis.
Dr. Assaf Caspi: Dr. Assaf Caspi is deputy director of Sheba Medical Center’s psychiatric division and co-founder of Mentaily, an Israeli startup developing AI for mental health. His work on the recently approved LIV system for psychiatric triage aligns with the Harvard study’s vision of AI empowering clinicians as an aid.
Dr. David Reich: Dr. David Reich is the chief clinical officer of the Mount Sinai hospital system, overseeing clinical innovations including AI. He described the Harvard o1 study as a call to action for thoughtfully integrating AI into workflows to enhance patient care.
Prof. Arjun Manrai: Prof. Arjun Manrai is an associate professor of biomedical informatics at Harvard Medical School and head of an AI lab focused on healthcare applications. As a lead author of the recent Science publication, he noted the o1 model’s eclipse of physician baselines across diagnostic benchmarks and called for clinical trials before routine adoption.
Harvard Medical School: Harvard Medical School is a premier institution advancing medical education, research, and AI integration into healthcare practices. Researchers from the school, in collaboration with Beth Israel Deaconess and Stanford, conducted a recent study demonstrating the o1 model’s superiority over doctors in handling messy ER scenarios.

Study Design: The Harvard study pitted OpenAI’s o1 against physicians in six experiments using real ER cases, NEJM complex discussions, and structured management scenarios.
AI Limitations: Researchers noted the study used text-only inputs, excluding images like X-rays, with ongoing work showing rapid multimodal improvements.
Integration Path: Experts advocate AI for ER triage and second opinions to catch diagnostic errors, pending controlled trials on patient outcomes.