TL;DR
An Ontario audit reveals that 9 out of 20 AI note-taking systems for healthcare providers frequently produce incorrect or fabricated information. The findings highlight concerns about the accuracy and safety of AI tools in medical settings.
Ontario’s Office of the Auditor General has reported that nine out of 20 AI note-taking systems approved for use in healthcare settings routinely produce inaccurate or fabricated information, raising concerns about patient safety and data reliability.
The audit evaluated 20 AI scribe systems used by physicians, nurse practitioners, and other healthcare professionals across Ontario. The evaluation involved simulated doctor-patient recordings, with medical professionals reviewing the AI-generated notes for accuracy.
Findings revealed that nine systems fabricated information, including treatment suggestions and patient conditions not discussed during consultations. Twelve systems inserted incorrect drug information into patient notes, and 17 missed key details about patients’ mental health issues. Six systems either partially or fully overlooked mental health concerns discussed in the recordings.
OntarioMD, a support group for physicians adopting new technologies, recommends manual review of AI notes, but the report notes that none of the approved systems include mandatory accuracy attestation features. The evaluation process itself was criticized for giving disproportionate weight to criteria like domestic presence, with only 4 percent of the score dedicated to note accuracy.
Why It Matters
This report underscores significant safety and reliability concerns regarding AI tools used in clinical settings. Inaccurate medical notes can lead to misdiagnoses, incorrect treatments, and compromised patient safety. The findings also raise questions about the adequacy of current evaluation and oversight processes for AI healthcare systems, emphasizing the need for stricter standards and accountability.
H&P notebook – Medical History and Physical notebook, 100 medical templates with perforations
- Number of Templates: 100 medical history templates
- Designed For: Medical students by students
- Reference Sheet: Includes medicine reference sheet
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Background
The use of AI in healthcare is expanding globally, with systems designed to assist clinicians in documentation and decision-making. Ontario’s AI Scribe program was initiated to streamline note-taking and reduce administrative burden. Previous studies have shown that consumer AI models often produce unreliable medical information, but this is among the first evaluations focusing on AI tools for licensed healthcare providers in Ontario.
The audit’s findings come amid broader concerns about AI safety, bias, and privacy, with critics arguing that current evaluation frameworks may inadequately assess critical aspects like accuracy and security.
“Inaccurate weightings could result in the selection of vendors whose AI tools may produce inaccurate or biased medical records or lack adequate protection to safeguard sensitive personal health information.”
— Office of the Auditor General of Ontario
“More than 5,000 physicians in Ontario are participating in the AI Scribe program, and there have been no known reports of patient harm associated with the technology.”
— Ontario Ministry of Health spokesperson
Express Scribe Pro Transcription Software with USB Foot Pedal (Digital Download,License and Download Information Will be Inside The Box
- USB Transcription Foot Pedal: Heavy-duty Infinity IN-USB-3 pedal
- Transcription Software: Express Scribe Pro Transcription Software
- Transcription Headset: Included headset for transcription
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What Remains Unclear
It remains unclear how widespread the use of these AI systems is beyond the evaluated sample, or whether the issues identified are being addressed by the Ministry. The impact on patient safety and the potential for future errors are still under assessment, and the effectiveness of current oversight measures is uncertain.
Clinical Medicine AI: The Physician's Guide to AI Medical Scribes, Ambient Intelligence, and the Future of Clinical Practice
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What’s Next
The Ontario Ministry of Health has yet to publicly respond to the audit report’s recommendations. Further investigations and stricter evaluation protocols are expected, along with potential updates to procurement standards and mandatory accuracy attestations for AI healthcare tools.
Medical Nursing Reference Card Set, Healthcare References for Students, Nursing Reference Notes, Nursing Cards, Clinical Badge Cards, Educational
- Number of Cards: 10 double-sided reference cards
- Material: Durable, waterproof plastic cards
- Portability: Compact size for pocket or lanyard
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
Are patients at risk from these AI note-taking systems?
While no direct reports of patient harm have been confirmed, the audit highlights that the AI systems can produce inaccurate or fabricated information, which could potentially lead to misdiagnoses or incorrect treatments if relied upon without manual review.
What measures are being taken to improve AI accuracy in Ontario healthcare?
The report recommends stricter evaluation criteria, including mandatory accuracy attestations and improved oversight. The Ministry has not yet announced specific policy changes but is expected to review its procurement processes.
Will healthcare providers be required to verify AI-generated notes?
Currently, OntarioMD recommends manual review of AI notes, but no mandatory verification process exists. Future policies may mandate verification to ensure accuracy and safety.
How does this affect the broader adoption of AI in healthcare?
The findings raise concerns about the reliability of AI tools in critical medical settings, potentially slowing adoption until standards and evaluation processes are improved.
