This is my exact concern, patients will feel compelled to say they consent even if they do not, whether it is because of politeness or deferring to authority
When Rebecca Green arrived for her first appointment with a urologist, she was asked whether she consented to having the appointment transcribed by artificial intelligence (AI).
She said yes without hesitation.
"I don't want to be in her bad books or be that annoying patient who takes up more of her time," she said.
I've seen enough errors made by humans that I always check my record after an appointment.
Clinicians should be well educated on how this technology works but they aren’t. They’re given basic legal liability nagging (which, tbf, absolutely says to review the output). For my own doctors almost all of them use this and I refuse consent every time they can’t answer very basic questions: are recordings encrypted end to end, or at all? Are recordings destroyed after generation, or at all? Are the recordings processed by the emr vendor (typically epic or oracle) or a third party? Who is the third party? Are generated notes used for reinforcement learning and/or model refinement? Etc. so then it’s quite easy to say no; I will not consent to a technology that you don’t understand the basic security of because you don’t “do computers”
The penalties for violating this should be severe and supervision is even more necessary. I know for a fact when you use these tools you must attest that you reviewed the output for accuracy with every single note submission, because the people selling the models don’t want to touch that liability at all. What does this matter though if enforcement never occurs? The government is severely lacking at managing this kind of behavior. The only hope is insurers going at it, not out of altruism but because it gives them a reason to revoke payment for services
Exactly. I can really imagine these tools van be helpful and save time, but ffs don't just blindly trust everything like that. If you brought in a new coworker to transcribe for you you'd also first just see if it's OK, right?
And yeah, not sure if we can ever fully trust such an AI (actually maybe humans as well?) so reviewing and supervision will be necessary.
If only someone could've seen this coming!!! /$
