Rethink Imaging

EP 11 • January 23, 2025

The Psychology Behind Radiologist Errors with Elizabeth Krupinski

Featured Guest

Elizabeth Krupinski, PhD

Professor and Vice Chair for Research, Department of Radiology and Imaging Sciences, Emory University •

Listen Now34 min

Also on:

Let's Talk

Beyond dose

Ask us what your imaging data can actually show you.

Start a Conversation

Share This Episode

Subscribe to Rethink Imaging

Radiologists interpret thousands of images under conditions that push human perception to its limits. In this episode, Chris St. John talks with Dr. Elizabeth Krupinski, an experimental psychologist who has spent her career studying how radiologists actually see. She breaks down the three categories of interpretive error her eye-tracking research has identified: search errors, where the eye never lands on the finding; recognition errors, where the eye sees it but the brain never flags it; and cognitive errors, where the reader considers the finding and dismisses it. She also explains why radiologists performed worse than undergraduates at finding Waldo, and what that says about how domain-specific expertise really develops.

The conversation then turns to the conditions that make errors more likely. Krupinski’s NIH-funded research found that after just eight hours of reading, diagnostic accuracy drops by about 5 percent, with both false negatives and false positives rising, and her team confirmed the effect with an objective measure of visual fatigue borrowed from ophthalmology. She and Chris also cover reading room ergonomics, from monitor calibration to ambient light and chairs, the isolation and environmental pitfalls of reading from home, how malpractice risk shapes call thresholds, and AI-aided personalized training that gives residents the immediate feedback clinical practice rarely provides.

Host

Chris St. John

•

Featured Guest

Elizabeth Krupinski, PhD

Professor and Vice Chair for Research, Department of Radiology and Imaging Sciences, Emory University •

Watch the Episode

Key Takeaways

Eye-tracking research sorts radiologist misses into three categories: search errors, where the eye never fixates on the finding; recognition errors, where the eye lands on it but the brain never flags it; and cognitive errors, where the reader looks at it, considers it, and decides it is nothing.
Expertise is domain specific, not innate. In a Where’s Waldo study from the early 1990s, radiologists were no better than lay people at finding Waldo, and probably worse than undergraduates. Skill comes from the volume and diversity of cases seen, and search patterns measurably change across each year of residency.
After only eight hours of work, diagnostic accuracy drops by about 5 percent, a statistically significant decline, with both false negatives and false positives increasing. Most radiologists work 10 to 12 hour days, so the real-world effect is likely larger.
Krupinski’s team measured fatigue objectively with an autorefractor, a device from ophthalmology that tracks focusing ability. After eight hours, radiologists and residents were physiologically less able to focus, which matters when the job is finding fine details in images.
AI-aided personalized training is a promising fix: a University of Wisconsin system for breast imaging identifies each resident’s weak areas, serves targeted case sets from path-proven cases, and gives the immediate right-or-wrong feedback that clinical practice rarely provides.

Full Transcript

Elizabeth Krupinski Part 1 Raw Transcript

[00:00:00] Chris St John: Once again, no pressure, free flow. If it turns out it’s a little shorter, that’s great. If it turns out it’s a little longer, we can edit it down. Um, great. Uh, and then did you have any, did you have any concerns about the intro or the questions?

[00:00:14] Elizabeth Krupinski: Nope, all is

[00:00:15] Chris St John: good. Great. Absolutely. Okay. We’re just going to go for it then.

[00:00:18] Chris St John: Okay. Great. So today on Frame by Frame Rethink Imaging, we are very excited to welcome Dr. Elizabeth Kropinski, a pioneering expert in the psychology of medical imaging. Dr. Kropinski is a professor and vice chair for research in the Department of Radiology and Imaging Sciences at Emory, there see, there it goes, is a professor and vice, tripping over my words, is a professor.

[00:00:45] Chris St John: and vice chair for research in the Department of Radiology and Imaging Sciences at Emory University School of Medicine. With a background in experimental psychology, her research focuses on understanding the cognitive, her [00:01:00] research focuses on understanding the cognitive, cognitive, I’m so sorry, I swear I’m literate.

[00:01:10] Elizabeth Krupinski: Go on vacation.

[00:01:14] Chris St John: Her research focuses on understanding the cognitive and perceptual mechanisms that influence how radiologists interpret medical images. Today we will dive into the psychology behind radiologist error, explore the challenges of burnout, and discuss how psychological principles can be applied to improve the accuracy and well being of radiologists in practice.

[00:01:34] Chris St John: Welcome Dr. Krupinski.

[00:01:37] Elizabeth Krupinski: Thank you. Good to be here.

[00:01:39] Chris St John: Great. It’s so good to have you. Um, now, you know, let’s, let’s start off. You, you have an interesting background in experimental psychology. What initially drew you to the intersection of, uh, psychology, but, and radiology together?

[00:01:56] Elizabeth Krupinski: Pure chance.

[00:01:58] Chris St John: Uh huh.

[00:01:59] Elizabeth Krupinski: Uh, [00:02:00] so I was, My background, like you said, is in experimental psychology, and, uh, you know, what that means is a whole variety of things to a whole variety of people.

[00:02:08] Elizabeth Krupinski: Um, but as an undergrad, what it meant was looking literally at the neurobiology of the visual system, and, um, at the cellular level, and putting electrodes into rats visual cortex. And that simply did not really appeal to me once I really, you know, got into it. I wanted to do something where I would want, you know, to see more of an immediate impact and to see how what I was doing would actually impact people in society.

[00:02:33] Elizabeth Krupinski: Um, and so when I started at my master’s, um, I was doing more kind of real world type stuff. And my advisor at the time, um, had, uh, his old advisor actually, and they were looking for someone to come into their lab and work. And in my master’s, I was looking at the perception of art. And how one person could look at a painting and say, wow, that’s amazing.

[00:02:57] Elizabeth Krupinski: Um, you know, look at those brush strokes and everything else. [00:03:00] And somebody else is saying, it’s red. You know, what’s, what’s the big deal? Um, you know, and why, and what in the, what features in a painting will make people react one way or another. And I realized then that, you know, if I go down that route, and it’s fascinating.

[00:03:14] Elizabeth Krupinski: Uh, you know, it’s amazing what leads to people liking art versus design. someone else saying no. Um, but I realized, you know, I would end up teaching 100 percent of the time, maybe have my summers off to do research. And really nobody funds research in the perception of art. And again, back to the impact, I really didn’t see how that was going to have a huge impact on society or people.

[00:03:35] Elizabeth Krupinski: And so I said, yeah, sure, I’ll, I’ll go, you know, interview. It seems like an interesting possibility. And that lab happened to be run by a radiologist and a cognitive psychologist, actually an educational psychologist. who were looking at errors in radiology from that perceptual cognitive point of view.

[00:03:53] Elizabeth Krupinski: And it was like, okay, this is what I want to do for the rest of my life. Please hire me.

[00:03:57] And,

[00:03:58] Elizabeth Krupinski: [00:04:00] uh, then I, I spent, uh, five years, uh, with them. And I also got my PhD. That was at UPenn. I got my PhD at Temple. Uh, working in their lab and doing all my research on, uh, medical image perception. And so, literally, it was pure chance, but once I, once I found it and got into that lab, that was it.

[00:04:21] Chris St John: Ha! That’s very funny. I, my, I, I was an art student in college. Uh, I actually did, my, my senior thesis was a, uh, well, you know, it was, it was a, um, What’s the word? It was like an immersive art piece designed to, like, get people to think about how the space they are in affects the art that they are looking at.

[00:04:46] Chris St John: So I had, like, traveled around Europe a bunch and, like, noticed where the benches were placed in specific galleries and, like, was curious about how, like, the social dynamics of, like, Seats in certain spots [00:05:00] affected people’s viewership of art and like the, the emphasis put onto different works just based on how people move through the space, so it’s, it is funny to, to, to, to be talking to another art to imaging, uh, person, um.

[00:05:15] Chris St John: Because

[00:05:15] Elizabeth Krupinski: we all went into different things.

[00:05:17] Chris St John: Yeah, oh, absolutely. Um. Beautiful. So, so Dr. Krupinski, in your research, you focused on the cognitive and perceptual mechanisms that influence how radiologists interpret medical images. Can you explain just a little bit about that and then how these processes start to contribute to mistakes?

[00:05:42] Elizabeth Krupinski: Yeah. So, I mean, when you, you know, take it down to its absolute lowest level, what, what is looking and interpreting a radiograph? It’s looking at a picture and making a decision. Now, that’s, let’s take it down all the way down. Um, now, obviously it’s more complicated than that. Um, but looking at a [00:06:00] picture that involves human perception, which has its limitations.

[00:06:05] Elizabeth Krupinski: Um, you know, we are, in our visual systems, are influenced by a ton of things. One, our basic wiring. Um, we are limited in, uh, the types of things we can see. Uh, you know, for example, we don’t see ultraviolet. or infrared light, yet other animals do. So the information is out there and available, but our, the way we’re wired does not allow us to see those things.

[00:06:29] Elizabeth Krupinski: Um, we are good at certain things and not good at others. Um, we have a certain dynamic range that we’re able to see. And so, from the perceptual side, we have, like I said, certain limitations. In terms of what we’re actually able to extract from the world, and in this case images, in order to then send that information up to the brain, which is the cognition part, which to some extent has limitations as well.[00:07:00]

[00:07:00] Elizabeth Krupinski: Um, you know, the brain and that eye brain interface is formed and shaped by our experiences, what we’ve encountered, the limitations of our visual system, and and our cognitive systems, biases that we may have, things we’ve encountered in the past. So our entire history of, you know, radiologists interacting with radiology images and so on, the types of images they see, the type of training they receive, the types of, the spectrum of diseases that they’ve encountered, all of this shapes our ability to perceive images and so on.

[00:07:38] Elizabeth Krupinski: Put on top of that, the fact that Our brains don’t just click into one thing and that’s what you’re doing. Other thoughts can intrude as well. So if you’re worried about, um, you know, the, the football game tonight, and you know, you’re, you’re sitting there, who’s going to win the national championship? Um, you know, that’s kind of in the [00:08:00] back of your mind, and every now and then you might, you know, You know, kind of wander your thoughts, uh, and start thinking about that rather than the task at hand, interpreting an image, or you might worry about something else, or you may be interrupted.

[00:08:12] Right.

[00:08:13] Elizabeth Krupinski: There’s a lot of things that can happen that disrupt the cognitive processes and what you were doing in order to perceive the information in an image and extract it. And that’s where errors can occur, um, when, when some of these things go wrong. Um, and there’s a lot of reasons for errors. Uh, and over the years, we’ve kind of broken them into, you know, sort of three main categories.

[00:08:37] Elizabeth Krupinski: Um, there’s sort of search errors, uh, which, you know, if you, if you think about an image, say, with a chest nodule, and you’re looking around, and you simply don’t even look at the target. And it’s not big enough to kind of see through your peripheral vision or anything. You simply miss it. You don’t look at it.

[00:08:57] Elizabeth Krupinski: And there’s a variety of reasons why that may occur. It may [00:09:00] be so small, so well hidden that, you know, it’s out of the realm of what you could see without really dedicated, you know, oh, wow, there it is. Um, but again, you could be distracted and simply not look at that part of the image. And then there are, um, recognition errors where you actually do look at it.

[00:09:22] Elizabeth Krupinski: But Your eyes see it, they start to process it, but it doesn’t quite hit cognition. The brain doesn’t recognize it as, hey, pay attention, something’s potentially there. And then there’s the third category, which we call cognitive errors, which is you actually look at it, you consider it, and then you say, no, I don’t think that’s

[00:09:45] Right.

[00:09:46] Elizabeth Krupinski: You know, and we’ve, we’ve found this out through eye tracking over the years. Um, but yeah, there’s a lot of reasons why errors occur. Um, like I said, straight from basic biology through just the environment within which we [00:10:00] live.

[00:10:00] Chris St John: Yeah. Well, I, it’s, it’s funny. I, I, as you were explaining those types of errors, I’m, my mind immediately went to like a where’s Waldo book where every page, okay, I need to find Waldo He may or may not be on this page.

[00:10:16] Elizabeth Krupinski: Right? Exactly. Right?

[00:10:18] Chris St John: It’s like, he might be there. And so like, you know, you know, I think I saw a study that was like, here is the optimal eye pattern to like try and find Waldo, um, thinking about that, but like with so much on the line and you’re looking at it. All day long, and you don’t know if he’s there or not.

[00:10:37] Elizabeth Krupinski: Exactly. You know, and the funny thing is, we did a study years and years ago, um, in early 1990s, where we looked at radiologists and just lay people, people from the lab, doing Wears Waldo. and radiologists are no better at finding Waldo than anybody else.

[00:10:58] Elizabeth Krupinski: They have this innate [00:11:00] capability to find things. Um, they don’t, uh, they’re, like I said, we, we did Waldo and, you know, they were in fact probably worse than undergraduate students at finding Waldo.

[00:11:08] Chris St John: So, so you’re saying the,

[00:11:12] Elizabeth Krupinski: the,

[00:11:13] Chris St John: the radiologists in fact, were, were performing less or not performing as well as the grad students at Where’s Waldo, which I guess makes me wonder, like, how much does experience play into the fact that radiologists are making mistakes like

[00:11:34] Elizabeth Krupinski: a little bit yeah I mean we’ve been able to look at and we’ve done this in radiology.

[00:11:38] Elizabeth Krupinski: We’ve actually done it in pathology as well where we’ve looked at In every study that I do actually looks at both residents and attendings Because I like to look at you know How do people with different levels of experience perform at some of these different tasks and almost always there’s differences Um, and it, it, it [00:12:00] truly is amazing.

[00:12:01] Elizabeth Krupinski: We did the, the study in pathology that we did. We were able to get a cohort of pathology residents right at the beginning within, within one week of starting their first year of residency. Then the first year of their second year for, I mean, first week of their second year, first week of their third year, first week of their fourth year.

[00:12:19] Elizabeth Krupinski: So we were able to get them. And what we did was we ran them on the same set of images. And we, we track their diagnoses and we track their eye tracking, how they searched. And we were able to see how expertise evolved as a function of those four years in residency. I mean, their, their, their search patterns changed, their ability to recognize things, make diagnoses changed.

[00:12:44] Elizabeth Krupinski: And it, it literally is that simple. You know, it is something that evolves and it’s a function of the number of images that you’re exposed to, the diversity of the images, the types of diseases and so on that you’re exposed to. Plus, I mean, obviously [00:13:00] there’s some, you know, you’ve got to have some ability.

[00:13:04] Elizabeth Krupinski: Um, you know, not, not everybody who goes into pathology or radiology is actually good at it.

[00:13:12] Right.

[00:13:12] Elizabeth Krupinski: And some people leave. Uh, and some people are the ones that go to, I, you know, if I, if this, if it’s a chest image and I’m looking, I’m going to Dr. X and, you know, I know that they’re like the world’s expert in, in this.

[00:13:25] Elizabeth Krupinski: And there’s other ones where, you know, well, maybe let’s go ask somebody else. . Right. Um, but it is, it, it it is something that evolves over time. It is domain specific. I mean, if our radiologists had spent time looking for where’s Waldo in their spare time, I’m sure they would’ve been experts at Where’s Waldo as well.

[00:13:44] Right.

[00:13:44] Elizabeth Krupinski: Um, but it, it is a, a skill that evolves. And it is both perceptual and cognitive. Like you said, there, there’s search patterns, like with Waldo. As you gain expertise, there are optimal ways to search. And it may be idiosyncratic [00:14:00] to the individual person, um, but there are ways to make your search efficient and effective and then get that information to the brain.

[00:14:09] Elizabeth Krupinski: And your brain evolves as well to be able to figure out, yes, that’s something, no, that’s not something that’s an artifact or that’s just, you know, normal anatomy looking kind of funky.

[00:14:20] Chris St John: Right. And so like, as, as, as we, they radiologists are developing these skill sets, getting better at looking, changing the way that they look, gaining experience, um, You know, I, I imagine it helps a lot up to a certain point, but then at what point does overconfidence or fatigue start to, like, cross over with that, you know?

[00:14:46] Elizabeth Krupinski: Mm hmm. I, I’m not sure about overconfidence. I mean, I don’t think anybody’s ever really studied that per se. Um, but fatigue, certainly. Right. Uh, burnout, I mean all of those things. [00:15:00] And we’ve done a number of studies. We had, uh, about 10 years worth of funding from the NIH to look at the impact of fatigue on diagnostic performance.

[00:15:09] Elizabeth Krupinski: And, uh, we did a whole variety of studies from very, uh, Basic type tasks are very simple tasks like, you know, finding nodules in chest images all the way through more complex tasks like looking for different signs of trauma within, uh, CT images. And fatigue has an impact. Um, we, we started out by, you know, it’s funny how we started out.

[00:15:33] Elizabeth Krupinski: It is, when I run my eye tracking studies, you know, I bring people into my lab and typically it’s over lunch. You know, because that’s when the radiologists are free. And I was noticing on a couple of them that, you know, they came to my lab and I was having trouble getting the signal that you need to record eye tracking.

[00:15:52] Elizabeth Krupinski: And so I looked more closely and, you know, their eyes were like drooping. And, you know, I was like, you’re falling asleep, dude, what’s going on? [00:16:00] And they said, you know, it’s just so exhausting. That, you know, there’s so many cases that we have to read and so on, that, you know, by, by noon, it’s, it’s just sometimes you’re just exhausted and you’re tired.

[00:16:11] Elizabeth Krupinski: And, and that got me thinking, okay, well, you know, if they’re this tired by noon, and their eyes are drooping in my lab, what are they like in the clinic? And so that was sort of the inspiration for the first grant that we submitted. And we ended up getting funded, which was great. It was me and a colleague from the University of Iowa, Kevin Burbaum, and we were able to, like I said, sustain this for about 10 years, looking at the causes of fatigue and its impact.

[00:16:39] Elizabeth Krupinski: And we started out by seeing, can we find a physiological biomarker in a sense? Being able to objectively measure fatigue. Um, and so he said, okay, well let’s look at the eyes. That’s the most likely place to look because otherwise the answers could be subjective. Are you tired? Yeah, I’m tired. Okay. Well that, that’s kind of subjective.[00:17:00]

[00:17:00] Elizabeth Krupinski: And so we looked at, um, uh, a device, um, that ophthalmologists use, um, on auto refractor, which basically tracks how well can you focus? Cause if you think about when you’re sitting in front of a computer, Or even in front of the TV, I guess, for hours on end. You’re, you lose your ability to focus. Your eyes get dry, you’re, you know, you’re blinking, you’re moving back and forth, and sometimes you start to see double.

[00:17:26] Elizabeth Krupinski: And that’s a hallmark of visual fatigue. And so, like I said, we talked to some ophthalmologists, and they said, you know, an autorefractor, that’s probably the best thing. And what we showed was that, indeed, by the end of only eight hours, we were able to measure physiologically. that radiologists and residents were less able to focus.

[00:17:50] Elizabeth Krupinski: And again, if you’re less able to focus, how are you going to find fine details in an image?

[00:17:55] Right.

[00:17:56] Elizabeth Krupinski: That then said, okay, we have an objective measure. We also had some [00:18:00] subjective measures, uh, you know, rating how fatigued are you and so on. And then we said, okay, we’ve got objective measures. We’ve got subjective measures.

[00:18:06] Elizabeth Krupinski: Let’s go into the lab and let’s see if it has an impact. And through all these studies that we did in chest and bone CT and so on, what we found was that And we typically study people after only eight hours. Now, I don’t know many radiologists who only work for eight hours. They’re like 10, 12 hours. And so after only eight hours, what we were finding was that there were significantly, statistically significant drops in diagnostic accuracy after only eight hours of work on average by about 5%.

[00:18:41] Elizabeth Krupinski: Um, and that, you know, in the world of radiology, that’s pretty big. And the changes were not just kind of shifting their criterion, they were actually getting more false negatives or misses, and they were getting more false positives. So, both bad decisions were increasing. And [00:19:00] we did eye tracking studies as well, and what we found with the eye tracking was they would, Their patterns would get, when you’re fresh and you’re on top of things, your patterns are very efficient and you’re finding things and you’re super kind of going through the image and you’re seeing things coming and making decisions going on.

[00:19:18] Elizabeth Krupinski: When you got more and more fatigued, they kind of slowed down. Um, kind of idled, looked around, took them longer to find things. Sometimes they didn’t, like I said, their errors went up and so on. So a lot of things change with fatigue. And, yeah, it, it impacts diagnostic accuracy.

[00:19:39] Chris St John: Right, like, like, like trying to read a book when you’re tired and you end up reading that same paragraph over and over and you’re like, I’ve read this paragraph four times, I have no idea what it says.

[00:19:48] Chris St John: And

[00:19:48] Elizabeth Krupinski: that’s exactly what it is. I’ve looked at this chest image five times. I don’t have a clue. I gotta go back again and see if it’s really there. I’m gonna, you know, I’m gonna gird my loins and here I go. You just [00:20:00] can’t do it. I mean, fatigue is fatigue. It’s hard to push through it.

[00:20:03] Chris St John: Yeah, absolutely. Um, with that, are there environmental factors like lighting, equipment design, workspace ergonomics that can aid radiologists in improving their performance if they’re fatigued or honestly, even if they’re feeling at their best?

[00:20:21] Elizabeth Krupinski: Yeah, in general, I think ergonomics is absolutely critical. Um, and I think Um, as volumes have increased as we went digital, I think this became even more critical and radiologists started to realize it. Um, you know, when people are working with film, you know, this is a long time ago, 35 years ago, I guess.

[00:20:42] Elizabeth Krupinski: When people were working with film, they would read a batch because someone from the film library would come in and put all these films on what we used to call a film alternator and the radiologists would sit down, go through all those cases. And then they’d have to take a break because the film [00:21:00] librarian would have to come take them all down, put them back in the folders, and put up a new set.

[00:21:04] So

[00:21:04] Elizabeth Krupinski: there were built in breaks.

[00:21:06] Right.

[00:21:07] Elizabeth Krupinski: Um, and you would have to, when you wanted to read, uh, the patient’s history, you would have to look away from the images, you would look at the film, you know, the information in the, in the film folder. patient history and so on you would have. And so there were, your eyes were doing very different things with film.

[00:21:24] Elizabeth Krupinski: When we got into this digital environment, then people started to realize, oh, okay, it’s more than just having the lights down low. Because we knew from film that you needed to have the lights in the reading room down low. I mean, you needed to dark adapt. And that, that was critical. But, we didn’t think that much about the chairs that people sat in.

[00:21:43] Elizabeth Krupinski: What the ambient light really has to We didn’t think about noise or air conditioning and all these things. And over the years, I’ve studied a lot of factors, as have others, that have impacted performance. [00:22:00] And what a lot of us started out with, I started out with, was looking at the monitors themselves.

[00:22:07] Elizabeth Krupinski: Probably still is the best thing. It’s got the best resolution. Uh, you know, it doesn’t have the same kind of contrast. But when we switched to monitors, there were a lot of concerns about that, because especially 30 years ago, monitors were nowhere near what they are today.

[00:22:23] Right.

[00:22:24] Elizabeth Krupinski: Um, and so what we were looking at and the types of factors that did influence performance and, you know, monitor brightness, monitor contrast, uh, how you calibrate the monitor, um, pixel pitch and pixel size, uh, whether or not you’ve got one big monitor, say like a 10 megapixel monitor or two five monitors side by side, you wouldn’t think that that would make a difference, but it does.

[00:22:52] Mm hmm.

[00:22:53] Elizabeth Krupinski: The ambient lights make a difference and that interacts with the monitor. The monitor, [00:23:00] um, the chair you sit in, uh, whether, whether you’re fatigued, whether you’re hungry, how many interruptions you get. So yeah, the environment within which you operate and the tools that you have clearly impact, uh, diagnostic accuracy and efficiency.

[00:23:17] Elizabeth Krupinski: And over the years, what I found is that, you know, sometimes it’s more efficiency that matters. Sometimes diagnostic accuracy is not affected that much, but people, their efficiency is, they slow down, it takes them longer, and especially these days when radiologists are reading so many images, that often is far more frustrating, uh, and having an impact on people’s well being and burnout, than being accurate, it’s their efficiency.

[00:23:48] It

[00:23:48] Elizabeth Krupinski: takes so much time to get through things when you’re sitting in an uncomfortable chair that’s making it, you know, move around and change your position and this hurts and that hurts. That’s going to impact [00:24:00] everything. It’s going to impact your speed as well as your efficacy.

[00:24:03] Chris St John: Right. And I mean, this is, this is maybe like a little bit of a, sequitur jump, but, but hearing you talk about this gets me and my mind thinking about, um, Yeah.

[00:24:16] Chris St John: Yeah. like, telemedicine coming into the conversation, right? And so you have like your ergonomic environment where you’re reading these images, but then you have like the larger structure around it, right? And so like, there are unique challenges and opportunities for someone reading these images remotely.

[00:24:39] Chris St John: Um, do you think that like, Like, how does, how does this differ psychologically from like a traditional in person environment?

[00:24:50] Elizabeth Krupinski: I think in, in two ways. One, your home environment, which is where, you know, let’s, let’s face it, that’s where a lot of people who are reading remotely are. They’re at home. Um, you, you [00:25:00] have to pay attention to all the things I just talked about.

[00:25:03] Elizabeth Krupinski: ambient lighting. You can’t sit in your living room with big, wide windows. It may be nice and comforting. Sounds great. Great. Sounds great. Well, letting the light in is not good for monitors. It can cause reflection. It changes the contrast. It does all sorts of stuff. It affects your, your, your perceptual system.

[00:25:24] Elizabeth Krupinski: Um, so you’ve got to be able to, truly replicate a good reading environment in your home environment. Um, and that may not always be possible. Um, and so, you know, when we have people reading remotely, you know, you gotta talk to them and say, you know, okay, ambient light’s this. You can’t have your dog and your kids running back and forth and, you know, doing this.

[00:25:45] Elizabeth Krupinski: You gotta create a space, a comfortable chair, because what I’ve heard from a lot of people working at home is, you know, they don’t always take breaks. Possibly even more so than people in the real reading [00:26:00] room. Because you just kind of, you’re, you’re in your little radiology cave.

[00:26:06] Right.

[00:26:06] Elizabeth Krupinski: And you know, you’re, you’re just kind of there.

[00:26:09] Elizabeth Krupinski: Um, some people may take all their breaks, who knows, but I’ve heard that some people don’t and you kind of get into this, you know, you start at 8am and next thing it’s midnight and it’s like, wow, okay, where did the day go? Um, but you have to pay attention to the environment. Okay. And I think one of the things that reading remotely you miss out on, and it’s a little harder to replicate is just the social interaction.

[00:26:35] And

[00:26:35] Elizabeth Krupinski: to me, I think that’s important. Um, you’re not interacting really with residents in the same way. I mean, sometimes you’re going to bring in the resident remotely, but they’re over there on another monitor or they’re, they’re just kind of, you know, maybe they’re on a cell phone, who knows, depending on how you’re connected up.

[00:26:52] Elizabeth Krupinski: It’s not quite the same and you’re not interacting with, um, other radiologists in the room. You [00:27:00] can’t really, you know, stop a second and say, Hey, you know, Fred, can you come over here and take a look at this? And, you know, well, what are your thoughts on this? You have to stop, you have to enter teams or zoom and say, you know, now you’re free.

[00:27:12] Elizabeth Krupinski: And, you know, it just gets more complicated and I think people just don’t do it.

[00:27:15] Right.

[00:27:16] Elizabeth Krupinski: Um, so, and that to me, that, that social part. is potentially even more negative, um, than all the environmental factors.

[00:27:29] Um,

[00:27:29] Elizabeth Krupinski: because I think it truly does, you know, impact people and, um, could potentially lead to burnout and fatigue as well as other things.

[00:27:37] Chris St John: Yeah.

[00:27:38] Elizabeth Krupinski: It’s very isolating, I think.

[00:27:41] Chris St John: Yeah, I mean, it’s, it sounds incredibly isolating. I mean, I, I work from home a decent amount of the time. I’m in the office now, but, um, Yeah, I mean, I find myself Honestly, having way more bandwidth working in the office and having like, you know, like a two minute sidebar with the person next to [00:28:00] me, all of a sudden feeling significantly more refreshed rather than once again, like home alone in my office where I’m just like going and going and going and it’s like, Oh, I’ll make my lunch.

[00:28:10] Chris St John: You know what? I’ll just eat my lunch while I keep working. Uh, it

[00:28:14] Elizabeth Krupinski: varies. I’m sure. I’m sure there are people who, you know, and I know some tell a radiologist or people who, you know, and they do it fine. There’s others where I’m sure it’s not as fun.

[00:28:24] Chris St John: Yeah. So I, to, to, to, to, to keep pivoting a little bit while, while we’re talking about like mistakes from a, from a psychological level.

[00:28:35] Chris St John: Um, I’m curious about like radiologists perception of risk and its influence on their decision making. So for example, is there a tendency to be more cautious or conservative with like certain types of findings? Um, And how might that affect the accuracy of the interpretations?

[00:28:57] Elizabeth Krupinski: I, I think, you know, it’s, it’s going to be an individual [00:29:00] thing because everybody experiences risk in a different way.

[00:29:04] Elizabeth Krupinski: It’s also going to differ as a function of the modality, um, the, the specialty that you’re in. Uh, you know, for example, you know, uh, breast imaging. Uh, if I. or a radiologist, you know, call something and, you know, it’s going to require follow up in a biopsy and so on. Yes, getting a biopsy or an ultrasound or an MRI, some kind of follow up exam.

[00:29:34] Elizabeth Krupinski: Um, you know, it is impactful on the patient and so on. But if I’m reading a chest image and recommending a biopsy, um, that’s potentially a lot more invasive. then a breast biopsy. A breast biopsy. Um, you know, it’s not going in through the lungs and risking a pneumothorax. Um, so, you know, every decision has its consequences.

[00:29:58] Elizabeth Krupinski: And I think, yes, everybody is [00:30:00] aware of what those consequences could be. And to some extent, I think that guides people’s willingness and unwillingness to call something. Especially in, you know, sort of that, that middle range of, you know, it’s either, you know, if it’s obviously there or obviously not there, people are happy.

[00:30:19] Elizabeth Krupinski: It’s always the funny ones in the middle that, uh, what do I do now? Um, you know, what do I recommend? And in some cases, it’s easy to recommend, okay, let’s do this follow up. In other cases, like I said, it’s not that easy of a decision. And on top of that, I mean, you know, let’s face it, the United States is a litigious society.

[00:30:40] Elizabeth Krupinski: And so that’s always a possibility in the back of people’s minds too, you know. Right. If I call this and it’s a false positive, am I gonna get sued if I don’t call this? And it truly is something, it’s a false negative. Am I gonna be sued? Am I

[00:30:53] Chris St John: gonna get sued? ?

[00:30:55] Elizabeth Krupinski: So, yes. You know, it’s gotta impact people’s decisions.

[00:30:59] Elizabeth Krupinski: And you know, [00:31:00] it’s kind of interesting that, that you can go from one practice to another and there are, there’s sort of a, a culture of an institution and people start to recognize what’s acceptable. in that institution versus another institution. And it’s not like saying that, you know, once it’s an institution, they’re going to call everybody back and another one not.

[00:31:23] Elizabeth Krupinski: But, you know, there’s, there’s fine levels of differences. And you kind of got to get used to what’s the culture there. And every, you can go to any radiology department in any division and ask, okay, who calls back people the most? You know, or, or, you know, who, who’s, you know, Who’s more conservative and who’s more liberal?

[00:31:45] Elizabeth Krupinski: Everybody will know. The techs will know. All the other radiologists will know. So everybody has their own thresholds. And I think that is a reflection of risk. Um, you know, does it contribute to errors? Everything contributes to errors. I mean, nobody’s perfect. Um, [00:32:00] and unfortunately our, our, our legal systems very rarely recognize that.

[00:32:04] Elizabeth Krupinski: There’s an expectations that, you know, physicians and radiologists are going to be perfect. Simply not. I mean, some things you’re, you’re, you’re going to miss and there’s going to be a variety of reasons for that. Um, so I, I, I think it’s, you know, it’s one of those things that’s always in the back of people’s heads.

[00:32:22] Elizabeth Krupinski: Um, and, and it can influence some decisions, but I don’t think it’s like a driving force. That’s, you know, What’s at the front of everybody’s minds when they’re making decisions. Radiologists really are, are out there trying to do their best, uh, to find what’s going on with a patient and to be able to convey that information to the other providers and, you know, be involved in the diagnostic process so they can, you know, have the best outcomes for the patients.

[00:32:53] Yeah.

[00:32:54] Elizabeth Krupinski: Yeah. So there, but you know, I mean, it’s just a fact of life.

[00:32:58] Chris St John: Yeah, absolutely. Are [00:33:00] there, are, you know, are there, are there training techniques or approaches that you think could, like, help reduce error?

[00:33:11] Elizabeth Krupinski: I think that’s what the whole training process is all about. Yeah, I mean, absolutely,

[00:33:15] Chris St John: right? But, I mean, based on your, based on your research, like, are there specifics that, that get you excited?

[00:33:21] Elizabeth Krupinski: You know, I think the whole, concept of, um, personalized training. Um, you know, there’s folks that are looking at ways to, you know, in in normal clinical situation, you’re not going to encounter every single possible disease and at a high enough volume. Um, so what some people are doing is creating like test sets.

[00:33:42] Elizabeth Krupinski: And I have one colleague, Lonnie Salkowski of the University of Wisconsin, who’s looking at breast imaging. And what she’s doing is creating, um, sort of an A. I. aided system where the trainees go in, the residents go in, they read sets of images, and the AI [00:34:00] in a sense says, okay, you know, they’re, they’re really good at masses.

[00:34:04] Elizabeth Krupinski: You don’t need to see those anymore. Let’s concentrate on micro calcifications or, you know, asymmetry.

[00:34:11] And it

[00:34:11] Elizabeth Krupinski: will then adjust the type of training that this person is getting. And this, these are things that she’s hoping people will, you know, she’ll get the system up and running and people will do in their spare time to augment.

[00:34:23] Elizabeth Krupinski: the training that they’re doing in, in the clinic. And what she’s finding is that, you know, the residents love it because it is giving him one exposure to different types of cases and to immediate feedback.

[00:34:35] Right.

[00:34:35] Elizabeth Krupinski: She’s using cases that are already path proven in the clinic. You rarely get that immediate feedback.

[00:34:41] Elizabeth Krupinski: You know, you’re right, you’re wrong. That doesn’t happen. Whereas this type of training where you do get that immediate feedback and you start to learn, you know, here’s my weaknesses, here’s my strength. Maybe I need to work on my weaknesses a little bit more. That’s the type of training that I think can really help.

[00:34:58] Elizabeth Krupinski: And, and these days when you [00:35:00] can augment it with AI and help AI kind of figure these things out, I think that’s sort of a really cool direction that we’re going in.

[00:35:08] Chris St John: Yeah, absolutely. I mean, I, you know, I’ve, I’ve been talking with so many folks about like the way that AI is coming into play in, in imaging in general and, you know, high level stuff, getting down into the nitty gritty stuff.

[00:35:21] Chris St John: But no, I mean, it’s, I, this is, this is the first time I’ve like heard someone talk about using it as a qualitative training tool, like specifically in helping folks identify their own strengths and weaknesses rather than like these larger conversations that they’re having. It’s it’s really cool as as just like a qualitative training tool.

[00:35:46] Chris St John: Sorry, hang on. I’m pulling up my questions. So just to jump into the conversation, we’re going to edit this out. But, um, I feel like we are at like a pretty good part, like a, like a [00:36:00] pretty good, uh, we’ve got a pretty good part one in the making talking about, Like the psychology of mistake making so I might ask you one or two more questions And then if it’s cool with you We can transition a little bit more into burnout and and what’s going on in the background there.

[00:36:17] Chris St John: Okay. Okay amazing Actually, oh yeah, I’m gonna, I’m gonna ask this time question as the end of part one. Um, so, uh, Dr. Krupinski, radiologists are often facing, like, incredibly tight time constraints and high pressure situations. How does, like, the pressure of time impact their cognitive load, uh, and decision making accuracy?

[00:36:42] Elizabeth Krupinski: You know, I, I think it’s gonna depend on the individual. You know, it’s, you’ve got the people who, you know, I thrive on deadline, you know, put me under pressure and stress. I, I just excel. And there are some people that, that, that, that works for. And then there’s others that, you know, no, that, that, that’s not going to work.

[00:36:58] Elizabeth Krupinski: And they, you know, they just kind of fall [00:37:00] apart. So I think to some extent it is an individual thing. Um, but overall, I think it is leading to, uh, you know, uh, potentially more errors. I mean, there, there literally is only so much. an individual can do, uh, effectively. And when you think about, you know, especially some of these, you know, CT and MRI cases, or when you start to fuse PET MRI and PET CT, and you’ve got multi modality imaging, and you’ve got, you know, patients with long histories, and you’ve got to go into the patient record to sometimes figure out what’s going on.

[00:37:33] Elizabeth Krupinski: You know, all of these things contribute to, the pressures that the radiologists are feeling. And yes, it undoubtedly is going to lead to errors, you know, whether you can trace things back and document. Yes, this occurred because of this. That’s a little less straightforward. I mean, there’s, you know, correlation does not equal causation.

[00:37:57] Elizabeth Krupinski: Um, but clearly, I think that, you know, there’s, [00:38:00] there are increases in correlation. All of the factors these days that could lead to potentially to more errors. But on the other side, there are also a lot of efforts going into ways to figure out how to reduce those errors.

[00:38:15] Um,

[00:38:15] Elizabeth Krupinski: you know, whether it’s through AI, whether it’s through optimizing the environment, whether it’s through, you know, looking at different countries where they do double reading, for example.

[00:38:25] Elizabeth Krupinski: Um, you know, so there, there are strategies and we are looking at them. So, You know, is it, is it a doomsday thing that, you know, radiologists are under all these pressures? No, it’s not. Um, you know, and they’ve faced them over the years, they will continue to face them and their jobs will continue to evolve and the types of tools that they incorporate into their workflow in order to ameliorate some of these errors and address some of these things.

[00:38:51] Elizabeth Krupinski: That’s gonna change too and you know it’s going to evolve and I think it’s it’s it’s not something that’s negative it’s something that’s very positive because we’re we’re [00:39:00] acknowledging that you know things are changing and we have to change other things as well in order to keep the radiologists at the top their game.

[00:39:09] Chris St John: Yeah, absolutely. Um, and with that, Dr. Krupinski, I want to thank you so much for, for having this first half of the conversation with me and a little peek for our listeners. Um, Dr. Krupinski and I will actually be continuing our conversation in a part two, uh, specifically getting a little bit deeper into radiologist burnout and its scientific underpinnings.

[00:39:32] Chris St John: So please join us, um, for our next episode where we will continue this conversation. Um, That was just for the editors to use, obviously. You’re not going anywhere. I’m not going anywhere. Yep, yep,

[00:39:44] Elizabeth Krupinski: yep. All is good.

[00:39:46] Chris St John: Um, and just, just for me to, to take a little break from, um, You are just I love the way that you talk and explain things.

[00:39:58] Chris St John: Well,

[00:39:58] Elizabeth Krupinski: thank you.

Questions from This Episode

Why do radiologists miss findings on medical images?
Dr. Krupinski’s eye-tracking research identifies three failure modes: search errors, where the eye never lands on the finding; recognition errors, where the eye sees it but the brain never registers it as important; and cognitive errors, where the reader considers the finding and dismisses it. Distraction, interruptions, and the built-in limits of the human visual system all feed these errors.
How much does fatigue affect diagnostic accuracy?
Across roughly ten years of NIH-funded studies with Kevin Berbaum of the University of Iowa, Krupinski found statistically significant accuracy drops of about 5 percent after only eight hours of work, with both misses and false positives increasing. Since most radiologists work 10 to 12 hours, the clinical impact is likely larger than the lab numbers show.
Can training reduce radiologist errors?
Yes. Expertise develops from exposure to a high volume and wide variety of cases, and Krupinski’s studies show search patterns and diagnostic ability changing year over year in residency. She is most excited about AI-aided personalized training: systems that detect a trainee’s weak areas, serve targeted case sets, and deliver immediate feedback from path-proven cases.