Rethink Imaging

EP 48 • July 9, 2026

Why Most Radiology AI Fails: Workflow, Governance & Integration Problems

Featured Guest

Dr. Tessa Cook, MD, PhD, FSIIM, FCPP

Vice Chair of Practice Transformation, Department of Radiology; Director, Imaging Informatics Fellowship • Penn Medicine

Listen Now42 min

Also on:

Let's Talk

Beyond dose

Ask us what your imaging data can actually show you.

Start a Conversation

Share This Episode

Subscribe to Rethink Imaging

The best AI model in the world changes nothing if it never connects to the way radiologists actually work. Dr. Tessa Cook, Vice Chair of Practice Transformation at Penn Medicine, has been making that argument since before large language models existed. She started in informatics as a resident building Radiance, an open-source radiation dose tracking tool she gave away in 2010, and now co-chairs the AI Steering Committee that governs every AI tool entering Penn’s clinical environment. She walks Chris St. John through the three-phase evaluation every tool faces: retrospective model testing, a limited prospective trial with a handful of users, and a wider rollout where stakeholder feedback decides whether a purchase happens at all.

The conversation also covers the work Dr. Cook considers the real payoff of informatics: closing the follow-up loop. Her team built ARNIE, an automated recommendation tracking engine, which grew into Penn Medicine’s enterprise High Risk Follow Through Program. In roughly its first year, the lung nodule use case alone identified overdue follow-ups that led to 12 early cancer diagnoses, several in patients who did not know they needed the test. Add her take on measuring AI ROI in more than dollars, why ground truth is messier than vendors admit, and how she trains fellows in a program with more than 100 alumni, and this episode is a working manual for making technology stick in clinical practice.

Host

Chris St. John

Host, Rethink Imaging • Imalogix

Featured Guest

Dr. Tessa Cook, MD, PhD, FSIIM, FCPP

Vice Chair of Practice Transformation, Department of Radiology; Director, Imaging Informatics Fellowship • Penn Medicine

Watch the Episode

Key Takeaways

Penn evaluates every clinical AI tool in three phases: retrospective testing against radiologist-established ground truth, a limited prospective phase with a small group of users, then a wider rollout. If the model fails phase one, it never reaches the clinic, because a tool that does not work adds inefficiency to a workforce already short of radiologists.
The High Risk Follow Through Program, which grew out of the ARNIE recommendation tracking engine, caught overdue lung nodule follow-ups that led to 12 early cancer diagnoses in roughly its first year. Some of those patients said they did not know they needed the test.
AI ROI is more than dollars. Workflow efficiency, lower cognitive burden on clinicians, and better patient safety and outcomes all count, and the financial return follows from them.
Ground truth is rarely clean. Cook prefers concordance and discordance framing, because an AI reading only pixels lacks the clinical context a radiologist uses: priors, symptoms, family history, devices, and the actual clinical question.
Pennsylvania’s Act 112 required imaging facilities to notify patients of findings needing follow-up within three months. Penn’s research found the letters rarely beat the ordering office to the punch, which is why Cook’s team builds automated systems that track whether follow-up actually happens instead of relying on notification alone.

Full Transcript

Rethink Imaging Podcast Transcript

Guest: Dr. Tessa Cook MD, PhD. FSIIM. FCPP, FAAR, FACR

Host: Chris St. John

[00:02:07] Chris St. John: Welcome everyone back to Rethink Imaging. Today I am joined by Dr. Tessa Cook. Tessa, it is so good to have you here today.

[00:02:12] Tessa Cook: Great to be here, Chris. Thanks for inviting me.

[00:02:17] Chris St. John: Yeah, of course. So I have sent you a bunch of questions before the fact. And of course, my first question for you is not going to come from that sheet because I was doing a little additional interview prep and research and I found a super short interview piece on the SIIM website where you said that if you could choose anyone as a mentor, you would choose Chef Gordon Ramsay. And as a former chef, as somebody who has indulged in a bit of reality television, I felt like I would be remiss if I didn’t start out by asking you to expand on that answer a little bit, because I feel like it would help me get to understand how you’re thinking a little bit better.

TESSA COOK 00:002:57 I feel like you dug into some sort of time machine. My goodness. All right. You brought this up. We’re jumping in. So back in the day, before I had kids, which was now the better part of 11 years ago, my husband and I used to love to watch all of Gordon Ramsay’s various shows—the ones where he would tear into people and help them sort their stuff out. But also just the “idiot sandwich,” “this is a hockey puck”—I mean, those quotes still find their way into our kitchen from time to time.

And if you actually sort of peeled back the bluster part of it, which is the made-for-TV—and also I will say that if you watched the American shows versus the shows in the UK, there was a distinct personality difference, which was also fascinating—but there were a lot of leadership lessons in there. And it was a lot about working with people, leading a team, having integrity in the process. And I think it’s actually really good television. The drama part, if you like that too, great. But I think that’s really where that was coming from: the man knows a thing or two about leading teams successfully and working hard, and it was cool.

[00:04:23] Chris St. John: Yeah, absolutely. Because of my restaurant background, I was literally just talking about this with [unclear: sounds like “Dr. Sodex in”] the other day when he was talking about the importance of radiologists understanding bits and pieces of medical physics and wanting to work with the techs, learn how to run the machines, and gaining a more comprehensive understanding of the members of his team, what they’re up to, how they work in order to optimize. And I think that’s kind of the same message here, especially with Chef Ramsay and the brigade kitchen approach.

[00:04:57] Tessa Cook: Yeah, very similar sort of high-pressure environments. Lives, perhaps not as at stake quite so much in a kitchen not that you want to get any of your customers sick either but lots of parallels, I think, being able to successfully run a team, get people to work together, and also with some pride, for lack of a better word, in the eventual deliverable. We take patient care very seriously, and he clearly takes good food very seriously. And so there were many analogies to draw there. It was cool.

[00:05:41] Chris St. John: Yeah. Okay, cool. Well, thank you for indulging me for that little tangent. And so now to focus more on you yourself, you have this unique background with all this engineering experience and then medicine. You have the MD and PhD in bioengineering. I’m curious how your kind of interdisciplinary start shaped the way that you think about radiology and workflow problems.

[00:06:19] Tessa Cook: Medicine was always the eventual goal, if you will. Engineering was not really on the horizon until I got to junior year of high school. It was a decision that impacted the subsequent year, so I cut it a little close there. But yeah, it was interesting. So I had a good friend when I was a junior who she was a senior and applying to colleges and things like that, and told me about biomedical engineering, which I hadn’t really heard of before that. And I had always had an interest in computer science from an early age, but always also knew that medicine was the ultimate goal.

And the more I learned about BME, the more I realized this is kind of perfect for me. And so I ended up with a double major in biomedical engineering and computer science from Johns Hopkins. But really, it just kind of many things in my career trajectory, I didn’t plan for them. I just sort of followed opportunity where it took me, and that’s how I got there. And so, I loved engineering. I think it still impacts what I do every day. Radiology is the perfect clinical companion for the engineering interest, and informatics certainly, because I problem-solve all day, whether it’s in the clinic, whether it’s with my innovation team, whether it’s with the 3D lab. That is the part of my job that I truly enjoy is solving problems and fixing things.

[00:08:13] Chris St. John: Yeah, and so one of I know you have multiple titles, but one of them, if I’m correct, is Vice Chair of Practice Transformation. Is that still a title? Yes? Okay, beautiful. Just making sure. And so, I’m not super deep into the academic radiology space. I host this show, I learn what I learn from our guests and out in the world. I’m curious what practice transformation means at a high level to you and in your day-to-day.

[00:08:43] Tessa Cook: Yeah, I think you can define it different ways, and I know people define it in different ways. I think of practice transformation as using technology, using innovation, using team building to change and improve the way that we deliver care. And it doesn’t always have to do with technology. It really has to do with sort of shining a light on a problem and trying to figure out if there’s a way to optimize a process to improve experience, whether it’s the patient experience, whether it’s the healthcare worker’s experience, regardless of whose experience it is. And sometimes it just takes a small change, but it takes someone looking and identifying there’s need for that change to sort of start the process, if you will.

[00:09:37] Chris St. John: Right, and I feel like that’s absolutely a running theme throughout your professional career. So you built what was called Radiance dose tracking software and just gave it away for free during your residency in 2010. Is that roughly right? I know we’re going a way back.

[00:09:57] Tessa Cook: Yeah, that was my first real serious foray into informatics started with dose monitoring. I had a number of phenomenal mentors in the informatics space in residency, but [unclear: sounds like “Bill Boone”] and Woojin Kim in particular. Everybody knows Woojin; Woojin is internationally renowned. 2009, 2010 would have been my second and third years of radiology residency. And it was around the time that there were these very well-publicized instances of patients being overexposed to radiation from CT with visible side effects, right? Hair loss, skin redness, things like that.

And it really shone a light on a problem. And the hard part was, okay, well, how do you get to this data? And so we got creative with how we could get to the data. The specialty as a whole moved forward from a standards perspective. There was a DICOM standard to do this for years; it just hadn’t been adopted. And so, it was kind of the perfect storm of people paying attention to this data, starting to optimize protocols, looking at benchmarking and creating diagnostic reference levels. I mean, you look at how far things have come in the last 15-plus years. It’s really cool to see, but that was my first real big informatics project was Radiance and dose monitoring.

[00:11:35] Chris St. John: Yeah, it’s just fun to see how things have changed, especially with the new technology. So when you went to Penn and took on this title, Vice Chair of Practice Transformation, were there any specific problems that you were trying to shine a light on at that point? Like, did you have things top of mind or did you kind of go in carte blanche to see what you found?

[00:11:57] Tessa Cook: So I’ve only had the title since 2022. I’ve had the role in a fashion for some years even prior to that. And what we do is—so I basically run the Innovation Hub in our department and was very fortunate to learn from our Chief Innovation Officer at the time, Roy Rosin, how to do this stuff—not that I will pretend that I can do it nearly as well as Roy can, but I learned these little nuggets of wisdom that Roy taught us over the years. Making it easy for people to do the right thing, reducing cognitive burden.

And we really started out very focused within the department, looking at problems around closing the follow-up loop, identifying and figuring out how to better identify patients who had a follow-up recommendation on an imaging study and trying to make sure that they got whatever that follow-up was, whether it was a downstream imaging study, a biopsy, et cetera. And so we’ve worked on variations of that theme a lot over the years.

The other thing that we’ve done a lot with is resident education. So the team has supported a variety of tools developed and continues to support a variety of tools that help our radiology residents throughout their training, to get feedback about their cases on call, to help them make sure that they’re hitting all of their milestones over the four years of residency, and now looking at incorporating LLMs to do a lot of this also, and really trying to build tools that make it easy for radiologists, not just trainees, to get at data that it would be very hard for them to get at manually. Keeping track of interesting cases, getting feedback on the yield of your biopsies—these are not things for which systems exist. And it would be unrealistic to expect a radiologist for every interesting case to make a note for themselves and then keep checking back every couple of months to see what happened to that patient. So these are sorts of things that we’ve built over the years, many of them focused in the department, but then also things that have grown out of the department, which has been really exciting.

CHRIS ST. JOHN 00:014:34 Yeah, for sure. And so I want to follow up about your patient follow-up work. Do you mind explaining the Pennsylvania Act 112 and your research surrounding it?

[00:14:52] Tessa Cook: Yeah, so we have legislation in the state of Pennsylvania. It’s called the Patient Test Results Information Act, and I think it was signed into law in 2018. What it required was that for any finding which a—and I think I’m quoting from the legislation now—”reasonably prudent person” would seek medical attention within three months, the imaging facility should notify the patient. And I think the notification could be different things. A lot of practices opted for a letter in the mail, some sent electronic notifications, but either way, if there was something that the patient needed to follow up on within three months, the imaging facility was required through this law to let them know.

And so the way that most of us approached this was to flag those reports in some way and then generate a list of patients to whom the notification had to be sent. And so we went back after a few months and we looked at, at least to the best of our knowledge, whether it was making a difference. Could we tell from the medical record whether anything was actually happening sooner because this letter was getting sent out? And perhaps not surprising to some of us, the answer was no. In the majority of cases, the office had already notified the patient before the letter arrived. So I guess it was a nice backup, or perhaps it was serving as a reminder, but it was not the initial communication.

But it did require us in Pennsylvania to do a lot of additional work, and we’re all still doing this additional work. I know that the legislation has gone back through the state legislature a couple of times for some potential revisions, but I don’t know that the requirement to send this notification has changed since 2018.

[00:17:12] Chris St. John: Right, that language, “a reasonably prudent person,” it’s kind of—well, I hear that and my gut response to that is I don’t think I know many reasonably prudent people. Like, I think human beings and particularly, you know, I’m not trying to get on a soapbox here, but like particularly folks in the US healthcare system, where things are expensive, access is not necessarily easy for a lot of people. Like when you say reasonably prudent, I think back to when I was working in restaurants and the folks that I worked with—like we were all on paper reasonably prudent people, but I do not think the large percentage of us would have been showing up for those follow-ups.

[00:18:02] Tessa Cook: Yeah, there’s lots of reasons that people don’t go to their follow-up, right? They forget, they have to reschedule, maybe they never get around to scheduling. Life happens, right? And so that’s why we have spent so much time trying to build systems to help people, to remind people to make sure that—particularly when it’s a finding that could be an early cancer, and if you go get that follow-up and you get diagnosed early, that makes a big difference for your health going forward. And so we don’t want anybody to end up with a late diagnosis because they missed a follow-up, but exactly to your point, life happens.

There are lots of reasons why these downstream exams don’t get done. Sometimes they get done in other systems and we don’t have access to that information, right? There’s no one choice for where you go for your care necessarily, and that’s fine. The only downside to that, the main downside to that, is that there’s not enough good information exchange between facilities for those of us taking care of patients to know what is truly still incomplete or outstanding and where to effectively nudge.

[00:19:22] Chris St. John: Yeah, and so you’ve done a lot of work trying to bridge this gap. You built an automated radiology recommendation tracking engine. Can you tell me a little bit about that? Is that really what the acronym stands for?

[00:19:42] Tessa Cook: That is really what the acronym stands for. We put a lot of thought into that one, ARNIE. Yeah, so the idea behind ARNIE was that—and really this is because we built this before large language models existed, right?—this was heavily dependent on the hard work of my colleagues in the department to actually flag these cases with a very discrete set of data elements that included which organ needed the follow-up, what the follow-up should be, and when it should be obtained. And so what we were able to do was mine all this data and start to look for whether or not people had appointments scheduled, whether they needed a nudge, or whether their clinician needed a nudge.

And so we tried a variety of nudges. We tried a lot of things. Some things worked, some things didn’t. But eventually this basically, in a few different permutations, gave rise to something that we have now across Penn Medicine called the High Risk Follow Through Program. And the goal of High Risk Follow Through is across the enterprise, and we are building out use cases incrementally. So we haven’t covered everything across the entire health system yet, but we’re working on a variety of things, not just within radiology anymore. We’re looking at other kinds of testing. We’re looking at laboratory testing. We’re looking at patients who get diagnoses that could put them at high risk for chronic illness later and really just trying to create a series of earlier interventions so that we can help people who need help sooner and save them from bad outcomes down the road.

[00:21:48] Chris St. John: Yeah. How do you, with the High Risk Follow Through Program, how do you think about measuring success?

[00:21:56] Tessa Cook: That’s a great question. I’ll give you one example. So we started with lung nodules on CT, and within the first year of the program—I think maybe not even the first year—we had identified a number of patients who had had overdue follow-ups, and 12 of them were diagnosed with cancer. And so those were 12 patients that are now being treated. We caught cancers early, and we were able to get them plugged in with the resources they needed to start their treatment earlier. And a few of them even said, “I didn’t even know that I needed to get this test done.” And so those are real—those are lives saved, hopefully, through this intervention.

[00:22:48] Chris St. John: That’s fantastic to hear. So let’s expand out a bit beyond follow-up. I’m curious, what other issues are you shining your flashlight around at these days? Like workflow inefficiencies, things with interpretation and reporting or scheduling?

[00:23:06] Tessa Cook: There’s a lot of AI happening right now, you might be surprised to hear.

[00:23:10] Chris St. John: Well, yeah. And you’re doing a lot—you’re doing like agentic AI work with Woojin as well, am I correct?

[00:23:19] Tessa Cook: We’re doing some agentic stuff. We’re looking at sort of the whole spectrum, everything from what you would consider narrow or a really specialized AI that has a very particular task to all of what you can do with generative AI and reporting to foundation model draft reporting solutions now, because the field is really moving fast. And if there’s one thing that we don’t do in healthcare, it’s move fast. And so we’re all learning and trying to figure it out.

I co-chair our AI Steering Committee, so we oversee all of the AI that ultimately ends up in the clinical space and make sure that there’s governance around it, that there is rigorous evaluation before we put something into the clinic. Because AI is heavily touted these days as a solution to the workforce workload mismatch, to all of the inefficiencies in our workflow, but you run the risk of creating more inefficiency if you don’t do this right. And so we have really been trying and are very deliberate in how we deploy solutions and what solutions we deploy because we really do want to increase efficiency.

And it’s not just about efficiency because everybody says, “Well, what’s the ROI of AI?” How would you like to measure the ROI of AI? Because it’s not necessarily always in dollars. If you can increase workflow efficiency, that’s great; that ultimately helps patients. If you can decrease the cognitive burden on clinicians, that’s great; that helps the clinicians, and it also helps the patients. If you can improve patient safety, if you can improve patient outcomes—these are all good things, right? To me, this is all ROI. And I realize that I perhaps have the easy job because I’m not the one that makes the budget, but the budget has to make sense as well. But I really do think that the eventual ROI might have to be measured in dollars, but initially all of these other aspects are important as well, and you can eventually get to the financial benefit from everything.

[00:25:51] Chris St. John: Yeah, yeah, for sure. I want to harp on just a little comment you made at the beginning of that answer where you were talking about AI governance. And like pretty recently, I feel like y’all have wrapped up Penn’s AI governance model. Can you tell me a little bit about what that process looked like? I mean, I think you were at it for seven years.

[00:26:18] Tessa Cook: So I don’t know that I was involved in all of that. We’ve had our governance group in radiology since 2018 or 2019, so we’re definitely at the seven-year mark. I wouldn’t say wrapped up—we’re still going, but there’s the radiology-level governance piece, but there also is enterprise-level governance. And so we feed into their structure as well. We’re sort of the clinical imaging kind of first step, and then it continues there through the rest of the organization because we can’t unilaterally put anything into our environment if the enterprise hasn’t vetted it appropriately as well.

So it’s ongoing; it continues to evolve. I will be the first to admit we made it a little bit onerous at the beginning, and we have made it far less onerous for everybody, ourselves included, to review these tools, to have a liaison with a potential industry partner, to do the evaluations, prepare the data to do the evaluations—all of these things. And it’s work. I’m not going to tell you it’s not work; it’s work.

Because it is also part of the American College of Radiology’s Arch AI designation, so it’s the ACR’s recognized center for healthcare AI. And I should probably disclose that I’m now the Commission Chair for Informatics for the ACR. Governance is a big component of Arch AI as well, because you need to have a process around figuring out what AI you need, does it work on your data in your practices, keeping an eye on it, knowing what’s in the environment, what’s not in the environment, knowing how it’s working, so actually doing monitoring. So all of these components are important, and especially as the tools evolve, you face new challenges in kind of keeping up with all of these components also.

[00:28:28] Chris St. John: Yeah, I feel like I need to get you a hat rack for all of your different roles.

[00:28:31] Tessa Cook: I just need a hat rack for all of my different hats because I got through most of my life not owning a single hat, and now there’s a little collection growing.

[00:28:43] Chris St. John: Yeah, you have an abundance of hats on at all times. And so when you’re evaluating AI tools, do you mind just giving me a high-level understanding of what your criteria is and what your approach to thinking about it is?

[00:29:00] Tessa Cook: So we have a three-phase approach, and the way that it works is the first phase is really: does the model work? It really what it comes down to is model performance. So let’s say if we’re talking about pixel-based AI, take some representative cases, we de-identify them, we get the solution, we run them through, and then we look at the results. And we’ve got ground truth that’s been established by radiologists in some fashion—whether in some use cases you can take the original report and extract it from there, or in other instances you actually have radiologists creating new ground truth for you based on the images.

And phase one is really retrospective, and it’s: does the model work? Because if the model doesn’t work, we’re not going to put it into the clinic just to see it not work more. Again, it comes back to efficiency and having not enough radiologists to take care of all the patients that need care, and so we want to make sure that we don’t introduce inefficiency or a tool that is just not going to help.

So if the retrospective phase is good, we move into a limited prospective phase. So we give a small handful of users access to the solution. They use it prospectively in the course of routine patient care. And then we’re still evaluating model performance, but we’re also at that point evaluating user experience. And if that phase goes well, we expand it to a few more users just to make sure there’s nothing that we’re not accounting for.

We are geographically a large health system. In terms of specialty, we have a lot of depth and breadth, and so depending on this particular AI solution, we want to make sure that it really adds value for everybody. And if we get to that point then, and everybody feels like it’s important to move forward with a purchase, then we do. But the stakeholders all have input at any point in this process, and we take their feedback very seriously.

[00:31:18] Chris St. John: Yeah. What is your sort of approach and thought about reaching ground truth? Because I feel like it’s kind of nebulous.

[00:31:28] Tessa Cook: Yeah, it feels like that sometimes, right? And I think it really depends on what you’re talking about, right? If it’s a measurement, you can say, “Well, the AI said it was four and a half centimeters, but then the radiologist reported it as five and a half centimeters.” And that is a discordance. Now, it’s entirely possible that either of those were wrong, right? Did the radiologist mistype? Did they measure incorrectly? Was the AI measurement wrong? I mean, this is why we talk about concordance and discordance, right, and not ground truth or reference standard, because unless you’ve got a perfect ground truth, it’s hard to make that comparison.

One of the things that we aren’t doing very much in the AI space right now is that a lot of the primary processing of the data is really of the pixel data, right? So stroke AI, intracranial hemorrhage, pneumothorax—imaging study goes in, AI result comes out.

[00:32:43] Chris St. John: Yeah, like the classic examples for radiology AI, I feel like, right?

[00:32:51] Tessa Cook: Yeah, but you could say for anything else, right? Whether you’re trying to detect a disease process in—whether it’s cardiac MRI, whether it’s a pneumonia detector, or whatever it might be. Depending on the clinical context, the answer could be different. But if the AI is only looking at the imaging and not having any appreciation for the clinical context, that could also be a reason why the AI output is discordant from the radiologist’s interpretation.

And another ground truth, as you called it, could actually be patient outcome—biopsy, for example, right? So if you’re talking about breast cancer, imaging-based risk assessment for breast cancer, and you’re making predictions into the future, you might have a data point in the future that tells you whether the AI prediction was accurate or not. But you might have to wait some period of time, right? So it does get complicated.

And so I think we are really trying to come up with surrogate measures for lack of a perfect ground truth. But one of the big challenges now is that we don’t get this clinical context piece readily incorporated to be able to have—AI is not really approaching the imaging the same way that I as a radiologist approach the imaging. Because I have a whole additional set of data points that the AI model does not, and I know how to interpret those data points.

[00:34:35] Chris St. John: Do you mind telling me what they are?

[00:34:38] Tessa Cook: It depends, right? So for example, I’m a cardiovascular radiologist, right? So if I pick up a cardiac MRI to interpret, I want to know what the clinical question is. I want to know: has the patient had an echo? Have they had a cath? You know, what sort of symptoms have they had? Is there a family history? Do they have a cardiac device? All of these things are information to me, right? Not all of them are necessarily encapsulated in the imaging directly.

If we see something on a chest X-ray, for example, in isolation, that might look like an acute pneumonia. But if you have a prior study from six months earlier and that same opacity was there, that could actually be scar in the lung. And so it’s all of this additional information, right? Does the patient have a fever? Are they immunocompromised? Is there some reason they should have pneumonia? Did they have a family member or a close contact that was sick last week? These are all useful data that AI does not yet consider.

[00:35:45] Chris St. John: Okay, beautiful. Okay, so you direct the Imaging Informatics Fellowship at Penn, and I’m kind of curious just how you approach training the next generation on informatics and emerging technology.

[00:35:58] Tessa Cook: Yeah, so when I was a resident at Penn, I mentioned there were a number of faculty in addition to Bill and Woojin—[unclear: sounds like “Kurt Leinwatz”] was there, [unclear: sounds like “Steve Horry”] was there—and it was this like four luminaries in the field, all of whom were my clinical attendings, also were my informatics mentors. And I used to go to meetings and people would say, “Oh, it’s so great. You have the fellowship there and you’re training all these people.” And I would have to say, “What fellowship? I’m still a resident and we don’t have a fellowship.” So I heard this enough times that by the time I joined the faculty, I said to my future—well, I guess my also current department chair—”You know, I think we should start a fellowship. The rest of the world already thinks we have one. Let’s do this.”

So we actually went to visit Paul Nagy at Hopkins and saw how he ran his program, which was a very different—structured somewhat differently than what we ended up doing, but he was very gracious to host us. And really what I wanted to do was sort of create a version of what I had been able to experience as a resident, right? To be able to have access to amazing leaders in the field, to make those connections. Because that’s what my mentors did for me, right? They helped me to make these connections in person—because those were the days where everything was in person, right?—to actually help me create my network in informatics. And so that was one of the things I wanted to achieve with this program, but really to give people an in-depth exposure to all of what really matters in clinical imaging informatics right now: the fundamentals, the standards, the importance of interoperability, how you achieve that.

And then, you know, while we were sort of going about our weekly meetings, all of a sudden now there’s AI. And despite the fact that it is the Imaging Informatics Fellowship, I will admit we talk a lot about AI, right? Informatics is a lot more than just AI, and all that fundamental stuff that connects the pipes and keeps the data flowing is absolutely critical. But we talk a lot about AI too and how to implement it, how to do the governance, how to evaluate things, how to think critically about tools.

And so we will have more than 100 alumni in about a month, a little over a month, which is really exciting. The program started in 2013 with eight fellows, and across those alumni, we have vice chairs of informatics, we have CMIOs, we have associate CMIOs. People have really been incredibly successful, and it’s really heartening to see. We have these mini-reunions sometimes at meetings where there’s a little critical mass of us, but you know, it’s hard to get 100-plus people together at any point.

But it’s been a great program, and we do a mixture of things. So I basically call all my friends in the informatics world and I say, “Hey, will you hang out with my fellows for an hour?” And they are incredibly gracious with their time. And so I really do appreciate the fact that people from other departments are helping my residents and my informatics fellows learn. It is very generous of everybody to do that; it really is sort of a testament to the informatics community. We want more people. We want people who are excited about this field to be able to get involved and get plugged in, and so that’s part of it.

They all do a project. We do journal clubs, so every time we meet, someone presents a paper. It forces everybody to sort of think about what you read in the literature and how to really critically analyze what’s out there. And we work through the second edition of the Practical Imaging Informatics textbook that was published through SIIM, and we learn. It’s hands-on learning, and sometimes things work, and sometimes things don’t work, but you learn.

[00:40:23] Chris St. John: Yeah. Forgive me for pulling at this thread and we don’t have to get too into it. But when you were talking about bringing up AI usage like within the Informatics Fellowship, there was like a part of your tone that almost seemed apologetic. And to me, I feel like AI and the utilization of it as a tool feels like it’s almost impossible at this point to do informatics work without—I mean, of course you can do it without using AI as a tool, but it feels like kind of woven in at this point. And I’m curious if I was right about that tone and what was behind that.

[00:41:05] Tessa Cook: You were right about the tone, but not the intent behind it, actually. So we regularly sort of trade notes about, “Yeah, I tried doing this in Claude code or Codex, or Gemini’s my favorite,” or everybody’s—we’re constantly doing all that stuff. So I have no problem with them using AI and learning how to use AI responsibly, and that’s an important, very important piece of that, not just for my fellows, for my profession, quite honestly.

The apologetic piece of it is that I am continuously aware of the fact that you can have the best AI out there, and if you can’t figure out how to connect it into your radiology practice’s workflow, it doesn’t matter how good it is. And that’s the fundamental informatics piece of it that I think is still so important, and that we can’t ignore while we’re talking about all the new exciting stuff.

And so that where the apologetic tone came from is, yeah, we do spend a lot of time talking about AI, and sometimes I feel like maybe more than we should. So I try very hard not to make it at the expense of them learning about all the other fundamental, important, critical things: DICOM, HL7, FHIR, IHE, right? And the very important role that IHE has in using all of those standards to help us build tools that actually make a difference.

[00:42:44] Chris St. John: Okay, so to change Tessa hats one last time in this interview while I have you here, you just wrapped up serving on the SIIM board as chair. And I’m curious just like if chairing influenced your approach to what you’re working on, specifically with innovation at Penn.

[00:43:06] Tessa Cook: I had a really great time serving on the SIIM board. I was very fortunate. I got to sit at that table for years before I was actually a board member as a committee chair, various committees. And again, because my mentors got me a seat at the table, right? And it’s just the value of mentorship and sponsorship and creating that network and that community. So by the time I became SIIM Chair, I felt like I knew a lot about SIIM.

So my term ended in 2024, and so I served a year as the past chair. [unclear: sounds like “Nabil Sufter”]—it’s hard to believe, time really flies—Nabil’s finishing up his term as SIIM Chair. [unclear: sounds like “Alex Tobin”] will take over in July. And so there’s this great lineage of chairs that preceded me and chairs that succeeded me, all of whom I learned from.

And I should also give a shout-out to the SIIM CEO, to [unclear: sounds like “Cheryl Crider-Carrie”], who is a self-proclaimed governance geek. And I would say I think the thing, the biggest thing I learned from my time on the SIIM board was the value of governance. And I learned a lot from Cheryl and have taken a lot of that back, both to Penn and to my role at the ACR. So these volunteer roles are just so valuable to us as well. Certainly, we help our professional societies achieve a variety of things, but I’ve learned so much from volunteering with a number of different organizations, and it’s been a really valuable experience for me.

[00:44:51] Chris St. John: Well, with that, I feel like that is all the time we have today. Tessa, it has been so much fun having you here. And I promise you I will give you a little bit more time before I start hassling you down the road again.

[00:45:10] Tessa Cook: It was a great conversation, Chris. I had fun. Thanks so much.

[00:45:15] Chris St. John: Thank you so much. This has been Dr. Tessa Cook, a cardiovascular radiologist and national leader in imaging informatics and practice transformation. Dr. Cook, once again, thanks for being here.

Questions from This Episode

Why do so many radiology AI tools fail in practice?
Dr. Cook’s answer is workflow. You can have the best AI available, and if it does not connect into the radiology practice’s daily workflow, it does not matter how good the model is. That is why Penn’s evaluation process weighs user experience and stakeholder feedback alongside model performance before any purchase decision.
How does Penn Medicine evaluate AI before clinical use?
Three phases. First, retrospective testing on de-identified representative cases against radiologist-established ground truth. Second, a limited prospective phase where a handful of users work with the tool during routine care while the team tracks both performance and user experience. Third, expansion to more users across the health system, with stakeholders able to stop the process at any point.
What is the High Risk Follow Through Program?
An enterprise-wide Penn Medicine program that identifies patients with overdue follow-up recommendations and intervenes before a missed test becomes a late diagnosis. It started in radiology with lung nodules on CT, where it produced 12 early cancer diagnoses in about its first year, and is expanding to laboratory results and other diagnoses that put patients at risk for chronic illness.