Eliot Siegel Official Transcript

Chris St John – 00:00:02: Welcome to Frame by Frame: Rethink Imaging, a podcast by Imalogix. Here, we explore the intricate world of medical imaging, aiming to dissect the field and inspire both professionals and curious minds alike. I’m your host, Chris St John. Today, on Frame by Frame: Rethink Imaging, we are honored to have Dr. Eliot Siegel join us. Dr. Siegel is a leading figure in the realm of medical imaging and currently serves as the professor and vice chair at the University of Maryland School of Medicine’s Department of Diagnostic Radiology. In addition, he is the chief of radiology and nuclear medicine for the Veterans Affairs Maryland Healthcare System. His groundbreaking work in imaging informatics and his pivotal role in transitioning the VA Maryland healthcare system to a filmless environment underscore his profound impact on the field. Dr. Siegel’s extensive research reflected in over 200 articles and multiple books has set new benchmarks in digital imaging and PAC systems. Dr. Siegel’s influence extends beyond his academic and clinical roles. He is also a strategic advisor and consultant for various prestigious organizations, including the National Cancer Institute, and UNU. His leadership in the Society of Computer Applications in Radiology and his fellowship with the American College of Radiology and Society of Imaging Informatics in Medicine highlight his dedication to advancing medical imaging. Joining us, Dr. Siegel is going to share his invaluable insights into the intersection of AI and medical imaging, and we’re going to explore the future directions of this rapidly evolving field. Welcome, Dr. Siegel. Thank you for letting me get out, that massive introduction.

Eliot Siegel – 00:01:44: Yeah, thanks, Chris. It’s great to have the opportunity to chat with you, and thanks for the super nice intro. I really appreciate it.

Chris St John – 00:01:50: Yeah, absolutely. I appreciate you being here today. And so we were just talking a little bit about a childhood experience that you had hearing about computers developing personalities, AI, if you will, based on just amassing mass quantities of data. So given that your career spans several decades of pioneering work in medical imaging and someone who has this long history with cutting-edge technologies like AI, do you have any moments post-childhood, 10-year-old, where you saw the potential of AI in medical imaging specifically, once you were getting more involved in the imaging field?

Eliot Siegel – 00:02:32: Yeah, I mean, I can really think of a couple of milestones there. The first one was as a fourth-year medical student, I had a background in computer science as an undergrad and had been working my way through medical school, making some money doing programming for a variety of different interesting projects. And the chairman of radiology knew that I was interested in radiology and actually brought me along to one of the seminal pivotal meetings in radiology. It would have been probably 1985. And there was a meeting with a group or consortium of hospitals that wanted to create a radiology information system. And so we flew up to Boston together and they talked about taking radiology billing information and information from a variety of different scanners and creating something that would allow. One to be able to optimize workflow and radiology reporting. And back then that was really exciting. And they formed a consortium called RISC, R-I-S-C, Radiology Information System Consortium. And so at that meeting in Boston, you know, as a third year medical student, I was exposed to a fair amount of discussion about how, well, this is all really nice to do all this with text and billing and reporting. But sometime in the future, maybe distant future, we’ll actually be able to have images, stored on the computer and be able to do primary image interpretation. And that was kind of an aha moment. And, you know, I thought with my interest in computer science and background and strong interest in diagnostic radiology, that it would be really fun to somehow be involved in a project like that. And little did I know at that point that I would be actually able to participate in the entire process. And actually, as I’m remembering it now, I was a third-year medical student. And so it would have been 1980, actually. And so this was before I actually was recruited to the radiology program at University of Maryland. So as a third year medical student, being interested in radiology and hearing the whole presentation in Boston at the conference and kind of coming up with sort of the seminal idea of I would love some how to be involved with this process. And little did I know. You know, that 13 years later that I would have created the world’s first digital and film this radiology department at the Baltimore VA Medical Center affiliated with the University of Maryland. So I think that was my aha moment. And as I’m remembering it now, I wasn’t a radiology resident, but just a medical student thinking about applying to radiology.

Chris St John – 00:05:20: Yeah. Can you talk a little bit about that transition of going filmless for the first time?

Eliot Siegel – 00:05:26: Sure. So as I then finished my radiology residency after that incident as a medical student getting to go with the chairman and getting recruited for the department, and as I was thinking about what to do having graduated my radiology residency, I was asked by the chairman to stay in the department and actually given the opportunity at that point to be able to stay as a faculty member, but also be in charge of the department at the VA hospital. And that was a really unique opportunity because I was going from being a radiology resident immediately to becoming the head of the department at the VA. And so the attending radiologists who were, you know, had oversight over me, all of a sudden now they were working for me when they were at the VA. So it was a very inverted career where I sort of graduated from my residency in a position where I was the head of the department and being the head of the department. Part of the reason that I was interested in the VA in particular at that point, in addition to genuinely being really interested in wanting to help radiology residents was the whole idea of being able to create a filmless and digital department. And so I was given the opportunity to be able to design a department because the timing was such that a new radiology department and a brand new hospital was going to be moved onto campus from a location about five miles away. And so in building the new hospital and in being the head of the radiology department, I had the opportunity to design a whole new radiology department with new equipment, new workflow. But also I decided at that point that I was going to do it with the idea in mind of being the first department in the world to make the transition to digital. And the idea of all the advantages of going, digital were really fantastic. One of the heads of the department, the head of the Department of Medicine at the VA, made the comment, tongue-in-cheek, that the VA had always been filmless because you could never find the films when you… So that was part of the challenge to actually achieve this any image, anytime, anywhere, and be able to make images essentially accessible to anybody who needed the images when they needed to be accessed. This was something that nobody in the world had at that point. And the idea of combining the technology that I knew about from computer science and what was available at that point, including computed radiography that allowed us to make film acquisition digital also, and the ability to be able to utilize relatively low-cost computers so that we would be able to put imaging workstations and monitors throughout the medical center gave us sort of the critical mass for the first time to be able to do that. And in addition to the idea of any image, anywhere, anytime, I was intrigued at the potential for the computer to enhance the image, and actually help us to be able to make a diagnosis. And so the only way to be able to do that was to essentially make the transition from film to a digital environment. And the challenges associated with that were really considerable, but coming right out of residency and sort of bright-eyed and bushy-tailed and probably naive, I essentially thought it would be an incredibly fun and really cool project, which serendipitously it actually turned out to be.

Chris St John – 00:09:04: So just to try and draw a parallel and take your experience back then, let’s jump forward in time a little bit to today, right? So integrating AI systems into medical imaging involves various challenges. What are some of the most significant hurdles you see in embedding AI algorithms into clinical workflows? And then how can healthcare systems work to overcome these obstacles for more seamless integration?

Eliot Siegel – 00:09:33: Yeah, so there are so many different challenges. And I think that’s part of the reason that even 31 years later, after we made the transition and opened the world’s first digital radiology department, the vast majority of radiologists in the US and throughout the world are not really taking advantage fully of the capabilities of AI. And part of it is the fact that AI at this point is relatively a niche set of applications. And so if you’re looking to find lung nodules or find intracranial hemorrhage or find pulmonary embolism or determine pediatric bone age, there’s the capability of doing that. There are a myriad of challenges with adoption of AI into routine radiology clinical practice. And that’s part of the reason that so few radiology practices at this point are really taking full advantage of the potential of AI. And so one of the challenges is that the AI applications, that exists now, are relatively niche applications. And so we can find lung nodules on a chest CT scan or a conventional x-ray. We can find intracranial hemorrhage. We can look for pulmonary embolism. We can do pediatric bone age determination in patients with endocrine disorders. But beyond that, applications are relatively limited. The biggest success story, I think, that’s an exception to that is mammography, where it looks as though an increasing number of practices are taking advantage of the fact that AI in mammography can decrease the relative performance of expert mammographers who have had special training in mammography with those radiologists who are actually across the country reading the majority of the mammograms who are doing it in addition to other tasks that they do in radiology and are not necessarily specialists. And so for the largest radiology groups, particularly the outpatient groups, they are incorporating and embracing mammography AI as a way to bring many of those radiologists up to higher expert or to closer to expert level and some of the experts as a essentially checker on them. And I think that’s probably the most clinically efficacious application of AI in radiology at this point. So one is that it’s been relatively niche. The other is it’s been difficult to consume multiple AI applications. And so, we started to have a variety of different vendors, startups and others who are providing these AI applications, and consuming any one requires a business associate agreement and IT security issues and figuring out, do you consume it in the cloud or do you consume it locally? And sometimes it would take 6, 12 or more months to be able to actually add a single application, but it doesn’t scale. So when you go to 2 and 4 and 8 and 16 or more applications, you don’t really want to go through that process. And so what we’re seeing is the emergence of platforms with regard to AI applications where one is able to determine a platform and then that platform provides multiple different applications. And so it’s more similar to advanced visualization systems where you buy one advanced visualization product and you get many different software applications or with a major providers of imaging equipment. Such as GE and Siemens and Philips, Canon, et cetera. You know, you purchase a modality such as an MR, CT scanner, and you get a large number of bundled applications. And so with AI, I think we’re going to move increasingly to consuming those on applications where AI applications become much more like a menu and you add items to your platform. And so you can have one agreement with the platform provider rather than having to have separate agreements with each one of the AI provider. And so I think that’s been a challenge. The lack of standards for incorporating AI applications has been another issue. The other question is that there hasn’t been a huge amount of testing of AI applications out in the wild. And so AI applications are developed, trained, and tested locally. But then everybody has found a degradation in performance when those same AI applications are applied in other settings, with other machines, with other populations, with other radiologists and departments, etc. And so that variability has been something that has not been super well documented with regard to a consumer reports type of mechanism to be able to actually see how these AI applications are performing in other settings. And so I think people have been waiting a while to sort of see how some of all of that ends up sorting out with regard to the robustness and resiliency of these AI apps as they’re applied to other systems. And then, of course, people talk about drift, where AI works at a certain level. And then as time goes on, and people end up upgrading their software and hardware systems, or populations change, or other things change, the performance of the AI applications tends to drift downwards, also. So I think there is concern about the performance of the AI systems. And I think people are waiting to see how that ends up shaking out. And we have more and more radiologists now have been using AI applications who are sharing some of their experience with it. But there really is not any clearinghouse for determining the efficacy of these AI applications. The FDA currently tests the process for clearance of the AI applications and how that information was determined as far as the testing, training, etc. But it really does not at this point track the performance as post-drug surveillance ends up testing and getting feedback on drug performance after FDA clears it, and it’s actually being utilized. We don’t have that mechanism for the FDA currently, nor do we have anybody that’s really generally sharing the results of their AI applications outside of their own institution. And so I think questions about efficacy are part of the relative slowness of AI. The other issue, of course, is generalizability. I mean, it may be that when I look at a chest CT scan, there are hundreds of different things that I’m looking at as I review the study, including things related to the musculoskeletal system, the spine, upper abdomen, vascular system, etc. And so we don’t have a wide variety of different applications. We don’t have an AI that is generally able to learn how to interpret a modality, but the AI applications are very focused at this point. And so I think that that has resulted in relatively slower application of AI, plus the FDA clearance process, even though hundreds of applications have been cleared by FDA. I think some of the largest radiology groups are actually developing their own internal AI applications with their own developers, and then not going through the FDA process, but taking advantage of their own data within their own system and their own process for testing it. So I think there have been lots of challenges to AI adaptation.

Chris St John – 00:17:10: Yeah, for sure. And so with those concerns about efficacy and different solution suppliers, different business, different for profit businesses, I feel like there’s a lot of ethical implications that come along with that. Super complex, ethical implications. How do we try and address those ethical considerations, especially biases in AI algorithms? Or like, are there measures we should be putting in place? Like, how do we approach this conversation from an ethical conversation when we have all of these different voices contributing different pieces and different tools and different factors?

Eliot Siegel – 00:17:52: Yeah, so it’s a great question. I think, first of all, we should acknowledge that humans, radiologists, have their own biases. And so I think that these ethical questions have always existed, but they become more focused when we’re looking at a computer or a machine and asking about its own biases. I know that radiologists that have practiced in certain geographic environments and with certain populations, if we brought them into another environment, they would bring their own areas of bias with them. And so I think that’s part of the issue. The issue is interesting with regard to bias in that people talk about wanting to have AI trained with as much diversity as possible, but these training sets can be very challenging to acquire. And most of them are from a limited number of institutions that may not be representative of the whole U.S. or certainly global population.

Chris St John – 00:18:50: Right.

Eliot Siegel – 00:18:51: The challenge is I like bias in the sense that if I’m reading a particular patient who is in a particular geographic area, particular ethnicity, et cetera, I might want to have a training set and testing set developed for that population. If I see a lung nodule when I’m at the VA hospital, the likelihood that it represents cancer, for example, is different than when I cross the bridge over to University of Maryland because the patients have had different exposures. And so it may be that I want to actually simultaneously have a general set that maximizes diversity, but then specific data sets that may be specifically developed for certain subpopulations where that bias is actually built into the system. There are a number of other ethical concerns, though, with regard to AI. One ethical concern is what level of disclosure should a patient have as far as their study being interpreted? Utilizing AI? Should I know whether AI is being used or not? What happens if AI disagrees with the radiologist? Then the radiologist decides to ignore the AI. Should the patient be able to know what the AI said? Or is that something that should just be part of the decision tools of radiologists? We have one large radiology outpatient network that is giving patients the option of paying additional money to be able to get an AI opinion that’s given to their radiologist at the time of interpretation. Right. And so, you know, should the folks that are not able to afford that additional money have an idea of what the actual added value is of the AI? And shouldn’t AI be utilized in all patients? And so, you know, that’s an interesting question. Should AI be available for patients to use themselves so that they can get their own backup opinion and idea? I think that challenge just scares many of us because of the fact that, there could be an infinite number of non-vetted AI programs and patients often are not educated or sophisticated enough in radiology or healthcare to understand which ones are working, which ones aren’t, and some of the implications of those. And so there are huge ethical considerations. The whole ethical consideration of who should be able to have AI. Should it essentially be universal or should there only be a subset of practices that utilize AI? And then what is the responsibility if one determines that AI software is not working well to be able to inform clinicians and patients about that? And I think the whole idea of ethics in AI becomes a really interesting topic when we move this from humans to AI. And then, of course, there’s the question of autonomous AI. People have suggested that if AI could pick and choose the cases that it’s confident, and leave the rest to humans, it may be that it could read 90% of cases at a human or superhuman accuracy. And so should we allow autonomous AI? And then what are billing issues associated with all of that? And so these issues about whether databases to train AI have inherent biases that could essentially create a bias against the particular ethnicity or males versus females, that’s a general issue, that applies to healthcare AI systems in general. And so I think it’s a fascinating area. And the ethics, I think, are something we’re just beginning to scratch and understand the surface of. And of course, then there are medical legal issues related to AI. Right now, for mammography across the country, you may be surprised to learn that virtually all practices erase the AI marks that mammography or computer aided detection systems make on the images once the radiologist has had those. And that those are not accessible for clinicians or other radiologists or patients to be able to access. And part of the reason given for that is medical legal reasons, which is a fascinating topic. And so do we save and store AI markings moving forward in the future in mammography and other areas is an area of some level of controversy. And certainly you could have some pretty strong pro and con opinions. I participated in a debate at the Society of Imaging Informatics meeting a few months ago where we debated some of these topics about ethics of AI and responsibilities related to storing images and then using images with or without patient permission for training AI for the future. And what was interesting about the debate was we two debaters were told that there was a coin that was going to be flipped right before each one of the topics that we debated and we’d have to pick one side or the other. So it really underscored the point that there are good arguments to be made by everybody pro and con a lot of these different issues.

Chris St John – 00:23:59: Yeah, absolutely. I mean, especially, you know, like you were just kind of saying, especially as it applies to privacy, right? There’s almost too much for me to chew on right now for me to have any sort of formed opinion on it.

Eliot Siegel – 00:24:11: Yeah. One thing that’s important is to just remember that we’re focusing on radiology.

Chris St John – 00:24:16: Right.

Eliot Siegel – 00:24:17: But AI is emerging in so many different areas, and there are so many systems in the hospital that will be making decisions about who to treat, how to treat them, who to admit, who not to admit, what medications to give patients. And there are so many documented cases outside of radiology in particular at this point where decisions are made that seem unfair to certain populations or subpopulations that people are having to go out of their way to try to correct. And so I guess I just want to underscore that it’s not just radiology, but as we move toward AI in healthcare in general, we have to be really careful about treatment issues and reimbursement issues related to decisions that AI makes.

Chris St John – 00:24:59: Right. Yeah, it’s funny you say that about the AI podcast. I guess I was in college about 10 years ago or so. I was an art student. I was not pre-med or in this world at all. But as an art student, I did a, I guess it was performance art piece, like AI fear-mongering in about 2014.

Eliot Siegel – 00:25:22: Oh, cool. Yeah, I was doing debates back starting around 2016 when AI, awful predictions about the end of radiology and the end of healthcare with AI were being made. I was actually defending humans against AI, even as an AI researcher and as an AI, you know, author, et cetera. It’s been a really cool debate, and it’s certainly heating up lately, even more than back in 2016. In 2016, people were talking about, well, maybe we can replace a couple professions like these lowly radiologists and a couple of others, and now everybody’s. You know, scratching their heads as to, you know, what it. Can and can’t do and will be able to do in the future. So it’s been really fascinating. It’s been fascinating watching the journey of AI as kind of somebody that was very deeply involved in it before it really became popular. And so it’s been great to see the journey and to be able to still see what’s coming out from a cutting edge perspective. That podcast AI really, is technology that’s just been released in the last many hours or so, I believe. Right.

Chris St John – 00:26:39: Yeah, I mean, I first heard the term singularity in high school, I think.

Eliot Siegel – 00:26:43: First of all, yeah.

Chris St John – 00:26:44: And yeah, exactly. And so since then, I have been, fear is the wrong word. I’ve always been like so trepidatious about it and so curious, but also a bit. Yeah, I’ve just been like, this is crazy stuff.

Eliot Siegel – 00:26:58: Yeah.

Chris St John – 00:26:59: And it’s super fascinating. And now to be at Imalogix, you know, working at a company where we’re doing, you know, it’s AI powered. There’s lots of machine learning stuff, but. You know, still far and away from singularity or an AGI, but it is funny to find myself here career-wise after years of spouting fears of singularity.

Eliot Siegel – 00:27:19: That’s cool. You know, when I was a 10-year-old kid, I read a book called The Moon is a Harsh Mistress, and it was by Robert Heinlein. And in the book… What happens is there’s a computer network. In this futuristic sci-fi novel on the moon. And as it gets more and more data, it starts to become self-aware and it starts to essentially develop a personality. And, you know, the whole idea as a 10-year-old for me. Of a computer becoming self-aware just by adding more computers and more data, you know, seemed really strange. It didn’t seem logical how that could happen, but it was really intriguing to me, you know, how a computer intelligence and essentially personality could emerge just by having more data. And, you know, I never really would have believed necessarily that here we are in 2024 and just mining data that’s available, but vast amounts of it have really created emerging functions and features in the software that the designers really had no idea that would emerge. And I think that’s really fascinating. And it really, in many ways, mirrors that novel that I read as a kid that really turned me on to computers in the first place.

Chris St John – 00:28:44: Right. And so as these algorithms do continue to evolve, and as we continue to debate about the ethical implications of them, the human element is still present throughout all of this growth, through all of this planning, through all of this debate. So how do you see the role of radiologists evolving with this technology, and are there ways that they can adapt their skill set and knowledge to complement AI technologies? I mean, do you think they’ll be assuming new responsibilities? I asked a bunch of questions. I’ll let you take them.

Eliot Siegel – 00:29:19: Sure. So let me take a couple of those concepts. One is the idea of working smoothly with AI. And I really think it’s going to be a partnership where AI will help us to become safer and better and hopefully even more productive radiologists as time goes on. I have a car that is full self-driving. And so I can pretty much put in the location that I want the car to go to, sit in the driver’s seat and pretty much watch the car do practically everything. And as time has gone on over the last probably 10 years, I’ve been able to watch as the software has gotten better and has evolved. And I know, you know, the 5% of time that I want to take over and I know what it does well, and I know what it doesn’t do as well. I’m always vigilant though, because it’s not perfect. It does make mistakes, but it has helped me to avoid accidents. Where I’ve had a deer run in front of the car, or I’ve had two cars ahead of me stopped in the car had faster reflexes than I did. And so I think this partnership that, you know, where the software is constantly learning, I’m learning what my partner, the AI program does really well when I can relax when I need to take over and when I just need to watch it carefully. And I think it’s a good metaphor for what will happen in radiology. Also, I think that it’s really critical as time goes on. That we end up understanding what are the things that AI does really well, and then change the focus of what radiologists do rather than, you know, looking at every rib for a rib fracture on a CT scan and a trauma patient or hunting for every small lung nodule meticulously on every study that I read with a PET CT or CT study. You know, the idea of spending an increased amount of time looking at the perspective of what is the patient’s background clinical information? What are the implications of this? There was an article in the Harvard Business Review that suggested that AI was going to help tremendously with decision support and analysis and pattern recognition, but that judgment would still be under the domain of humans for quite a while. And so the idea of what does this mean is really important. As I work at the University of Maryland training residents, I find that as they go from first to second to third to fourth year radiology residents, then into their fellowship, they first learn the anatomy, then they learn how to make findings about what’s abnormal. But what takes the longest is for them to have the perspective on what do these findings mean? What are the implications in comparison to what their history is? And what are the implications with regard to comparing with the trend of that particular patient over time? And, you know, a greater knowledge of medicine than just making the findings to understand what are the implications. And I think humans are going to be better than computers at that for quite a while. So I really see a close partnership, but I want to spend more time in radiology training our residents on how to get information from the patient’s chart and prior studies and have more generalized information to figure out what are the implications of these findings? What are the most important things? And maybe spend a little bit less time going over some of the more rote sorts of memorization, you know, of differentials of basal ganglia calcification. Some can name 17 different items there. I don’t want to have them spending the time doing that. I want them to essentially understand how to interpret from a judgment perspective, what are the implications of what they’re seeing and to make cogent clinical recommendations that do best for the patient. So, you know, I really believe that the role of the radiologist is going to change as time goes on. And, you know, part of what I’m alluding to is the fact that AI is not only something that looks at patterns in medical images to make the findings, but AI is now better able to read and synthesize the patient’s electronic medical record, look for findings in prior examinations. It’s able to better make recommendations based on the imaging literature and the medical literature to help radiologists who are sitting in the cockpit essentially have excellent advice and excellent access to information. And so I’m really excited that as you’re implying the role of the radiologist of the future is going to change, I believe, fairly dramatically in the direction of one who coordinates information associated with images and is much more able to contribute to overall patient care and outcomes than we may have in the past because we have that additional information. We have those insights. And so I think AI, not only making findings, but also extracting information that I need that’s relevant, helping me to make decisions and then communicating those decisions to the right people at the right time when the information is needed. So much information that we create in a radiology report is lost in the words in the report and not discoverable. And so making radiology reporting and information machine intelligible, so other AI or other specialists are able to consume that. I see all of those things happening in the future. And I think it’s really exciting. I mean, I would be really excited to be a radiology resident today with all of the things that are coming on board. I remember a few years ago when there was the scare, maybe eight years ago, and people were giving talks such as at the American College of Radiology meeting about the end of radiology and AI was going to replace radiologists. And I think what we’re finding is that AI really will help us reinvent radiology. And when fathers who were radiologists were asking, should their daughters and sons go into the specialty based on where I see things in the future? To me, I would even more strongly recommend radiology now knowing how it’s working with AI. And the emphasis is going to be much more on the things that I wanted to go into medicine for in the first place, which is being able to add my overall judgment and perspective to help patients.

Chris St John – 00:35:45: Yeah, it’s cool. The more and more I’m talking to folks about this, really, I feel like when I’m reading a lot of the discourse on this, things have a tendency, at least in writing, to come across as very black and white, right? As you were saying, like, oh, the AI is going to replace radiologists. Like, okay, great. I love the way you’re talking about this as a collaboration and as a partnership. It’s remarkably comforting to me as someone who, you know, I have a messed up shoulder. I’m going to have to get imaged probably this week. You know, it’s so fun and so interesting to think about this collaboration between us as humans with a human intelligence collaborating with the machine intelligence rather than just choosing one or the other, right?

Eliot Siegel – 00:36:30: Yeah.

Chris St John – 00:36:31: Like using it as a tool to better the human side of medicine.

Eliot Siegel – 00:36:36: Right. Let’s take your messed up shoulder, for example. I mean, your messed up shoulder is associated with a history of signs and symptoms that you’ve had as far as how it’s quote unquote messed up. And at some point you’ll get images of that. And those images need to be interpreted based on what you want to do or unable to do with your shoulder or the pain that you’re having when you’re having that pain. And then as time goes on. The question is, should you have an intervention? And if so, what should that intervention be? And so is your radiology report just going to comment essentially on whether or not you’ve got a tear or whether or not there is tendinopathy and, you know, the status of your shoulder in multiple anatomic ways? Or will there be information saying people who had this constellation of signs and symptoms and findings typically did well with this? Should there be a follow up? Examination after an intervention that you had? What is the best combination of ways to work up those data? And how can we best contribute? The difference between somebody who’s just looking at your MR and basically making the findings and just describing that, I think, is a lesser report than somebody interpreting what happens with that constellation of findings and what interventions have been done, what the implications were for those interventions. Not to supplant your orthopedic surgeon. But essentially to provide that orthopedic surgeon with deep insights as to a combination of findings that are made in the MR of your shoulder and related to what they end up seeing and doing clinically. So, you know, it’s really communication and collaboration with your orthopedic surgeon to figure out what are the features based on your signs and symptoms that one ought to be looking for in your MR? What are the most important things to report and share? And what? Insights do we have in our AI database to help guide those decisions in addition to the personal expertise of your surgeon with the entity that you end up having and then figuring out the probabilities? And at some point, your orthopedic surgeon will be interfacing with her AI software and talking and making an AI combined recommendation for what you should do next.

Chris St John – 00:38:56: Yeah, it’s like a giant game of human to machine to human to machine to human to machine, like, whisper down the lane.

Eliot Siegel – 00:39:03: Yeah. And that gets back to the whole idea of platforms where there’s the potential of ensembles. And so you might have three AI programs essentially combining the findings from your MR of your shoulder and saying, here’s the level of confidence. And so maybe what one misses, another one picks up, or maybe what one calls, another two end up saying, no, that’s probably not the case. Or could you have AI software that essentially does a subset of findings, and then somebody takes a look and wants to see whether or not there’s an AI program that specifically looks at the cartilage in your shoulder, determine whether or not there may be some quantitative metrics, and then that would turn it over to a second AI program that would be launched by the first system. And so you can imagine, as you said, AI talking to human, human to AI. AI to different AI, lots of possibilities. And I think these platforms of communication should allow us to have collaboration with multiple AI systems and multiple unions as well. And ultimately, hopefully, we can get you back to 100% with your shoulder.

Chris St John – 00:40:11: We’ll have to see about that. It’s going to be a long healing road. I made an oopsie.

Eliot Siegel – 00:40:15: Yeah.

Chris St John – 00:40:15: So thinking about this now in terms of like the ensemble, right? Various AI systems and human and AI teams talking to each other.

Eliot Siegel – 00:40:25: Yeah.

Chris St John – 00:40:26: It gets me thinking a lot about different sets of training data and the quality of training data going into these various machines.

Eliot Siegel – 00:40:36: Yes.

Chris St John – 00:40:37: What are the best practices for curating high quality and diverse data sets to train these AI models, especially if they’re going to be working in an ensemble?

Eliot Siegel – 00:40:47: Yeah. So I used to, during the time that I was also in charge of radiology at the VA and vice chair and professor at University of Maryland and did work at both institutions, I also spent time at the National Cancer Institute and I had responsibility for imaging informatics. Specifically, I had the opportunity to lead an effort called the CABIG, the Cancer Bioinformatics Grid. And as part of that project, we were tasked with creating an online database of lung nodules. And so there was a collection of those lung nodules. We wanted to share the collection, allow people to download those. And as time went on, we realized, well, we not only wanted lung nodules, but we wanted prior cases where there were prior nodules so we could look at the evolution of those nodules. Then we thought, well, let’s make this a general purpose archive for cancer. Let’s take breast MR studies and brain MR studies and PET studies. And so that ended up evolving into the National Cancer Imaging Archive. And so that archive has been utilized by a huge number of software developers. And much of the software that has been developed in AI has come from that huge dataset. But setting up a dataset like that can cost tens or hundreds of millions of dollars. Each one of those cases is incredibly carefully collected, annotated by multiple annotators, stored, vetted, scrubbed for any personal health information. And so it’s a really expensive, really slow process. And so individual institutions, some of the largest academic institutions have created their own archives from their own data. And there’s been some sharing, but I think concerns over privacy and security have slowed that down. What I would like to see, is for us to take better advantage of sort of non-completely annotated or non-completely supervised, unsupervised or semi-supervised mechanisms of learning where we don’t formally have to have somebody retrospectively analyze or annotate each case. But every time a radiologist reads a case, it essentially self-annotates. And so it may be that radiologists end up circling or highlighting certain features. As a routine part of their image interpretation, even if they’re not doing that, it generates a rapport. If I decided that one of the areas that I’d like to polish up on is MRI cardiac image interpretation or MR pituitary image interpretation, I might go back in my own PAC system at my own institution and call up the last 100 cases in that area and look at the images and then see what the report looked like. And then look at the images again and kind of teach myself from the archive that was there. And as a human, I can essentially look at the findings, look at the report, and as time goes on, learn. And I can generalize and pick things up with a relatively smaller number of cases than most convolutional neural networks or transformer systems can do. But those are getting more efficient and able to learn with smaller numbers of cases. And so the point I’m trying to make is that the way residents and fellows and students learn is by watching an example, seeing a case, seeing how it gets reported out. There’s no doubt in my mind that as time goes on, we are going to be teaching our AI algorithms, utilizing that. The question is, can we extract information from our reports that’s machine intelligible that will allow AI to learn, realizing that it may be imperfect, realizing that sometimes it’s going to get it wrong, but if you have a large enough sample of cases, just as is the case with a human, that eventually with larger numbers, it will get it right. And so I believe that there will be a future that every time a study is interpreted by a radiologist, there will be some notation of whether they’re a specialist or subspecialist of what their relative performance is. And then AI will essentially constantly learn from feedback of radiologists. And then you’ll have radiologists with AI that are better, and then their results will train AI in the future. And so there’s no doubt in my mind, especially with the emergence of generative AI, which we haven’t really even talked about in large language.

Chris St John – 00:45:24: I know, right?

Eliot Siegel – 00:45:26: Maybe for another time.

Chris St John – 00:45:27: Yeah.

Eliot Siegel – 00:45:28: But what we’re finding is that we actually are able to self-annotate studies from the reports and from some marking. And so part of the routine clinical practice is going to be the annotation of the future. We’re super indebted to all the radiologists and experts that did that annotation to get things kicked off and started and super grateful for the hundreds of millions of dollars spent on data sets like RIDER and LIDC and the lung cancer nodule database and others. But I think as time goes on, we’re going to figure out how to make routine clinical practice its own annotation. And at that point, it’s really going to take off. You mentioned the singularity. I think we’re going to see massive improvements in AI for radiology once we figure out a way to create that continuous learning loop.

Chris St John – 00:46:19: Right. Yeah. So looking ahead then, I mean, we were kind of touching on this, but what do you see as the potential for transformative changes in medical imaging over the next 10 years or so? Are there any emerging trends or technologies that you believe are going to redefine how we approach diagnostics and patient care?

Eliot Siegel – 00:46:41: Yeah, I think the biggest technology trends are going to be the emergence, and we’re starting to see this already, of AI that learns with smaller numbers of cases or even potentially a single case. I think we’re going to see, as I mentioned, more utilization of imperfect training, but large amounts of that training and more sophisticated extraction of meaning or semantics from radiology reports. I think that large language models are really going to help us do that. The other thing that I think we’re going to see in the future is the ability to look at trends of change of imaging studies over time. Right now, amazingly, or maybe not amazingly, we’re still in sort of the early Flintstone era, I think, headed slowly towards the Jetsons. But right now, AI mainly looks at one current imaging study, whereas as a radiologist, except for emergency department type cases and limited other cases, my goal is to say, are things getting better or worse? What’s changing over time? Cancer care, mammography, especially looking for malignancies is so dependent on change over time and decisions about treating patients with a variety of medications and cardiology, various aspects of surgery, and certainly in oncology are based on change over time. Right now, for the most part, the AI algorithms have not been trained with data sets that essentially have change over time. And I think as we get to extracting routine clinical cases, we’ll have training data sets that allow you to see that. So a lung nodule that I’m evaluating, if it hasn’t changed in 15 years, I know that it’s highly unlikely to be malignant. Whereas if I’ve just met it for the first time and I’m pretending that I have to make my interpretation on that one case, I may have a much higher level of uncertainty. And so I think this is going to be a huge trend in the future, is to look at trends over time as images change. And I think it’s going to add a lot. I think adding to decision support about whether a study’s positive or negative, looking at the history is going to be really important. And we radiologists, we humans use that information all the time. AI at this point, for the most part, is not utilizing that. So I see so many changes occurring as time goes on with the technology, with the ability to be able to generalize to a much greater degree on smaller number of cases, to have routine image interpretation part of the process to look at trends and change over time. And also, I think that we’re going to see changes in FDA clearance, allowing some of these things, such as looking at change over time to be incorporated routinely into the AI algorithms. I think those will all really significantly accelerate things. I think in 10 years, I’m really confident that there will be much greater utilization of AI. By radiology. And I think it’s going to change our ability to spend more time communicating findings and also utilizing our judgment and making recommendations from this.

Chris St John – 00:49:58: Absolutely. And so another thing we’ve been talking about a lot on the show in general is, you know, understaffing in the field of radiology right now. And so looking to the future, right, if you were going to address young professionals, young researchers who are just getting into the field of medical imaging right now, particularly those interested in AI, what skills or perspectives would you recommend that they really focus on cultivating as they are getting into this field right now?

Eliot Siegel – 00:50:29: Yeah. So I think what people need to cultivate is not so much learning about the intricacies of convolutional neural networks and transformers and the math associated with that. I think there will always be a subset of people who are technically interested in those topics. I think what really is critical is figuring out how do you best utilize the tools? What does it mean if AI tells you that it’s suspicious that there’s intracranial hemorrhage? How can one then get AI to show you where it thinks it’s there, what its level of confidence is? How can you figure out what are the best AI tools? What are the questions to ask when you purchase AI as far as the training sets and predict what likelihood they will be relevant? How can you essentially assess your own performance or those of others with AI to make it so that you constantly get better and learn from AI? And one of the things we haven’t talked about, which people are discussing with the FDA, and the FDA’s is created, a white paper on is AI that learns. Right now, the AI that comes out of the box is static. And so even when I disagree with it and tell it I disagree with it or the platform, it doesn’t learn at all. It just maybe records that information or sends it back to somebody for development in the future. And the idea of residents and fellows and other students that constantly learn and get better are a lot less frustrating because once I’ve made a point yesterday, in all likelihood, They’ll remember it today, whereas AI right now is static. And so until there’s a new version of the AI that comes out, then I’m stuck with that same version and having something that continuously learns. So maybe it gets programmed out of the box with certain capabilities, and then it learns and customizes to the way I like to read or to my own patient population or geography, et cetera. And so I think that’s another thing that I’d like to see in the future. And having radiologists learn about the importance of how to best utilize AI, what types of, what’s important in AI, for example, looking at change over time and asking for that feature, looking skeptically at what works and doesn’t and how one can assess it. I think those are all things that we need to start training our residents about, and then how to be skeptical about AI and, you know, when to turn AI off. I mean, in my car, when it looks like the AI is not doing what I want it to do, I can quickly tap the brake and all of a sudden I’m in full control. You know, letting the radiology residents understand when to use it and when not to are really important. So I think people who have suggested that there needs to be a lot of technical training about data science and how that’s done, I don’t think we really need that any much more than driving those driving a car need to be educated in the specifics of, the neural networks. I think people who are trained in driving a car need to understand in what circumstance, you know, can you turn it on? When do you use it for parking? When don’t you? And how can you set the various features and modes to optimize the way that you want to drive? And so I think keeping that same analogy, there’s a huge amount of training that we need. And right now, I think we’re vastly under utilizing AI. And I think we need to start training our residents and fellows and others into what’s available and how to use it. And even if a particular institution doesn’t have AI implemented, I think training is going to be really important because they’re going to be spending their careers utilizing it. Right.

Chris St John – 00:54:11: And speaking of skepticism… As we’re starting to wrap up here today, as I think I mentioned to you, this whole field is all quite new to me still. So as a podcast host, as someone who is getting educated on all of this, AI or not, do you have any recommendations for me as I’m continuing to talk to folks? Or do you have any tips on a healthy skepticism that I should be having as I’m progressing and learning more and more about this field?

Eliot Siegel – 00:54:42: Yeah, I think part of the healthy skepticism that you should have is we’re all kind of, we all grow up, we’re all programmed, we all learn as time goes on to be skeptical about other human beings. You know, you meet somebody and, you know, maybe they’re being 100% honest, maybe they’re not, maybe they know what they’re talking about, maybe they don’t know what they’re talking about. And as time goes on, you kind of get to know them and essentially learn that. And I think people tend to think because something is based on a neural network, or it’s called AI, or it’s FDA cleared, that, you know, it really is going to do what it claims to do. We don’t put AI through the same board examination and same testing program that we do humans. And I think as you’re looking at the use of AI, it’s really important to kind of get to know it and to be able to sort of test it in routine use and kind of at the edges also. And I think maintaining that healthy skepticism is important. And I think, you know, having it essentially, we’re exploring the edges of what it does and what it can’t do and trying to see, you know, where it works, where it breaks, where it doesn’t is really important. The other thing that I think is important is that we look at its impact on performance. And so, you know, is it making us less efficient or more efficient? One of the concerns I had when we created the world’s first digital radiology department, was maybe people will spend all day adjusting the brightness and contrast of the images and zooming and roaming through an image, because you can do that infinitely. And what we found is that it actually increased efficiency and productivity. And so I believe that now, in general, AI is actually slowing radiologists down for the most part with most applications. I think in the future, we’re going to learn how to use it more efficiently, how to trust it, and how to have it do things that are essentially mundane and repetitive, like bringing images up, arranging the images, bringing the prior reports, extracting information from those reports, communicating findings, generating a report. You know, in studies that we did early on at the Baltimore VA, we found that radiologists only spend about 15% of the time it takes to interpret a study, making up their minds about what the findings are and what they’re going to say. The rest of the time is waiting for things to happen. And all sorts of other efficiencies in the process of arranging images, extracting information, reporting, communicating, and then waiting for the computer to do something else. And I believe there’s tremendous potential to increase that 15% to 70, 75% or so, which could create enormous improvements. People are looking at the question of how do you pay for and justify AI and having AI as a spell checker or grammar checker, or background checker is really helpful. It could increase confidence. It could decrease the difference between experts and, you know, less expert folks. But what people will really pay for is something that increases their efficiency and productivity while improving their relative accuracy. And at that point, it really becomes super cost-effective.

Chris St John – 00:58:00: Amazing. Well, I think that’s probably a great… Little bow to put on top of this.

Eliot Siegel – 00:58:07: Awesome.

Chris St John – 00:58:08: Dr. Siegel, thank you so much for joining me today. It’s been truly a pleasure, and I hope, if you’re willing, maybe we can have you back one day.

Eliot Siegel – 00:58:15: Sure, love to. And thanks for the really engaging discussion and the opportunity to share some of my thoughts about AI. It’s, I think, the hottest and most fascinating thing in diagnostic radiology and in medicine in general. And I think it will fundamentally allow us to provide better care and safer care for our patients. So it’s really a pleasure to have the opportunity to chat about it.

Chris St John – 00:58:36: Amazing. Thank you so much.

Eliot Siegel – 00:58:38: You’re welcome.

Chris St John – 00:58:41: Frame by Frame: Rethink Imaging is brought to you by Imalogix. Here, you’ll find engaging interviews with thought leaders, experts, and patients, sharing stories that showcase the transformative power of medical imaging. To discover how Imalogix is rethinking imaging in healthcare, visit imalogix.com. Be sure to subscribe to Frame by Frame: Rethink Imaging on Apple Podcasts, Spotify, or wherever you listen. And from all of us here at Imalogix, thanks for tuning in.