Deventer Hospital

Paving the way with AI at innovative Dutch hospital

Sectra interviewed innovation manager Wilco Kleine and ICT project manager Julia Larsen of Deventer Hospital. The focus of the interview was on their accomplishment in completing the proof-of-concept phase for BoneView, a Gleamer AI program that helps radiologists identify bone fractures. BoneView runs via Sectra Amplifier Services and is integrated into the Sectra PACS. To ensure the success, the pilot project was divided into the following phases: market analysis, retrospective study for local validation, integration testing in shadow mode, and application in clinical practice.

This local validation of AI in radiology is a learning process—one in which both seamless integration and the work processes of radiologists and other hospital staff play an essential role. In the interview, we talk about how AI can be used in the best possible way, and how this process should be structured.

Deventer Hospital will be using AI for various purposes, including reducing workload and improving patient outcomes. In addition, the logistical use of AI will support less complex assessments of an acute nature, the goal being to optimize emergency care, better distribute the workload and improve waiting times for patients. For instance, the BoneView application will allow automation of the initial detection of bone fractures.

According to Kleine, “This means a medical specialist is no longer required to assess potential bone fractures when someone comes into the emergency department (ED) at night. AI can perform the initial assessment, which the radiologists then double-check the next morning.”

In addition to being used for acute assessments, Kleine says AI can also help with complex assessments. As an example, he mentions an application for assessing pulmonary radiographs.

“To reduce the possibility of something being missed even further, we could use AI to work with the radiologist as an additional assessment, a second pair of eyes, so to speak. This review would then focus on “improving the quality of the assessments we perform. To select AI tools from the market, we chose to partner with Sectra Amplifier Services, which has a large range of validated AI tools available.”

Local validation of clinical application

Following the initial decision to adopt BoneView, the hospital did not start using the tool in clinical practice straightaway. Instead, a local validation using local data was an important step in instilling confidence in the results. This study used about 500 examinations from the hospital’s own PACS that had been reviewed by a radiologist at the hospital in recent years.

Kleine: “We compared the algorithm’s assessment with the radiologist’s assessment. How do these actually compare?” The resulting confidence rating depended on the composition of the local patient population, images, X-ray equipment, and radiologist assessment. After all, this rarely matches the parameters of the developers’ study exactly.

It’s not just about the total package of stability, speed, and performance of the AI application, but also the PACS around it.

Wilco Kleine, Innovation Manager, Deventer Hospital

Technical testing to ensure performance of the AI tool

The next step was technical integration. “Once we had verified that the AI application is reliable enough to deploy, we needed to assess whether the application also does what it promises in the workflow,” Larsen says. “One of the factors for success when it comes to clinical application is usability in a setting such as the one we have here in the emergency department. That means reviewing the images at the appropriate speed and not having a 20-minute delay.”

During the technical test, the AI application ran in the background for a week without radiologists being able to see and therefore act on the results. The technical test examined a variety of questions: Are all images processed? Do all results come back within the intended time limit? Is the speed not only high on average, but also without very slow exceptions? And are there any adverse effects on the rest of the system? Kleine: “It’s not just about the total package of stability, speed, and performance of the AI application, but also the PACS around it.”
AI applicatie Gleamer Boneview in Sectra PACS AI applicatie Gleamer Boneview in Sectra PACS
The guideline depends on the use case. “If you want to use it alongside a medical specialist, you apply different criteria than when you need to treat patients quickly. For instance, do you allow time for the AI application to develop within the clinical setting?” said Larsen. “We are a top clinical hospital, and this means that an application must be directly applicable in clinical practice and have proven reliability.”

For AI, the hospital does not want all kinds of stand-alone solutions, with their own technical interfaces. Rather, we want one connection: to the PACS. That way, we can easily switch to another AI tool. We want to avoid this becoming a complex technical issue every time.

Wilco Kleine, Innovation Manager, Deventer Hospital

The power of seamless integration

Larsen stresses the importance of seamless integration: “It is crucial that the AI application fits well into the current PACS workflow. Image assessment takes place inside the PACS. Precisely because of the [intended] efficiency, it is necessary to remain within the system. After all, you don’t want employees to have to click through several applications in order to make good decisions. The Sectra Amplifier platform helped with this.”

In addition to seamless integration, Kleine points out that “for AI, the hospital does not want all kinds of stand-alone solutions, with their own technical interfaces. Rather, we want one connection: to the PACS. That way, we can easily switch to another AI tool. We want to avoid this becoming a complex technical issue every time.” This in turn would also affect the performance and stability of the workflow, “so we deliberately chose one integration layer.”

The human factor in technological transformation

The human aspect of changing the way we work is at least as important as the technical changes themselves. Kleine and Larsen emphasize that this is primarily a project for healthcare professionals. Working on a multidisciplinary task force gave everyone a voice in the project. “That meant that the radiologist, lab technician, IT specialist, and department manager, all of whom had to guarantee this, were in the task force,” explains Larsen. Also, the structure of market analysis, retrospective research, working in shadow mode, and the pilot itself allowed plenty of space for developing trust within the team.

As Kleine explains, “Our team is made up of highly trained professionals who, understandably, want to be convinced of the validity of an AI application before they are willing to rely on it in their daily work. There are high demands in terms of accuracy, stability, and configuration. If we are too quick in introducing an application that has not yet been fully developed or has not been properly integrated into the PACS, this will only lead to dissatisfaction.”

The evaluation by the lab technicians after the pilot ended showed that a vast majority of them were positive about the use of BoneView and were eager to continue using it. Both during the day shift and in the evening and night shift, BoneView helped in the assessment of scans, with the result that lab technicians did not have to ask the radiologist for an evaluation as often. This, in turn, resulted in fewer disruptions to the radiologist’s work. For the patient, this means getting answers more quickly and less time spent waiting.

Vision for the future and recommendations

Deventer Hospital sees itself primarily as an “early adopter” or “fast follower”. As Kleine explained, “If the AI application is developed within an academic setting, we, as a top clinical hospital, can test its effectiveness in clinical practice and see how further use in smaller, local hospitals can be achieved properly.” After BoneView, the hospital will also be conducting pilots with other AI applications, for example, for detecting pulmonary nodules and assessing cerebral hemorrhages on brain scans. For this, structured integration into the workflow will always be essential.

“Ultimately,” says Kleine, “projects of this kind are always about collaboration between AI and the healthcare professional, with quality and efficiency for the patient at its core.”

In closing, Larsen notes AI’s growing importance in healthcare. “Opinions are divided, however, on how AI will find its place in clinical practice. In healthcare, we have to put the human element first. For this reason, it is good that healthcare professionals have the final say when it comes to application. They are the best judges of whether and how AI should best be used.” With that, it remains important for hospitals to keep the goal for each application in mind, such as working more efficiently, or better patient outcomes.

Featured products & services