A short term strategy to reap the long term benefits of deep learning and integrated diagnostics

Integrated diagnostics in the short term—the sharing of data

Radiology and pathology are both information specialties that differ from many other areas of medicine in numerous ways. The most fundamental characteristic of their work is that images (or slides) are the information bearers for their conclusions on diagnosis and treatment alternatives.

Radiology and pathology are very interdependent since findings in one discipline often strengthen or reject the other’s hypothesis. For example, a lesion identified by a radiologist may be confirmed or rejected as malignant by a pathologist.

Because the two disciplines are both making decisions heavily based on information extracted from images where findings are correlated, the enhanced sharing of images and information between the specialties would improve diagnostic procedures. For example, a pathologist that could access a patient’s associated radiology images could determine if the biopsy is representative for the lesion that the radiologist detected. Another example is to enable a rad-path discussion before the tumor board to decide on additional biopsies instead of taking that decision during the actual meeting, thereby avoiding a delay in the treatment procedure and avoiding time wastage during the meeting.

Today, shortages in information sharing between these two disciplines lead to inefficiencies, create frustration, and result in an incomplete patient overview. A short-term opportunity with integrated diagnostics is to enable access to the other department’s information and images, thus improving patient care and efficiency.

Integrated diagnostics in the long term—making use of deep learning

Deep learning (DL), has made a tremendous entry into medical imaging in terms of research achievements over the past number of years. Previous generations of machine learning efforts in medical imaging have been based on image content characteristics defined by humans, thereby limiting the input to what humans usually consider. In contrast, DL learns itself what details of the image are relevant based on labeled training examples. DL is based on artificial neural networks with many layers (hence ‘deep’), and the more layers used, the greater the degree of complexity that can be captured by the algorithm. The pace of advancements in the DL field is extremely fast, and many believe DL will soon be ready to be applied on a broad scale in healthcare routine practice.

Integrated diagnostics is one application where DL will have a significant impact. There is no doubt that great value lies in applying DL to the combined rad-path information used in integrated diagnostic workflows as a powerful decision support. It will also further increase the value of the shared information by recognizing patterns in historical data and guide radiologists and pathologists to the information that is relevant.

Don’t get me wrong, DL will never replace the radiologist or pathologist, however it will free up more time for tasks where the human capability can provide the most value and cannot be replaced by a machine. The role of pathologists and radiologists will move from extracting information from images to making advanced holistic analysis based on the information extracted by DL. The machine-human combination could diagnose more patients at a fraction of the cost and, in some cases, with higher accuracy. For example, recent research results indicate that computers could predict the grade and stage of lung cancer better than pathologists (1). The potential is huge, but barriers still remain, such as the legal aspects regarding responsibility when a machine is conducting more parts of the diagnostic procedure.

A successful integrated diagnostics workflow would balance the tasks between machine and human and thereby enable radiologist and pathologist skills to be scaled out to benefit more patients and cover not only groups of individuals, but also entire populations. It is my firm belief that at some point in the future healthcare providers that have not yet adopted integrated diagnostics tools facilitated by DL will not be able to live up to patients’ rising demands for shorter lead times and higher quality of care.

For integrated diagnostics to deliver its potential benefits to cancer care, it is crucial to utilize DL technology to take into consideration the many variables and dimensions of radiology and pathology data that would be impossible for the human brain to process.

A schematic picture illustrating a potential pathway of integrated diagnostics adoption.

Is there a medium-term integrated diagnostics?

To apply DL on integrated diagnostic workflows will take time, mainly due to the necessary validation of algorithms, the change of the existing reimbursement schemes, accreditation, tradition and the overall mindset in healthcare. It will also take time to find an appropriate balance between human-machine collaboration on one level and rad-path collaboration on the other.

A medium-term step to bridge the gap between short-term rad-path information sharing and benefit from long-term deep learning opportunities could be to start with rad-path image correlation and orientation tools. These could, for example, facilitate tasks such as determining where in the tumor the biopsy is taken. One example on this topic is to link the pathology image to the MR guided biopsy image, something that a few studies are already investigating today. Such correlation would be very beneficial to detecting sampling errors and to identifying different tumor mutations.

Correlation and orientation tools would utilize a semi-automated approach where input from the radiologist or pathologist would be needed. The application would learn to increase its precision over time to better recognize patterns and structures in the images to synchronize images from different modalities. Such a medium-term step would create an incremental adoption path, from image and information sharing across department borders to a full-scale integrated diagnostics that utilizes DL.

Lay the fundaments today for fully integrated diagnostics utilization in the future

One reason why integrated diagnostics is being so heavily discussed right now, even before DL is fully ready to be applied in routine practice, is that for DL to be applied to integrated diagnostics data, the data first needs to be put into an integrated storage. The difficulty with DL today is not the analysis itself but the access to high-quality structured data. Therefore, it is crucial for healthcare providers to enable cross-departmental access to patient data in order to be prepared to take advantage of the future benefits of integrated diagnostics.

It is a bit like when railway companies expand the existing railway network with rails that are designed for high speed trains that will be launched 10 years from now. Old trains can use the new tracks and thereby benefit today’s passengers, but the true benefit comes in the future when the old train fleet is replaced by trains that are twice as fast. If the railway network is prepared for future trains, this means that those fast trains can be introduced much earlier than if the entire old rail network is left to be replaced overnight.

One cost-efficient way to ensure that the healthcare organization will be ready to leverage future DL advancements and, at the same time, draw benefit from integrated diagnostics and the shared images and information today, would be to utilize the same infrastructure for digital pathology as for the radiology PACS. That would not only give bi-directional access to information, but minimize costly integrations, better utilize the existing IT infrastructure and create a common interface toward both disciplines, which would ease collaboration. Storing radiology images and pathology images in the same system makes it easier to ensure that the information from both disciplines is connected and accessible for each patient.

Integrated diagnostics will revolutionize cancer care, especially when deep learning can be applied to help process the enormous amounts of data. Until then it is crucial to establish data sharing between radiology and pathology to build the fundament for future DL utilization but, at the same, be able to take part of the early wins of correlation and orientation.

Inspiration & sources

1. Rubin, Daniel L and Snyder, Michael (2016), Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features, California, USA:
   http://www.nature.com/articles/ncomms12474
2. Jackson, Whitney L (2016), Q&A: The Radiology-Pathology Merger:
   http://www.diagnosticimaging.com/practice-management/qa-radiology-pathology-merger
3. Saurabh, Jha and Topol, Eric J (2016), Adapting to Artificial Intelligence Radiologists and Pathologists as Information Specialists, Department of Radiology, University of Pennsylvania, Philadelphia and Scripps Research Institute, La Jolla, California:
   http://www.diagnosticimaging.com/practice-management/qa-radiology-pathology-merger