Lakera co-publishes article in a Nature journal on testing medical imaging systems.

This article was originally posted on our company website. Lakera is an AI safety company that enables the future where humans can trust AI.

At Lakera, we believe that AI will give rise to completely new types of medical applications. We have the potential to diagnose diseases earlier and more accurately than ever before. At the same time, companies that want to innovate safely, have to put reliability and trust at the core of their development and operations. Not such an easy task!

Before these AI technologies are put into hospitals, it is important to evaluate their predictive performance and obtain regulatory approval. This requires appropriate testing methodologies that far exceed how testing is done in other industries — an undertaking that can quickly become overwhelming for development teams.

With Lakera’s products, we enable our customers to automate such testing as part of existing engineering processes and speed up regulatory approval.

“Lakera’s products enable us to rigorously test our computer vision models and prepare for MDR certification — both would have previously consumed a lot more resources from our team.”

— Tom Dyer, Head of R&D at Behold AI

We want to help more companies innovate safely with AI in the medical domain. This is why, earlier this year, we joined a committee of leading medical researchers and AI developers to discuss key aspects and conduct extensive literature reviews on test datasets in one of the leading and most challenging medical applications: pathology.

The paper that we have now published summarizes the results and derives general recommendations for the collection of test datasets in pathology and medical imaging.

We address several questions: Which and how many images are needed? How to deal with low-prevalence subsets? How can potential bias be detected? How should datasets be reported? What are the regulatory requirements in different countries?

We hope that these recommendations help AI developers demonstrate the utility of their products and to help regulatory agencies and end users verify reported performance measures.

Download paper

https://www.nature.com/articles/s41379-022-01147-y

Want to know more?

Lakera’s MLTest enables medical imaging companies to put reliability at the core of their engineering processes and to speed up regulatory approval. You can get started with MLTest right away, or get in touch with me at david@lakera.ai.