This article was originally posted on our company website. Lakera’s developer platform enables ML teams to ship fail-safe computer vision models.
This year has been exciting across all things AI. Amazing strides in drug discovery, breakthroughs on language models, and last but not least generative AI.
The field progresses so quickly that it often feels difficult to keep up with it. We hear you!
But we also believe that one question remains and is more important than ever: how do we deploy AI systems reliably and safely? There is no better time than right now to answer that question.
“There are things where reliability doesn’t matter, for everything else there is Lakera.”
Billions of new AI systems are going to make it into our world over the next few years, automating critical decision-making in hospitals, industrial environments, and everyday life. These systems can only benefit us if we make them reliable and safe.
At Lakera, we are more excited than ever to be working with some of the world’s most advanced AI firms and to support organizations in building, certifying, and operating AI that can be trusted.
As we wrap up the year, we’ve put together a couple of our most popular resources that the team shared this year. Have a look at them below and reach out to us any time you want to chat.
It’s now time to recharge for a couple of days and can’t wait to continue working with you in the new year! ✌️
🧪 ML reliability
Increasing ML reliability through systematic testing is at the core of what Lakera does. Here is a read through some of this year’s articles to make sure you are proactively making your AI as reliable and safe as possible.
Why testing should be at the core of machine learning development.
Lakera’s guide to machine learning testing.
Why ML testing is crucial for reliable computer vision.
😯 Bias in ML
“Data is a reflection of the inequalities that exist in the world”. While this might be true, developers have great potential to curb bias in their computer vision systems.
Establishing whether bias is present in a computer vision system is key to understanding how it will perform in operation. Bias manifests itself in numerous ways, from data collection and annotation to the features that a system uses for prediction.
The computer vision bias trilogy: Data representativity
The computer vision bias trilogy: Shortcut learning
The computer vision bias trilogy: Drift and monitoring
⚖️ Fairness and ethics
We are the first to ship a fairness assessment that checks for bias against protected categories in computer vision systems.
These tests allow, for example, to test the following:
- Is my pedestrian detector fair as defined by Equality of Opportunity?
- Is my model ISO/IEC 24027 compliant?
- Does my model discriminate against a given demographic?
MLTest implements the core metrics described in ISO/IEC 24027.
Learn more about our fairness package here.
We also welcomed Niki Kilbertus as an expert advisor on the topic:
🏗️ AI Certification
We know that AI must be reliable, robust, and fair — in short trustworthy. But how do you actually build, test, and certify your models?
At Lakera, we combine practical AI knowledge and regulatory expertise to support development teams and auditors in bringing transparent and trustworthy AI to market.
Case study: How Privately accelerated computer vision certification with Lakera.
Read about Lakera in DEKRA’s Annual Report: Working together for greater AI Safety.
🤔 Model selection
Model selection is damn hard. We see it everywhere: you compute some metrics during and after training and they don’t correlate with real-world performance. At Lakera, we strive to change that. MLTest makes model selection easier by providing more accurate indicators of real-world performance.
How robust are pre-trained object detection ML models like YOLO or DETR?
How to compare two computer vision models.
Stress-test your models to avoid bad surprises.
🏥 Healthcare
Have a read below about how we support some of the most advanced healthcare teams in bringing their products to market safely and efficiently.
Lakera releases robustness testing suite for digital pathology.
Lakera co-publishes article in a Nature journal on testing medical imaging systems.
One-on-one with Tom Dyer: What it takes to successfully build ML for healthcare.
Medical imaging as a serious prospect: Where are we at?
🌪️ Open-source
With our efforts to understand computer vision systems better, we have released an implementation of OpenAI’s CLIP model that completely removes the need for PyTorch. With this implementation, you can quickly and seamlessly install this fantastic model in production and on edge devices.
👐 Events & Media
We’ve participated and hosted in numerous events this year and look forward to doing even more in 2023.
Making artificial intelligence safe — this is how AI won’t make mistakes. (Handelsblatt)
Lakera has been named as one of the Swiss startups to watch. (Sifted)
Safe and testable computer vision with Lakera.
Our team members launched the MLOps Switzerland Community which already has over 150 members. New events are in the pipeline, join the group to stay up to date.
⚙️ Newly added integrations
Lakera ❤️ Voxel51
Learn more about how Voxel51 users can integrate Lakera to supercharge model evaluation in our developer docs or watch Mateo provide an overview here.
Lakera ❤️ MLflow and W&B
Keep track of model performance at scale with Lakera and MLflow or Weights&Biases. Watch an overview here or read our developer docs.
Lakera ❤️ DVC
Automate model testing with DVC and Lakera. We discussed the integration here and provided an example configuration in our developer docs.
More integrations are coming soon! Stay tuned!
