I’m doing Machine Learning Industrialization for more than 2 years and I’m thrilled to see it featured by McKinsey as top 2 in its 2023 tech trends!
Industrializing ML is about applying Software Engineering best practices to the whole AI modeling process since its first line of code. It is about Data Scientists focusing on math and stats at the same time that the AI artifact is casted as a software product aiming production environments. This is different from MLOps, which is commonly positioned as a mere wrapping activity that happens after and separated from AI modeling and before production. In the whole Industrialization practice, MLOps is a subset activity that happens in between, but quite apart, from both Data Scientists’ work and the infrastructure. Industrializing Machine Learning contains MLOps, plus other concepts that are even more important.
The term “industrial” is accurate precisely to antagonize with the artisanal way that Machine Learning squads usually operate nowadays. It’s common to see a lot of mathematics, good statistics, but few software engineering best practices, little DevOps, few design patterns, minimal automation, and limited standardization.
I practically invented Machine Learning Industrialization for myself when I was at Loft, out of necessity and intuition, in 2021. Work that I proposed and lead when I was there allowed us to scale from 4 models that were laborious to maintain and monitor, to over 70 models, without growing the team of Data Scientists. Those +70 are now easy to maintain, audit, observe, reproduce, retrain, find and handle in general.
One thought on “Industrializing Machine Learning”