Removing the gap between development and production in machine learning: MLOps
Explore DVC, CML and MLFlow to improve your machine learning developments
As a data scientist🧑🏻💻, I spend a lot of my time creating models for a huge variety of tasks. Creating a machine learning model is a complex task where you start cleaning the data, then you make the representation and at the end, you create the model. During this process is necessary to answer a lot of questions like:
- Which are the correct features for my problem?
- Is this model better than the one before?
- Which are the best hyper-parameters?
- I have a good model. How can I deploy into production?
- How can I monitor the results in production?
- I have a new version of my model. Which is the best way to replace the model in production?
MLOps can be applied to solve these questions and many others. In this post, I am going to present to you what MLOps is, and how we can use it in our machine learning projects. Besides, at the end of the post, I am going to provide a list of tools that made your life as a data scientist easier. Let’s start.
What is MLOps 📚
To understand what MLOps is, we need to go back in time and understand some DevOps principles and the way to incorporate them into our work. The objective of DevOps consists of allowing software developers to build software with a huge standard of quality. Some of the suggestions proposed on DevOps are:
- Version control of all the code to be able to other developers track your code
- Continuous Integration where a series of testing steps are passed to the code in order to be sure that we don’t have any errors in our development
- Continuous delivery to deploy our code into production with the less human interaction possible. Usually to allow this step is important to describe the hardware infrastructure we need as code, for example with AWS CloudFormation or Google Cloud Deployment Manager
- Monitor and logging to track how is working the system in production and detect in an efficient way possible errors
Now, with the DevOps suggestions in mind, we can try to search for similarities with our machine learning applications. I am going to describe some of the more used tools, but this is not a fixed solution. There are a lot of tools that can help you to integrate these principles into your projects. In the next section, I will tell you two of my favourites.
DVC and CML
To track experiments in machine learning is necessary to implement a strategy of version control. Our machine learning algorithm not only depends on code but also depends on the data, so we need to version both code (for example with git) and data (for example with DVC). By doing this versioning you and your mates can track each change in the code or the data, and see how it affects the model.
For the continuous integration (CI) and continuous delivery (CD) step, it is important to verify that the code is correct and the results of the model also are correct. For doing that you can use your favourite workflow tool like for example Github Actions. Using for example Github Actions, you can run the tests and train process of the model in a virtual machine with your data. If everything is correct the model can pass to the deployment stage. Recently has appeared a new tool called Continuous Machine learning or CML. This tool has an integration with Github actions and GitLab and allows creating reports of the training process and sending them to the team. An example of how the tool works can be seen in the image below.
In this example, when a piece of code is pushed to our Github repo the action runs. This action runs the training script. This script generates some metrics and figures. When the process finishes, you can use CML to generate a report and send it to the data scientist as a comment. Allowing these simple steps you can find bugs in your training process and have some quality steps that help you to be sure of the results.
MLFlow
Another tool that I think is very useful is MlFlow. The idea behind MlFlow is to bring data scientists a tool to control the experimentation, reproducibility and deployment of machine or deep learning models. With MlFlow you can:
- Track your hyper-parameters, metrics and images with MLFlow Tracking
- Save your models in a format that allows reproducibility in other machines with MlFlow Projects
- Deploy models in a very easy way to AWS or Azure with MLFlow Models
- Have a registry to centralize and save your models with MLFlow Registry
If I need to choose one of the four tools of MLFlow I choose the tracking platform. I use this platform all the days to have tracking of different experiments with their hyperparameters and metrics. The integration with Python is very easy and has a useful user interface that allows you to see the results and ask some questions to the results in a language similar to SQL.
Conclusions 📚
In conclusion, in this post, we have seen two tools that can be used to work faster in your developments and clean the line between development and production. First, we explored how to use DVC to get continuous integration in machine learning projects. Secondly, with MLFlow we analyzed among other options, the way to forget about using excel to save your hyper-parameters and metrics 😳.
These are not the only solutions, for example, some people have started exploring Cortex to get faster and accurate deployments into AWS. This tool allows developers to get an abstraction level of the infrastructure behind the API. Thanks to that you can define, for example, real-time APIs that autoscale automatically.
If you want to explore more about the wonderful world of MLOps you can check https://ml-ops.org. On this website, you can find more info about the motivation of using MLOps, best practices to monitor the models in production and a lot of tools that can be useful.
About the author
Marcos Esteve is a machine learning engineer at SolverIA. At his work, Marcos develops machine and deep learning models for a huge variety of tasks. He is quite interested in multimodality tasks and building data science apps. Contact him on Linkedin or Twitter
