How to make Deep Learning Reproducible in less than 100 lines of code

Hydra, Pytorch Lightning , MlFlow, DVC if you don’t know how to use this open source technology to create Reproducible Deep learning pipeline than continue reading.

The Problem

If you are working on industrial, usually, to make a successful software project working you need 2 major things:

• Working in team
• Be able to write maintainable code

This means to adopt standard procedure to be able to share data and code as faster as you can with the team or with whom will be interface with your work.

Transfer your knowledge on a software project or share your result needs time and is not the part of the project that you, as developer, like the most. At least for me.

Probably every time you have to share your knowledge or make some sort of presentation you think that is completely a waste of time.

In any case these a fundamental steps for a successful software project, because we are not creating a project for our-self but for a business goal.

But if we can cut this time lost and could concentrate on more important and funny thing such as:

Create new models that work in the real world

wouldn’t be good?

If it is some times that you are working in the AI field, probably you know how much is difficult to reproduce results from any model.

This is due to the fact that in a deep learning project, the developer can customize a lot of hyperparameters and also to get a model work at best, you need also a little bit of art.

So this create a difficult in letting other developers reproduce your result or continue your work.

Make your life easier

If you , as a developer, don’t keep track of your experiments then will be very difficult for another developer or for you itself after some time, to come bacl to your work and go on.

For example in an industrial project could happen that the project, on which you are working on, will be stopped due to different reasons. For example for other projects with higher priority.

So after some time having to go back to your unstructured code takes a considerable amount of time and is often not fun, but necessary to continue your project.

This also create a problem on understanding the project timing. Usually managers don’t know exactly what is the difference in time between starting from a structured code or unstructured one.

Another things is that this difference in time depends also on the ability of every developer. So is not completely deterministic.

Right now if you are a manager you can think:

the role of the developer is also to make code structured and reusable from the beginning, but this is the fun fact.

When every company starts a deep learning project, usually this project is not deterministic. No one inside the company will know exactly if, from the project, is possible to get some useful result.

In Deep Learning we need to experiment to say if something could work

During experimenting the goal of the developer is to find something that could work, and usually that code is not structured. Why? Because you are making experiments and usually experiment are not perfect.

If for whatever reason the project is stopped, and the developer start to work on another one. All of the little detail of the experiments could be lost, if no one that care of making track of all the detail of every experiments made.

The solution

Usually making track of all this little detail takes time and is boring. So in this picture comes into your help, as a developer, the open source tools that we mention early. MLFlow, Hydra,DVC, Pytorch Lightning

These tools let you to:

Create structured code for experimenting

• Hydra helps you to create standard configuration file in order to keep track to all of your parameters.
• Pytorch Lightning helps you to organize your Pytorch Code. This help a lot on code readability and make some procedure standard
• DVC can be used to keep track to your dataset version. So you can know exactly which data you have used to obtain that model
• MlFlow helps you to keep track on all the information during training. You can track metrics, loss ecc… This could be used to compare different models.

We in Ai4Prod, as a machine learning engineers, are trying to build an open source software for industrial AI. We are addressing this problem by integrating into our solution all these open source technology for you.

We have already applied our technology on some industrial projects for long term usage. As we discussed in this blog post we use Ai4Prod to speed the process of experimenting while be able to show our results easily and reduce the amount of work required to keep track of our experiments.

In the next article we will show you how we have implemented this workflow.

Stay tuned