This post aims to explain why Ai4prod simplies for anyone how to bring AI into production with C++.
Learning how to use Ai4Prod will provide you a tool to use to deploy Machine Learning in production.
If you feel lost on how to bring Machine Learning in production, then Ai4Prod guides you through the entire process from training to deployment in C++
If you have already tried to use Machine Learning in C++ and you have a lot of headache, then Ai4prod is built to help you optimize your work.
Who is Ai4Prod for?
Ai4Prod is addressed to machine learning or software engineers who are looking for an easy way to create and use their custom model in C++.
Ai4Prod is developed by machine learning engineers that everyday are trying to put AI into production exploiting C++. We built and open source it to help everyone to create value with AI.
Design Principle
Ai4Prod is mainly built for production environments, which have different design principles compared to research ones. The key aspects of Ai4prod are:
- Easy to integrate: it reduces the integration time in an already existing production environment.
- Easy to customize for different needs you could have to deal with, during the production phase.
- Works on different Hardwares and Operating Systems (Windows,Linux,Embedded)
- Speed up project delivery to clients
- Inference phase provides the same accuracy between Python and C++ models. Each model accuracy has been already verified.
- Standardize coding workflow helps code maintainability. In production, this is a must have, because usually you are not working alone and your code must be readable and maintainable.
The Typical production workflow
Usually if you start a Machine learning project you have to:
- Create a custom dataset
- Understand which model is better for your task
- Train different models to verify which one is the best choice
- Check if you achieve the required accuracy
- Deploy your model
- Monitor your model to prevent model drift
More or less these steps (1–4) are the same initial steps for all Machine Learning projects. Steps 5–6 could change depending on your project requirements.
We create Ai4Prod to simplify steps (3–6) which are essential to build a successful machine learning project and to easily deploy it in your production environment.
The Typical workflow with Ai4Prod
In the image below you can see all the involved operation that you can handle with Ai4Prod in a typical machine learning project:
With Ai4Prod you can use Python to train your custom models and C++ to deploy them.
Usually in Machine Learning Project we have 2 specific phase
- Training
- Inference
In Ai4Prod, Training is managed with Python while Inference with C++.
If you are not familiar with Machine Learning, the training phase is necessary to tune the model parameters to fit the training data. Then, the trained model is deployed inside your application and it starts to make predictions on never seen data: this is the inference phase.
To make an analogy: during training your are building your own car, while during inference your are driving it.
One of the possible methodologies to exploit your python trained models in a C++ application is to convert them in one of the possible existing format.
Therefore, Ai4Prod provides the possibility to:
- Train your custom models in Python with tested repositories.(STEP 1)
- Easily convert your python models for C++(STEP 2)
- Use the converted models and do inference in C++(STEP 3)
In Ai4Prod these 3 steps are called a Pipeline.
Inside Ai4Prod you could find different pipelines:
- Object detection Pipeline
- Classification Pipeline
- Tracking Pipeline
- Nlp Pipeline
- ecc…
How Ai4Prod repository is organized
You can reach Ai4Prod project at this link
You will find 2 repositories:
- Ai4prod: C++ inference execution
- Ai4prod_python: model training and conversion
The repositories are tightly coupled and they compose the different pipelines available with Ai4Prod.
We adapted all Python code in order to get a model ready to use in C++.
Documentation is provieded on our website https://www.ai4prod.ai/docs/
Code Example
In this example, we focus on the inference stage(STEP 3) of a classification Pipeline. We omit the training and conversion phases because you can find full examples inside the Ai4Prod documentation.
In order to have a standard process, we have divided the inference process in 3 sub-processes. For each model’s inference you will have:
- preprocessing(): All the pre-processing operations needed to properly input your data inside the model.
- runmodel(): Run model on different Hardware targets.
- postprocessing(): All the post-processingoperations needed to get correct results.
Threfore, you need very few lines of code to use ResNet50 for a classification task.
- Instantiate the class with correct parameters
2. Call the preprocessing(), runmodel() and postprocessing() to get results
Customizable for your needs
Here we show how you can customize the Ai4Prod inference phase (STEP 3) for your needs. Usually you need to do this if you change something during the training phase.
For example, if you change some input data transformations in Python, you only need to create a subclass of our ResNet50 and only change the pre-processing function to reflect these changes in C++.
In this way you can customize very easily every pipeline for your needs. A complete tutorial of the rules to follow will be available soon in our documentation.
How to start
Have a look on our getting started page to start creating your custom models and exploit C++ for your Machine Learning or Deep Learning applications.
