A quick guide on how to deploy a depth estimator at scale without creating a webservice, docker, and setting up the infrastructure.
Welcome to another tutorial of Develop, Deploy, and Integrate series. The main goal of that series is to uncover the fastest and easiest possible way to deploy a deep learning model at scale without any special configuration, long process of infrastructure setup, and difficult monitoring when the model is live. Just a simple way from model to actual use-case... in minutes!
All tutorials from the series are divided into three quick steps so that they can be easily applied to other machine learning models without any changes.
In this particular tutorial, we will deploy depth estimator model that is an essential part of every 3D vision system.
In the first step, we need to build and train a model so that we are able to perform inference. After that upload, the code to the repository, while weights of the model to some external storage if necessary.
A depth estimator model requires an input RGB image and outputs a depth image. The depth image includes information about the distance of the objects in the image from the viewpoint, which is usually the camera taking the picture. That technology is mainly used in self-driving cars, shadow mapping in computer graphics, and robot-assisted surgeries.
The two images below provide a clear illustration of depth estimation in practice.
Since building and training, such a model is a very long process, especially when you want to obtain plausible results. Therefore, we will use the trained MiDaS v2.1 model introduced by René Ranftl in his paper, where he describes a new tool that enables mixing multiple datasets during training, even if their annotations are incompatible. MiDaS v2.1, for instance, was trained on 10 different datasets with multi-objective optimization.
In order to run a model, you don't need to copy all files from the main repo, just the ones that are needed for inference. We have already done it for you, so you can either copy from our repo or the original one.
The last thing that you need to do is place the whole code in some git repo.
Step 2: Deploy depth estimator
The second step we will actually deploy a model.
The whole process of AI model deployment is highly influenced by the type of task and later the evaluation metrics.
For instance, a cooking recommendation engine needs to deliver best suited to your needs recipes while you are on the website (in seconds!). In this case, the most important metric is latency that needs to very low. Therefore for this task, you would consider choosing the deployment that works in real-time.
However, when you build a super-resolution engine, that improves the quality of the video, the main metric that you are looking for is some kind of image quality assessment. That is why batch deployment is a better choice because the response time does not play a big role in this case.
In conclusion, you have at least a grasp of how AI model deployment is a broad topic. As a result, in our case, we will go for real-time model deployment and we will use Syndicai Platform to easily deploy a model, without taking care of scalability, versioning, monitoring, and security.
Prepare a model
Our model is already trained and uploaded to the Git repository. Now, we need to somehow define how the model will take input and return output when being a webservice.
Thankfully, Syndicai does not require creating a webservice or docker. You only need to create two files syndicai.py & requirements.txt and place them in the main directory of your repository.
The first file, syndicai.py, consists of PythonPredictor python class responsible for model prediction. In this case, both input and output are in the base64 format, and the content of the files looks as follows.
import cv2 import numpy as np import urllib.request import matplotlib.pyplot as plt
import tensorflow as tf import tensorflow_hub as hub
from utils import *
def __init__(self, config): # the runtime initialization will not allocate all memory on the device to avoid out of GPU memory gpus = tf.config.experimental.list_physical_devices('GPU') for gpu in gpus: #tf.config.experimental.set_memory_growth(gpu, True) tf.config.experimental.set_virtual_device_configuration(gpu, [tf.config.experimental.VirtualDeviceConfiguration(memory_limit=4000)])
# load model self.module = hub.load("https://tfhub.dev/intel/midas/v2_1_small/1", tags=['serve'])
def predict(self, payload): # convert base64 to OpenCv format img = b64_to_image(payload["image_b64"]) / 255.0
Remember that your model has to be Active in order to work! As the output, you should also get base64.
If you get a correct response you are ready to go with the model integration. Go to the model Integrate page and use the code snippet to implement the REST API in your website, mobile app, or some platform.
For the purpose of that kind of tutorials we have managed to create a sample web app written in React that consumes deployed models and is the perfect example of the showcase your models. You can try it on the Syndicai Showcase page and get a feeling of what we are talking about.
In addition, you can fork the repository with the showcase page, because the whole code is open-sourced.
To sum up, you've done a great job!
In the above tutorial, you had a chance to learn how to deploy depth estimator model. As you could see, the whole process does not require you to create a webservice, docker, and infrastructure. Syndicai do it for you so that you can iterate faster and focus on what's important - model building and training!
* * *
If you found that material helpful, have some comments, or want to share some ideas for the next one - don't hesitate to drop us a line via slack or mail. We would love to hear your feedback!