This article was written by Chris Cozzi and originally appeared on the DataRobot Blog here: https://www.datarobot.com/blog/introducing-the-mlops-management-agent/
Deployment and Monitoring Challenges
Our DataRobot MLOps product has focused on resolving many of the issues with bringing models into production.
We have already tackled the challenge of model deployment, particularly in adding capabilities to bring a model closer to the prediction environment and the data to be scored. For example, last year we released the Portable Prediction Server to standardize a model’s deployment in a container behind a production-grade REST API. Additionally, we have recently announced a partnership and integration with Snowflake to expand deployment options by bringing models directly into the database.
We have also addressed the complexity that IT and DevOps have when monitoring models across their organization. Large organizations have many diverse teams aligned to different business units or departments. Equally as diverse are the tools, languages, and platforms that these teams have selected to solve their unique data-science problems. The MLOps Monitoring Agent is an MLOps feature for standardized model monitoring regardless of where the model is running or what language the model is written in.
New in DataRobot MLOps Release 7.1: The MLOps Management Agent
With a suite of deployment options and the ability to monitor models in all of them, the next challenge on the horizon was bringing the pieces together and accelerating the actual process of putting a model into production, wherever the prediction environment might be.
DataRobot MLOps already provided simple, one-click deployment of models by any persona, provided they were deployed on the MLOps platform itself. With the MLOps Management Agent, we set out to bring this simple experience to all model deployment options, whether the models are built in DataRobot and deployed externally or they are developed and deployed entirely with open-source tools.
A Standardized Lifecycle Management Framework
The MLOps Management Agent provides a framework to automate the entire model deployment lifecycle in any environment or infrastructure such as Azure, GCP, AWS, or your own on-premise Kubernetes cluster. The framework allows automation of environment configuration and provisioning, model monitoring configuration and provisioning, model deployment and replacement, and integration into core DataRobot MLOps functionality, such as Challengers, all via our intuitive MLOps UI designed for non-coders.
The MLOps Management Agent acts as an accelerator to the setup of an organization’s prediction environments. It also isolates this setup to occur just once, instead of each time a model needs to be deployed or replaced. This unlocks automated deployment and replacement for all users via the DataRobot MLOps UI, all while still adhering to the governed workflow and approval process built into the platform.
Plugins: One Framework, Every Use Case
The flexibility to provide this experience in any infrastructure or environment is accomplished by the MLOps Management Agent’s plugin architecture.
While the MLOps Management Agent is responsible for monitoring the actions occurring in the DataRobot MLOps platform and orchestrating them in the remote environment, the plugin itself is responsible for performing the many actions associated with the model lifecycle and environment configuration.
DataRobot MLOps will ship with many common plugins right out of the box, such as deployment in Kubernetes or Docker, deployment of Java Scoring Code to compute environments such as Apache Spark, or even simple, governed deployment to file repositories.
Each plugin can be used as is or, depending on the needs of an organization or environment, be modified or used as a template for a new ground-up plugin.
See It Live: Kubernetes Deployment on Azure
The capabilities of the new MLOps Management Agent can best be experienced visually. The following video demonstrates just one of the many possibilities for deployment using the MLOps Management Agent. In this video, model deployment is occurring in a private Azure AKS environment using the MLOps Management Agent and a Kubernetes plugin, driven from the managed, SaaS DataRobot MLOps product.
This plugin is responsible for the following:
- Provisioning of the MLOps Monitoring Agent to send statistics back to DataRobot MLOps
- Provisioning of a load balancer for a stable URL, through deployment and model replacement
- Provisioning models in one (or more, for scale) Portable Prediction Servers
- Auto-scaling additional containers as load increases