What are the MLOps methodologies responsible for integrating machine learning-based solutions

The potential of machine learning to map the complex relationships contained in data makes it possible to create behavioral profiles of individual users, personalizing business logic, content, and communication channels in a tailored way. In the new data-driven paradigm, the individual customer gains a much more significant role than before. From being the final link in business processes, the customer becomes the basic unit from which the analysts' work begins.

KEY INFORMATION

Steps in building a Machine Learning solution

• Collecting large amounts of data, often from different sources and even different modalities, such as text, image, or audio data.
• Verification for quality, proper format, or structure.
• The raw data must be processed in an appropriate way. This process is called feature engineering.
• The next step is to work on the AI model itself, train it and evaluate it.
• At this stage - through experimentation - the correct hyperparameters of the model and the training method are selected.
• Communication and integration between the different parts of the system must also be set up accordingly.
• Pipeline responsible for data processing must be properly integrated into the database.
• The model must be able to access the processed features and make the trained model available in a certain way, depending on further applications.
• The next step is the AI model deployment, i.e., providing the infrastructure within which the model will be made available to end users.
• Solutions are also needed to monitor the performance of the overall system, as well as the effectiveness of the model itself.

The best-known examples of data-driven and customer-centric companies are, of course, giants such as Amazon, Uber, and Netflix. They are undeniable leaders in their respective fields. However, they are regarded by both financial markets and business analysts as primarily technology companies. Thanks to the overwhelming success of the pioneers, many companies, hitherto building an identity as classic retailers, manufacturers, or service providers, are beginning to realize that the heart of their business is data.

Change to a data-driven company

Transformations of companies from classical patterns to data-driven methodologies involve major technological and personnel changes. Leaders with a near-monopoly position could afford to hire hundreds of scientists and thousands of top-notch engineers to develop the tools and technologies necessary to operationalize the "data-driven" idea. Directly copying their approach is impossible for several reasons.

One obstacle is the enormous cost of building their own solutions and the time it takes to do so. The biggest problem with such an approach, however, is the inadequate supply of qualified academics and engineers with experience in data science and machine learning and their reluctance to work for companies that are not identified with high-tech.

“The biggest problem in building a data-driven organization is the inadequate supply of qualified scientists and engineers with experience in Data Science and machine learning, and their reluctance to work for companies that are not identified with high-tech.”

The level of the technical complexity of behavioral modeling systems is very high. It is desirable, although challenging to implement, to generate real-time predictions based on gigantic data sets. In addition, it is necessary to consider the changing environment, such as seasonality in the case of e-commerce applications, which requires periodic modification of the models' work by training on current data sets. This is impossible without proper integration of multiple components to enable AI system deployment and solutions to monitor product quality and version control.

In the case of classic software, the processes involved in its implementation are based on the DevOps methodology, which allows the integration of all stages of the product life cycle: development, testing, deployment, and maintenance. It emphasizes the automation and optimization of individual steps of the process. However, in the case of applications based on artificial intelligence, the product life cycle and the requirements for the infrastructure that ensures its proper integration and automation are different. The methodology responsible for the elements specific to the integration of machine learning-based solutions is called MLOps.

Work on building, training, and evaluating an AI model

Machine Learning solutions require providing the infrastructure to collect large amounts of data, often from different sources and even different modalities, such as text, image, or audio data. They are then subject to verification for quality, proper format, or structure. The quality of the data directly translates into the quality of the final solution. For them to serve as input for the model, the raw data thus collected must be pre-processed in an appropriate way. This process is called feature engineering and involves extracting information from the data relevant to the problem the AI is meant to solve and processing it into a form that can be understood by the machine learning algorithm.

Only the next step is actual AI model development, together with training and evaluation. At this stage - through experimentation - we choose the right hyper-parameters for the model and the way to train it. We evaluate the quality of the model using metrics appropriate to our domain. Communications and integration between the different parts of the system must also be properly configured. Pipeline responsible for data processing must be properly integrated with the database. Further, the model must access the pre-processed features and share the trained model in a certain way, depending on further applications.

“However, in the case of artificial intelligence-based applications, the product life cycle and the requirements for the infrastructure that ensures its proper integration and automation are different. The methodology responsible for the elements specific to the integration of machine learning-based solutions is called MLOps.”

A crucial element is also the appropriate use of hardware resources, which will enable fast operations on large amounts of data, training of the model, and fast inference time, i.e., the model's response to specific input data. The next step is to deploy the AI solution, that is, to build the infrastructure within which the model will be made available to end users. Solutions are also needed to monitor the performance of the overall system, as well as the effectiveness of the model itself. All these steps represent a huge amount of work. Developing such solutions alone is a cost measured in years and dozens of FTEs.

Tools that automate the work of data scientists

At the same time, more and more companies in classic industries are tempted to build data science and ML competencies and know-how internally. How to reconcile these ambitions with market realities? Solutions that simplify the work and automate many activities common in the work of a data scientist and ML engineer come to the rescue.

There are many types of solutions that support modeling processes. The easiest to use are no-code and low-code systems - that allow non-technical users to create ML models in no time. They are offered by companies such as DataRobot and Abacus.AI. They are highly popular, especially among business users. Their limitations are closely related to their strengths. They must be resistant to any mistakes the user may make. Because of this, modifying their operation is possible to a limited extent.

At the other extreme lie cloud ML ecosystems and solutions - supporting repeatable steps such as data processing, model training and versioning, and infrastructure management. Examples of such solutions include Microsoft Azure ML Studio, Google Cloud Vertex AI, or Amazon Web Services Sagemaker. They bring the field of data science and ML to a common denominator for all customers and offer a subset of the most used patterns today. They are dedicated to tech-savvy users. While they can be costly to use, they also impose some limitations on modeling freedom.

Does the use of cutting-edge solutions bring non-technical or technical but non-science ML users closer to the capabilities of giants like Amazon, Uber, and Netflix? For today, to a limited degree. No-code solutions still require proper data pre-processing. Following the principle of "garbage in, garbage out," the burden is on the proper preparation of input data for the quality of models.

Cloud ML solutions, on the other hand, despite significant improvements for developers and analysts that reduce programming and operational workload, still require the user to have full technical, analytical, and scientific skills - from preparing the pipelines that process the data to training the models.

Real-time systems and their limitations

The most pressing unaddressed issue to date remains real-time systems. Real-time systems approaches used in industries such as ad-tech (e.g., Google AdSense) are highly specialized and have no generic equivalents for a mass audience. Meanwhile, optimal business scenarios are those that allow companies to react instantly to customer behavior.

For example, when a customer of a Polish bank withdraws money from an ATM at the Singapore airport, this event should be immediately noted by all predictive models. As a result, the customer will get a personalized offer for travel insurance, unless he lives there permanently.

The unexpected winning of a gold medal at the Olympic Games by home athletes can be celebrated with an instant promotional campaign for sports fans at the local grocery store. This type of capability is beyond the reach of most AI platforms today, due to the outdated approach to data processing and the limitations associated with the very generic approach of a periodically updated data warehouse.

Real-time behavioral modeling platform

Over the past four years at Synerise AI, we have succeeded in developing a real-time behavioral modeling platform. Unlike general ML solutions, we focus on predicting human behavior. This reduction in ambition, compared to general-purpose platforms, has allowed us to develop a set of ultra-efficient algorithms, the so-called Universal Behavioral Profiles, which are subject to immediate updates after each new observation of customer behavior.

Realizing the lack of appropriate data processing technologies to meet the demands of the most ambitious clients, we had to develop our own real-time database - dedicated to both analytics and behavioral modeling combined with business scenario execution. The paradigm in which we work involves the use of raw data, without prior aggregation, while all transformations take place on the fly, even on multi-terabyte datasets.

We started with no-code approaches, enabling us to offer personalization, recommendations, scoring and behavioral segmentation to non-technical users, among other things. Combined with automation and a business scenario builder, they allow thousands of applications to be realized. Because many of our customers use the models online or in mobile applications, for the sake of end-user satisfaction, the models must respond instantly to the latest information.

“With trained models at their disposal, analysts can "learn from AI" - analyze the latent relationships and behavioral patterns recognized by the models. By working on the raw data, it becomes possible to understand which historical customer interactions influence future decisions.”

Today our biggest customers want to build their own data science competencies. That’s why we open the Synerise Monad platform for them, along with algorithms and real-time models. Behavioral modeling is done completely automatically based on raw data, and the platform user's task is to select predictive tasks accordingly.

With trained models at their disposal, analysts can "learn from AI" - analyze the hidden relationships and behavioral patterns recognized by the models. By working on raw data, it becomes possible to understand which historical customer interactions influence future decisions. Such dependencies have so far been irretrievably lost in classic approaches based on manual "feature engineering" and pipelining transformations.

Data Science core competencies that represent internal "know-how"

We believe that the future of ML applications will also look similar in areas other than behavioral modeling. After all, the task of artificial intelligence is to provide information to businesses, not the other way around. The progressive automation of ML tasks seems to be at odds with widespread ambitions to build internal data science competencies. However, this contradiction is only apparent. After all, not many IT companies today design their own microprocessors, only to implement proprietary IT systems, as was once common. A similar future awaits the data science market. It is becoming increasingly irrational to expect an analyst to understand how an artificial intelligence model works or to be able to design one themselves.

What will be the core competencies of data science that represent the internal "know-how" of companies? The most relevant is the transformation of artificial intelligence capabilities into real business scenarios, and applications. Until now, this has been a secondary issue. The planning phase of applications lasted a few days, and the preparation of data and creation of a single model took months or years. Today, when it is possible to create a dozen different models every day, the analyst's work becomes more creative. Answers to the questions become important: What models to create? How to apply them? How to measure their impact on customers? How to estimate the short- and long-term impact on business ratios?

These are not easy questions. Finding good answers to them often requires excellent knowledge of a company's business processes. Some answers can only be obtained empirically - by conducting experiments to verify the hypotheses. The new task facing data scientists becomes interpreting the hidden dependencies and behavioral patterns found by artificial intelligence models and transforming them into proactive business actions.

Exploring new business applications of AI

There is a shift in emphasis from knowledge and skills related to the inner workings of AI models to discovering new business applications for such solutions. The analytics team is freed from repetitive tasks on the one hand, and on the other can discover new patterns in the data that are revealed by implementing new solutions.

The center of gravity shifts toward work that is more creative and more closely related to business needs, such as formulating new assumptions and finding further areas of application for new technologies. At the same time, the path from idea to implementation in artificial intelligence projects is becoming much shorter. 

The level of the technical complexity of behavioral modeling systems is influenced by:

• the desirable, though difficult to implement, generation of real-time predictions based on gigantic data sets;
• consideration of environmental variables, such as seasonality for e-commerce applications, which requires regular modification of the models' work by training on current data sets;
• integrating multiple components to enable AI system implementation and solutions to monitor product quality and version control.

Article was written by Maria Janicka and Jacek Dąbrowski

--------------- 

The original text was published in ITwiz, check out the original article in Polish.