What are machine learning models?
A machine learning model automates the process of identifying patterns and relationships that are hidden in data. It can use a combination of prelabeled or unlabeled data processed through various machine learning algorithms to determine the best fit for the problem to be solved.
Each machine learning algorithm represents a specific strategy to uncover patterns within a historical data set, according to Michael Shehab, principal and labs technology and innovation leader at PwC. The process of transforming machine learning algorithms into models consists of three components: representing the problem, identifying a specific task and providing feedback to guide the algorithm's quest for a solution. "The resulting model represents a function that has been learned or produced by the machine learning algorithm and is capable of mapping previously unseen examples to an accurate output," Shehab explained.
Selecting the type of model to use is a mixture of art and science. "There is no one-size-fits-all approach to understanding which model works for your organization," said Brian Steele, vice president of product management at customer analytics platform provider Gryphon.ai. Each model type will offer insights and results based on the data type and use cases. Additionally, the type and quality of the input data will drive the selection of certain types of models.
Types of machine learning models
The machine learning field is rapidly evolving. When it comes to describing approaches like those used in generative AI applications, new techniques are blurring the old methods of classifying models.
There's no commonly accepted classification standard as new models are added daily, said Anantha Sekar, AI lead at Tata Consultancy Services. Still, the most common classifications of machine learning models include supervised, semi-supervised, unsupervised and reinforcement learning. These major types should all be considered along with the objective and learning approach being used, Sekar recommended.
A generative AI model, for example, may involve multiple training approaches deployed in succession. It may start with unsupervised learning on a large corpus of data followed by supervised learning to fine-tune the model and reinforcement learning to continuously tune results after deployment. "Discussing types of models is like discussing types of humans," Sekar noted. "Since each one is ultimately unique, the classifications are useful mainly for broad understanding purposes."
Training machine learning models
Data scientists will each develop their own approach to training machine learning models. Training generally starts with preparing the data, identifying the use case, selecting training algorithms and analyzing the results. Following is a set of best practices developed by Shehab for PwC:
- Start simple. Model training should begin with the simplest approach. Complexity can then be added in the form of model features, feature sophistication and advanced learning algorithms. The simpler model serves as a basis for determining if the performance obtained through the added complexity will be worth the additional investment in time and technical costs.
- Create a consistent model development process. Given its highly iterative nature, a consistent development process should be supported with tools that provide comprehensive experiment tracking so data scientists can more readily pinpoint where their models can be improved.
- Identify the right problem to solve. Look for improperly defined objectives, wrong areas of focus and unrealistic expectations, all of which are often responsible for a model's poor performance or failure to produce tangible value. Building a model requires solid grounding to properly assess its development.
- Understand the historical data. The model is only as good as the data it will be trained on, so start with a firm understanding of how that data behaves, the overall quality and completeness of the data, important trends or elements of the data set related to the task at hand and any biases that may be present.
- Ensure accuracy. To avoid introducing bias, providing the model with inappropriate feedback or reinforcing the wrong behavior, carefully set measurable benchmarks for model performance. A machine learning algorithm learns through feedback from an objective or outcome set in the training data. If the calculations that generate the feedback aren't carefully defined and aligned to the expected values, the result could be a poor or non-functioning model.
- Focus on explainability. Data scientists who focus on why a model performs the way it does will produce better models. This approach requires more comprehensive model validation and testing. Explainability also provides insights into a model's underperformance, hypotheses of how to enhance performance and a global view of how a model functions to help develop trust among consumers.
- Continue training. Model training is an ongoing process over the life of the model, including the production stage, so it can be continuously improved.
Is there a best machine learning model?
In general, there is no one best machine learning model. "Different models work best for each problem or use case," Sekar said. Insights derived from experimenting with the data, he added, may lead to a different model. The patterns of data can also change over time. A model that works well in development may have to be replaced with a different model.
A specific model can be regarded as the best only for a specific use case or data set at a certain point in time, Sekar said. The use case can add more nuance. Some uses, for example, may require high accuracy while others demand higher confidence. It's also important to consider environmental constraints in model deployment, such as memory, power and performance requirements. Other use cases may have explainability requirements that could drive decisions toward a different type of model.
Data scientists also need to consider the operational aspects of models after deployment, called ModelOps, when prioritizing one type of model over another. These considerations may include how the raw data is transformed for processing, fine-tuning processes, prompt engineering and the need to mitigate AI hallucinations. "Choosing the best model for a given situation," Sekar advised, "is a complex task with many business and technical aspects to be considered."
Machine learning model vs. machine learning algorithm
The terms machine learning model and machine learning algorithm are sometimes conflated to mean the same thing. But from a data science perspective, they're very different. Machine learning algorithms are used in training machine learning models.
Machine learning algorithms are the brains of the models, Steele suggested. The algorithms contain code that's used to form predictions for the models. The data the algorithms are trained on often determines the types of outputs the models create. The data acts as a source of information for the algorithm to learn from, so the models can create understandable and relevant outputs.
Put another way, an algorithm is a set of procedures that describes how to do something, Sekar explained, and a machine learning model is a mathematical representation of a real-world problem trained on machine learning algorithms. "So, the machine learning model is a specific instance," he said, "while machine learning algorithms are a suite of procedures on how to train machine learning models."
The algorithm shapes and influences what the model does. The model considers the what of the problem, while the algorithm provides the how for getting the model to perform as desired. Data is the third relevant entity because the algorithm uses the training data to train the machine learning model. In practice, therefore, a machine learning outcome depends on the model, the algorithms and the training data.