Browse Definitions :
Choosing between a rule-based vs. machine learning system natural language understanding (NLU)

validation set

What is a validation set?

A validation set is a set of data used to train artificial intelligence (AI) with the goal of finding and optimizing the best model to solve a given problem. Validation sets are also known as dev sets.

Supervised learning and machine learning models are trained on very large sets of labeled data, in which validation data sets play an important role in their creation.

Training, tuning, model selection and testing are performed with three different sets of data: train, test and validation. Validation sets are used to select and tune the AI model.

Validation data sets use a sample of data that is withheld from training. That data is then used to evaluate any apparent errors. Machine learning engineers can then tune the model's hyperparameters -- which are adjustable parameters used to control the behavior of the model. This process acts as an independent data set for comparing the model's performance.

Even though validation data sets use training data for testing, it is not a part of either training or testing processes. This process acts as an unbiased evaluation of a model.

What are the differences between train, validation and test data sets?

Validation data sets are an important part of AI, machine learning and deep learning models, along with training and test data sets. These models use these data sets to identify and learn from data such as text images. After training, the models can be applied to areas such as text and image generation, natural language understanding or in the medical field. Testing, training and validation data sets are all used to prepare the model for operation, but are used at different points in its development:

How a data set for ML is separated.
The complete labeled data set is separated into an initial training set and then in smaller portions, validation and test data sets.
  • The training set is the portion of data used to train models. The model learns from this data. In testing, the models are fit to parameters in a process that is known as adjusting weights. Training makes up most of the total data.
  • Testing sets are only used when the final model is completely trained. These sets contain ideal data that extends to different scenarios the model would face in operation. This ideal set is used to test results and assess the performance of the final model.
  • The validation set uses a subset of the training data to provide an unbiased evaluation of a model. The validation data set contrasts with training and test sets in that it is an intermediate phase used for choosing the best model and optimizing it. It is in this phase that hyperparameter tuning occurs. Overfitting is checked and avoided in the validation set to eliminate errors that can be caused for future predictions and observations if an analysis corresponds too precisely to a specific data set.

Model training with training, validation and test sets should be split depending on the number of data samples and the model being trained. Different models might require significantly more data to train than others. Likewise, the more hyperparameters there are, the larger the validation split needs to be. It is also generally considered unwise to attempt further adjustment past the testing phase. Attempting to add further optimization outside the validation phase will likely increase overfitting.

Learn more methods to evaluate and improve machine learning models.

This was last updated in September 2023

Continue Reading About validation set

  • firewall as a service (FWaaS)

    Firewall as a service (FWaaS), also known as a cloud firewall, is a service that provides cloud-based network traffic analysis ...

  • private 5G

    Private 5G is a wireless network technology that delivers 5G cellular connectivity for private network use cases.

  • NFVi (network functions virtualization infrastructure)

    NFVi (network functions virtualization infrastructure) encompasses all of the networking hardware and software needed to support ...

  • phishing

    Phishing is a fraudulent practice in which an attacker masquerades as a reputable entity or person in an email or other form of ...

  • computer forensics (cyber forensics)

    Computer forensics is the application of investigation and analysis techniques to gather and preserve evidence from a particular ...

  • cybersecurity

    Cybersecurity is the practice of protecting internet-connected systems such as hardware, software and data from cyberthreats.

  • OKRs (Objectives and Key Results)

    OKRs (Objectives and Key Results) encourage companies to set, communicate and monitor organizational goals and results in an ...

  • cognitive diversity

    Cognitive diversity is the inclusion of people who have different styles of problem-solving and can offer unique perspectives ...

  • reference checking software

    Reference checking software is programming that automates the process of contacting and questioning the references of job ...

Customer Experience
  • martech (marketing technology)

    Martech (marketing technology) refers to the integration of software tools, platforms, and applications designed to streamline ...

  • transactional marketing

    Transactional marketing is a business strategy that focuses on single, point-of-sale transactions.

  • customer profiling

    Customer profiling is the detailed and systematic process of constructing a clear portrait of a company's ideal customer by ...