What is a target function?
A target function, in machine learning, is a mathematical representation of the ideal mapping between supervised learning inputs and outputs. Target functions are often written as f(x), where x represents the input and f(x) represents the correct output.
In real-world problems, the target function is not directly observable. To approximate the target function, supervised learning models use labeled training data to learn a hypothesis function, written as h(x), that closely matches the target function.
Once a machine learning algorithm converges on a hypothesis function, it can use the h(x) to make predictions about new, unseen data. This process, known as inference, is used in predictive analytics to identify patterns, forecast outcomes and support data-driven decision-making in real-world applications.
How a target function works
In supervised learning, the training process involves feeding labeled examples -- pairs of inputs and corresponding outputs -- into a machine learning algorithm. During training, the algorithm learns a hypothesis function that maps inputs to outputs by identifying patterns in the data.
Machine learning models use optimization techniques, such as gradient descent, to iteratively adjust the parameters of the model so that h(x) closely approximates f(x) for as many inputs as possible. This process can be described mathematically as y = f(x), where x is the input, y is the correct output and f(x) is the unknown target function.
Data scientists and machine learning engineers approximate the target function to build predictive models. When successful, a hypothesis function will minimize prediction error and enable a predictive model to perform well on both the training data and new, unseen data.
Target function vs. hypothesis function
In supervised learning, the target function represents the true, underlying relationship between inputs and outputs in the real world, and the hypothesis function is what the model actually learns and uses to generate new outputs.
Understanding the distinction between these two functions is important because it frames the entire learning process as an approximation task. The concept of approximation is essential for understanding how concepts like overfitting, underfitting and generalization influence a model's ability to make accurate predictions on new inputs.
Examples of target functions in machine learning
The nature of a target function depends on the type of machine learning problem that's being solved, and whether the solution involves regression, binary classification, multiclass classification or more complex nonlinear mappings.
- Linear regression. The target function assumes there is a linear relationship between inputs and outputs, e.g., predicting house prices.
- Logistic regression. The target function defines the relationship between inputs and binary class labels, e.g., spam vs. not spam.
- Decision trees. The target function is modeled as a nonlinear, hierarchical structure that maps inputs to outputs using a series of rule-based decisions.
- Support vector machines. The model learns a decision boundary -- or a hyperplane -- that separates classes in the feature space to approximate the target function in classification tasks.
- Neural networks. The model approximates complex, often nonlinear target functions using multiple layers of interconnected nodes with activation functions. Activation functions enable the model to learn complex mappings.
Importance of the target function in machine learning
In supervised learning, the concept of a target function is important because it guides the learning process and helps define what the model is ultimately trying to approximate. As deep learning, reinforcement learning and generative AI models become more advanced, accurately approximating the target function has become essential for building reliable predictive models.
Modern systems use a combination of approaches to effectively approximate the target function. They include the following:
- Loss functions that quantify how closely the model's hypothesis function aligns with the target function.
- Regularization techniques that help prevent overfitting while still capturing the underlying target function.
- Explainability tools, such as SHAP and LIME, analyze model behavior and provide insights into how a hypothesis function relates to the predicted outputs.
When a model's hypothesis function approximates the true target function, it can reduce error and improve generalization to unseen data. In turn, this enables machine learning engineers to build AI systems that are more trustworthy, interpretable and aligned with real-world outcomes.
Today, improving the approximation of target functions is vital for AI model accuracy, fairness and robustness, especially in high-stakes fields like healthcare, finance and autonomous systems. In these domains, even small prediction errors can lead to significant consequences, such as incorrect diagnoses, biased lending decisions or unsafe navigation.
Related concepts to target function
To fully understand target functions, it helps to be familiar with related machine learning terms:
- Training data. Examples with known inputs and outputs used to train the model.
- Generalization. The ability of a model to apply the learned function to new, unseen data.
- Overfitting. When a model learns the training data too well, its outputs can be based on noise instead of the underlying patterns it should use to generate new outputs.
- Model capacity. A model's ability to approximate complex target functions; higher-capacity models can learn more complex mappings, but risk overfitting.
Learn more about the history and evolution of machine learning in this comprehensive essential guide.