ai-course/docs/section1/Common Terminology/AI model training.md

AI Model Training: Principles and Paradigms

In artificial intelligence (AI), model training refers to the process of "teaching" a machine learning (ML) model to optimize its performance on a training dataset of sample tasks relevant to its intended use cases. This is the most critical step in the lifecycle of AI models.

Here's a breakdown of what model training means in AI:

• Purpose and Goal

  • The primary goal of model training is to enable the model to learn patterns and correlations from data so it can make accurate predictions or draw conclusions on new, unseen data.
  • In mathematical terms, the objective is to minimise a "loss function" which quantifies the error between the model's outputs and the desired outcomes on training tasks. For reinforcement learning, the goal is reversed: parameters are optimised to maximise a "reward function".
  • When the model's error falls below a predefined threshold, it is considered "trained".

• How it Works

  • AI training fundamentally involves feeding AI models large datasets. These datasets can be structured or unstructured, labeled or unlabeled, depending on the type of model and training paradigm.
  • During this process, the model adjusts its internal parameters (e.g., weights and biases in its algorithms or neural network nodes) to produce more accurate outputs. This iterative process is often called "model fitting".
  • Backpropagation is a common technique used in deep learning to calculate how each node in a neural network contributes to the overall loss, allowing for parameter optimisation.
  • Developers may need to fine-tune models after initial training by providing additional data and correcting outputs to improve accuracy or adapt to new use cases. This is also known as "pretraining" followed by fine-tuning.

• Models vs. Algorithms

  • While often used interchangeably, algorithms are the procedures (mathematical logic or pseudocode) used to generate predictions, whereas models are the outcome of optimising an algorithm's parameters to improve its performance on a specific dataset.
  • Two models can use the same underlying algorithm but have different parameter values because they were trained on different data.

• Types of Model Training AI models are typically categorised into three primary machine learning paradigms, each with unique use cases and training processes:

  • Supervised Learning:
    • Used when the model is trained to predict a "correct" output for a given input.
    • It relies on labeled training data (where inputs are paired with desired outputs, also known as "ground truth" or "supervisory signals").
    • Example: Training an algorithm to recognise cat pictures by feeding it images explicitly labeled as "cats".
    • This is the dominant form of training for deep learning models and underpins most state-of-the-art AI today.
  • Unsupervised Learning:
    • Used when the model is trained to discern intrinsic patterns and correlations in data without predefined outputs or labels.
    • The algorithm categorises data into groups based on attributes it discovers.
    • Example: A program looking through online sales data to identify different types of clients without being told what those types are.
  • Reinforcement Learning:
    • A "learn by doing" approach where an "agent" learns to perform a task through trial and error.
    • It involves an agent evaluating its environment and taking actions that maximise a reward function (positive feedback for good performance, negative for poor).
    • Example: Teaching a robotic hand to pick up a ball, or a chess program learning which moves increase the probability of winning.
  • Other types include semi-supervised learning (combining labeled and unlabeled data) and self-supervised learning (where supervisory signals are derived from the structure of unlabeled data itself).

• Computational Resources and Data Quality

  • Training AI models, especially large ones like Large Language Models (LLMs), can be very expensive in terms of compute power and time, often requiring thousands of clustered GPUs and weeks of processing. This is generally considered a one-time expense.
  • The quality and quantity of training data are critical to the model's effectiveness and accuracy; poor or biased data will lead to flawed models.
  • Overfitting, where a model memorises training data but cannot generalise well to new data, is a challenge that cross-validation helps to mitigate.

In summary, model training is the foundational phase where an AI system develops its "knowledge" by repeatedly processing data and refining its internal structure until it can effectively perform tasks by drawing conclusions from new inputs.