Added 2 sections; added logo

docs/section1/Common Terminology/AI model training.md

@@ -0,0 +1,44 @@
+# AI Model Training: Principles and Paradigms
+
+In artificial intelligence (AI), **model training** refers to the **process of "teaching" a machine learning (ML) model to optimize its performance** on a training dataset of sample tasks relevant to its intended use cases. This is the most critical step in the lifecycle of AI models.
+
+Here's a breakdown of what model training means in AI:
+
+- **Purpose and Goal**
+    
+    - The primary goal of model training is to enable the model to **learn patterns and correlations** from data so it can make accurate predictions or draw conclusions on new, unseen data.
+    - In mathematical terms, the objective is to **minimise a "loss function"** which quantifies the error between the model's outputs and the desired outcomes on training tasks. For reinforcement learning, the goal is reversed: parameters are optimised to **maximise a "reward function"**.
+    - When the model's error falls below a predefined threshold, it is considered "trained".
+- **How it Works**
+    
+    - AI training fundamentally involves **feeding AI models large datasets**. These datasets can be structured or unstructured, labeled or unlabeled, depending on the type of model and training paradigm.
+    - During this process, the model **adjusts its internal parameters** (e.g., weights and biases in its algorithms or neural network nodes) to produce more accurate outputs. This iterative process is often called "model fitting".
+    - **Backpropagation** is a common technique used in deep learning to calculate how each node in a neural network contributes to the overall loss, allowing for parameter optimisation.
+    - Developers may need to **fine-tune** models after initial training by providing additional data and correcting outputs to improve accuracy or adapt to new use cases. This is also known as "pretraining" followed by fine-tuning.
+- **Models vs. Algorithms**
+    
+    - While often used interchangeably, **algorithms are the procedures** (mathematical logic or pseudocode) used to generate predictions, whereas **models are the outcome** of optimising an algorithm's parameters to improve its performance on a specific dataset.
+    - Two models can use the same underlying algorithm but have different parameter values because they were trained on different data.
+- **Types of Model Training** AI models are typically categorised into three primary machine learning paradigms, each with unique use cases and training processes:
+    
+    - **Supervised Learning:**
+        - Used when the model is trained to predict a "correct" output for a given input.
+        - It relies on **labeled training data** (where inputs are paired with desired outputs, also known as "ground truth" or "supervisory signals").
+        - Example: Training an algorithm to recognise cat pictures by feeding it images explicitly labeled as "cats".
+        - This is the dominant form of training for deep learning models and underpins most state-of-the-art AI today.
+    - **Unsupervised Learning:**
+        - Used when the model is trained to discern intrinsic patterns and correlations in data **without predefined outputs or labels**.
+        - The algorithm categorises data into groups based on attributes it discovers.
+        - Example: A program looking through online sales data to identify different types of clients without being told what those types are.
+    - **Reinforcement Learning:**
+        - A "learn by doing" approach where an "agent" learns to perform a task through **trial and error**.
+        - It involves an agent evaluating its environment and taking actions that maximise a **reward function** (positive feedback for good performance, negative for poor).
+        - Example: Teaching a robotic hand to pick up a ball, or a chess program learning which moves increase the probability of winning.
+    - **Other types** include semi-supervised learning (combining labeled and unlabeled data) and self-supervised learning (where supervisory signals are derived from the structure of unlabeled data itself).
+- **Computational Resources and Data Quality**
+    
+    - Training AI models, especially large ones like Large Language Models (LLMs), can be **very expensive in terms of compute power** and time, often requiring thousands of clustered GPUs and weeks of processing. This is generally considered a one-time expense.
+    - The **quality and quantity of training data** are critical to the model's effectiveness and accuracy; poor or biased data will lead to flawed models.
+    - **Overfitting**, where a model memorises training data but cannot generalise well to new data, is a challenge that cross-validation helps to mitigate.
+
+In summary, model training is the foundational phase where an AI system develops its "knowledge" by repeatedly processing data and refining its internal structure until it can effectively perform tasks by drawing conclusions from new inputs.

docs/section1/Common Terminology/Embeddings.md

@@ -0,0 +1,38 @@
+# The Essence of Embeddings in AI and ML
+
+In artificial intelligence (AI) and machine learning (ML), **embeddings are numerical representations of data, such as text, images, or audio, transformed into dense, continuous vectors within a lower-dimensional space**. They are designed to be consumed by machine learning models and semantic search algorithms.
+
+Here's a detailed breakdown of what embeddings mean and how they function:
+
+- **Purpose and Goal**
+    
+    - The primary goal of embeddings is to **capture semantic relationships and patterns within data**. This means that data points with similar meanings or contexts are positioned closer together in this vector space, while dissimilar items are farther apart.
+    - They **translate various types of objects into a mathematical form** that AI models can easily process, as most machine learning algorithms are incapable of directly processing plain text or raw unstructured data.
+    - Embeddings are foundational for AI because they enable machine learning models to **find similar objects** and understand the relationships between them.
+- **How They Work**
+    
+    - Technically, an embedding is a **vector**, which is an array of numbers that defines a point in a multi-dimensional space. Each number in the vector indicates the object's position along a specified dimension.
+    - The **process of creating these vectors is called "embedding"** and typically uses deep learning, particularly neural networks. Neural networks, inspired by the human brain, have multiple layers, and the creation of embeddings often occurs in a hidden layer before further processing.
+    - During training, the neural network learns to automatically create this embedding layer, determining which dimensions to include and how words or data points relate to each other without explicit human definition.
+    - The **distance and direction between these vectors encode the degree of similarity** between words or objects. Common metrics for determining similarity include cosine similarity and Euclidean distance.
+    - **Word embeddings**, for instance, represent words as fixed-sized dense vectors of real numbers, capturing semantic relationships and contextual meanings based on their usage patterns in a large body of text. Contextual embeddings, used in transformer models, allow the representation of a word to change based on its surrounding context, capturing more nuanced meanings.
+- **Why They Are Used (Benefits)**
+    
+    - **Semantic Representation:** Embeddings capture semantic relationships, allowing models to understand meaning and generalize better.
+    - **Dimensionality Reduction:** They transform high-dimensional data (like text or images) into lower-dimensional representations, making them computationally more efficient.
+    - **Improved Generalization:** Models learn meaningful representations, which helps them perform well on new, unseen examples, even with limited labeled data.
+    - **Efficient Training:** Embedding layers in neural networks facilitate backpropagation and optimization, making training more efficient.
+    - **Faster and Cost-Effective Inference:** Once created, embeddings can be stored in vector databases. This significantly speeds up query responses and reduces computational costs, as the model doesn't need to re-process the entire dataset for every new query.
+    - **Handling Unstructured Data:** Embeddings are crucial for converting unstructured data (text, audio, images) into a numerical format suitable for ML models.
+- **Types of Objects That Can Be Embedded** Embeddings are versatile and can be applied to a wide range of data types:
+    
+    - **Words:** Word embeddings capture semantic relationships and contextual meanings. Popular models include Word2Vec, GloVe, and transformer-based embeddings from models like BERT and GPT.
+    - **Text:** Extending word embeddings, text embeddings represent entire sentences, paragraphs, or documents.
+    - **Images:** Image embeddings capture visual features and semantic information, useful for tasks like image classification and object detection.
+    - **Audio:** Audio embeddings capture features and characteristics of audio data for analysis and similarity metrics in applications like speech recognition.
+    - **Graphs:** Graph embeddings are essential for tasks in complex networks like social network analysis and fraud detection.
+- **Role in Large Language Models (LLMs)**
+    
+    - For LLMs, such as those used for ChatGPT, embeddings are taken a step further where **the context of every word, and the meanings of entire sentences, paragraphs, and articles, are converted into embeddings**.
+    - LLMs use embeddings to understand how different words relate to each other and the overall context of the language. This allows them to generate more coherent and contextually relevant responses.
+    - **Vector databases** are critical for LLMs, serving as a long-term memory to store and efficiently retrieve these high-dimensional embeddings. This enables capabilities like **Retrieval-Augmented Generation (RAG)**, where LLMs can access external knowledge bases to ground their responses in up-to-date and specific information.

docs/section1/Common Terminology/Inference.md

@@ -0,0 +1,63 @@
+# AI Inference: From Training to Action
+
+In artificial intelligence (AI), **inference** is the process where a **trained machine learning (ML) model** uses its learned knowledge to **draw conclusions or make predictions from new, unseen data**. This is distinct from the training phase, as inference involves the model **applying what it has learned** to make decisions or generate outputs **without requiring examples of the exact desired result**. Essentially, inference is the **AI model actively functioning or "in action"**.
+
+For example, a self-driving car recognizing a stop sign on a road it has never encountered before is performing inference. Similarly, a machine learning model trained on historical sports performance can make predictions about a player's future performance before they are signed to a contract, which is another instance of inference.
+
+### Inference vs. Training
+
+It is crucial to understand that machine learning inference and training are distinct phases within any machine learning project.
+
+- **Training**: This is the **initial phase** where an AI model learns patterns and correlations from large datasets, which can be structured or unstructured, labeled or unlabeled. Training often involves a process of trial and error, or showing the model examples of desired inputs and outputs. The model adjusts its internal parameters to optimize performance and minimise errors between its predictions and actual outcomes. Training is typically a **computationally intensive and expensive process**, but it is generally considered a one-time expense.
+- **Inference**: This process **follows AI training**. Once a model is properly trained and fine-tuned, it applies that training to new data to make additional inferences. The **better trained and fine-tuned a model is, the better its inferences will be**, though they are never guaranteed to be perfect. Inference is an **ongoing process** and can also be very expensive in terms of compute power, as the model constantly applies its training to new data.
+
+Understanding the difference between these two phases is important for allocating computational resources effectively. Using a pre-trained model for inference often requires fewer resources and can offer a competitive advantage in terms of time to market compared to training a model from scratch.
+
+### How Machine Learning Inference Works
+
+To deploy a machine learning inference environment, three main components are needed in addition to the trained ML model, which is typically software code implementing a mathematical algorithm:
+
+1. **Data Sources**: These systems capture **live data** from the mechanism that generates it.
+2. **Host System**: This system accepts data from the data sources and inputs it into the machine learning model.
+3. **Data Destinations**: This is where the host system delivers the output or score from the ML model. For instance, if a model calculates a fraud score, the destination applications might send an "approve" or "decline" message back to the purchase site.
+
+### The Role of Embeddings in Inference
+
+A key factor in how AI models, especially Large Language Models (LLMs), perform inference is their use of **embeddings**.
+
+- **Vector Embeddings** are numerical representations that convert various types of data—such as words, audio, or images—into **arrays of numbers (vectors)** that ML models can process. These vectors often have hundreds or thousands of dimensions, with each dimension representing a learned feature or attribute of the data.
+- **Semantic Proximity**: Embeddings capture the **semantic relationships and patterns** in the data. This means that semantically similar data points (e.g., synonyms or related concepts) are mapped to vectors that are close together in a high-dimensional space, while dissimilar items are farther apart. For example, the words "car" and "vehicle" have similar meanings and their vector representations will capture this semantic similarity.
+- **Querying for Inference**: When a user queries or prompts an AI model, the query itself is converted into a vector embedding. The vector database then calculates the distances between this query vector and the stored vectors to quickly find the most similar results, enabling fast and **context-aware retrieval**. This process, known as **similarity search**, allows searches based on the meaning or context of a query, rather than just keywords, leading to more precise and relevant results.
+
+Embeddings are foundational for AI because they enable machine learning models to understand relationships between words and other objects, not just exact matches. They are the backbone of generative AI applications like ChatGPT.
+
+### Use Cases for AI Inference
+
+Almost any real-world application of AI relies on inference. Some common examples include:
+
+- **Large Language Models (LLMs)**: LLMs can parse and interpret texts they have never seen before, leveraging their training to generate coherent and contextually relevant responses to user prompts. This includes tasks like answering questions, summarizing texts, translating languages, and generating creative writing or code.
+- **Predictive Analytics**: Once trained on past data, models can make predictions based on incoming data, such as forecasting market performance in finance or identifying trends in business intelligence.
+- **Email Security**: ML models can recognise spam emails or business email compromise attacks and make inferences about incoming messages to block malicious ones.
+- **Driverless Cars**: Inference is crucial for autonomous vehicles to recognise objects like stop signs and navigate safely.
+- **Research**: AI inference can be used to draw conclusions from scientific and medical data. For instance, deep learning models are used in medical imaging to support specialists in analysing images and assessing more images in less time.
+- **Conversational AI and Chatbots**: Inference enables chatbots and virtual assistants to understand user intent and respond appropriately by efficiently accessing and parsing relevant knowledge bases.
+- **Recommendation Engines**: These systems suggest products, movies, or articles by representing user preferences and item attributes as vectors and finding similar items based on vector similarity.
+- **Computer Vision**: This technology allows systems to extract, analyse, and comprehend spatial information from visual data to recognise objects, classify images, and make recommendations or take actions based on visual inputs.
+- **Retrieval-Augmented Generation (RAG)**: A key use case where LLMs retrieve relevant, up-to-date, and domain-specific information from external data sources (e.g., knowledge bases, databases) and integrate it into their responses, helping to overcome limitations like outdated knowledge or hallucinations.
+
+### Specialized Inference Techniques
+
+Beyond general inference, machine learning includes more specialised methods:
+
+- **Bayesian Inference**: This technique allows algorithms to make predictions by **updating their prior knowledge based on new evidence** using Bayes' theorem. It's more intuitive as it mimics how humans update their beliefs with new information. Applications include credit card fraud detection, medical diagnosis, image processing (noise removal), and speech processing (emotion recognition).
+- **Causal Inference**: This statistical approach is used in AI and machine learning to understand **cause-and-effect relationships** between attributes, rather than just correlations. It helps identify underlying causal relationships for more accurate predictions and can help mitigate bias by identifying its root causes in data. However, it requires large, high-quality data and strong statistical and domain expertise, and can be challenged by confounding variables.
+
+### Challenges and Considerations
+
+While inference is essential for making ML models useful in real-life applications, there are challenges:
+
+- **Computational Intensity**: Inference is an ongoing process that can be **very expensive in terms of compute power**, especially for large-scale deployments.
+- **Data Quality**: The effectiveness of inference heavily relies on the quality of the input data; poor source content can lead to inaccurate responses.
+- **Approximate Results**: Vector databases, often used for inference, frequently provide **approximate results**, which might not be suitable for applications requiring higher accuracy [internal knowledge, not directly in sources].
+- **Interpretability**: Some deep learning architectures, which underpin many modern AI systems, are considered "black boxes," making it difficult to fully understand why they produce certain results. This "lack of theory" surrounding some methods means confirmations are often empirical rather than theoretical.
+- **Bias and Unintended Outcomes**: AI models are trained on human-generated data, which can naturally contain flaws and biases. If biased information is fed into a machine learning program, the program will learn to replicate and perpetuate discrimination. This highlights the need for careful vetting of training data and human-centred AI practices.

docs/section1/Common Terminology/MCP.md

@@ -0,0 +1,68 @@
+# Model Context Protocol: AI's Universal Connector
+
+The **Model Context Protocol (MCP)** is an **open standard** that standardises how Artificial Intelligence (AI) applications, particularly **Large Language Models (LLMs)**, connect and interact with external tools, data sources, and systems. Its main goal is to solve the complexity and fragmentation that arises from integrating multiple AI models with a wide range of external services by providing a **universal, modular, and context-aware interface**.
+
+MCP is often compared to a **USB-C port for AI**, because it standardises connections between AI applications, LLMs, and external data sources, similar to how USB-C standardises how various peripherals connect to a laptop. This analogy highlights the adaptability MCP provides for connecting AI with different resources.
+
+### Why MCP Matters
+
+Generative AI models, including LLMs, are powerful but often **constrained by their isolation from external data**. LLMs are limited to the knowledge they acquired during their training, which can become outdated or lack specific domain information. Without external tools, an LLM cannot access real-time information. This can lead to generic or even incorrect responses, a phenomenon sometimes called "hallucinations".
+
+MCP addresses these challenges by:
+
+- **Providing access to specific and fresh information**. It enables AI systems, particularly **AI agents**, to retrieve and incorporate current, specific, and real-world data from various external sources such as databases, documentation, and APIs.
+- **Improving relevance, accuracy, and quality** of LLM responses by "grounding" them in up-to-date information.
+- **Reducing integration complexity**. Instead of needing custom integrations for every model-tool combination, MCP allows tool creators to build a single MCP server per tool and app developers to build a single MCP client per app, streamlining the process. It unifies LLMs and external services to communicate efficiently, offering a "plug-and-play" tool usage model.
+- **Supporting persistent, context-aware AI**. It enables stateful, multi-turn, and collaborative interactions, overcoming the limitations of stateless, one-off prompt-response models.
+- **Enabling modularity and reusability**. This makes it easier to integrate, debug, and extend AI capabilities across diverse workflows and systems.
+- **Enhancing security and control**. It facilitates secure, role-based access to data and tools and helps reduce hallucinations by grounding model outputs in real data.
+
+### Architecture and Components
+
+MCP uses a **client-host-server architecture**.
+
+| Component      | Role                                                                                                                     |
+| :------------- | :----------------------------------------------------------------------------------------------------------------------- |
+| **MCP Host**   | The application the user interacts with (e.g., Claude Desktop, IDEs, custom agents).                                     |
+| **MCP Client** | Lives within the host, managing a dedicated connection to one MCP server.                                                |
+| **MCP Server** | Lightweight external program that exposes specific tools, resources, and prompts to the AI model via a standardized API. |
+
+
+- **MCP Host**: This is the application that the user interacts with. It receives user requests and seeks access to context through the MCP. Examples include integrated development environments (IDEs) like Cursor or Claude Desktop apps. The host component contains the orchestration logic and can connect each client to a server.
+- **MCP Client**: The client exists within the host and manages a dedicated connection to one MCP server. It converts user requests into a structured format that the protocol can process. Multiple clients can exist with a single MCP host, but each client has a 1:1 relationship with an MCP server. Clients also handle session management, interruptions, timeouts, reconnections, parse responses, perform error handling, and ensure responses are relevant and appropriate.
+- **MCP Server**: This is a lightweight external program that exposes specific capabilities (tools, resources, and prompts) to the AI model via a standardised API. MCP servers act as middleware between AI models and their operational environment, parsing incoming context, storing relevant memory, and ensuring outputs align with the intended task. They can connect to both internal and external resources and tools. Examples of MCP server integrations include Slack, GitHub, Git, Docker, or web search.
+
+### MCP Server Details and Capabilities
+
+MCP servers expose capabilities through three main primitives:
+
+- **Tools**: These are **discrete actions or functions** that the AI model can call, such as APIs or calculators. For example, a weather service might expose a "get weather" tool, or a calendar service might expose a "create event" tool. When an LLM uses an MCP client to invoke a tool, the MCP server executes the underlying function. Tools can perform side effects or fetch data via an API request.
+- **Resources**: These are **read-only data items or documents** that the server can provide. Clients can retrieve these on demand. Examples include text files, database schemas, or file contents. Resources return data but do not execute actionable computations.
+- **Prompts**: These are **pre-defined templates** providing suggested prompts for optimal tool or resource usage.
+
+MCP servers also maintain a **context memory layer** that stores session, tool, and user-wise interactions, enabling persistent memory across sessions and collaborative workflows. They also define a **context schema**, typically in JSON, for representing contextual data to ensure interoperability and scalability.
+
+### Communication and Transport Layer
+
+Communication between MCP clients and servers occurs over a **JSON-RPC 2.0 session**. This transport layer is responsible for two-way message conversion, converting MCP protocol messages into JSON-RPC format for client-to-server communication, and back again for server-to-client messages.
+
+Two main transport methods are used:
+
+- **Standard Input/Output (stdio)**: Best for integrating **local resources** due to simple input/output transmission, used for lightweight, synchronous messaging.
+- **Server-Sent Events (SSE)**: Best for integrating **remote resources**, handling multiple asynchronous, event-driven server calls. HTTP POST requests are used for client-to-server messages, with SSE for the reverse.
+
+### Relationship with LLMs and AI Agents
+
+LLMs, without external tools, are skilled at subsequent text prediction, basic Q&A within their training data, sentiment analysis, and language translation. MCP provides LLMs with access to real-time, external information, which is crucial for AI agents to perform tasks, solve problems, and adapt to changing circumstances. AI agents use LLMs along with available tools, allowing the model to determine appropriate tool use, adapt to a changing environment, and form synthesised conclusions based on tool output.
+
+While traditional **Application Programming Interfaces (APIs)** also allow systems to access functionality or data, MCP is **purpose-built for LLM integration**. A key difference is that MCP supports **dynamic discovery**. An MCP client can ask an MCP server what functions and data it offers, allowing the LLM application to adapt to available capabilities without requiring a developer to update the client if the server's API changes. In contrast, traditional REST APIs don't typically expose such a runtime discovery mechanism. Furthermore, every MCP server follows the same protocol and patterns, regardless of the service or data it connects to, whereas each traditional API is unique.
+
+Interestingly, **many MCP servers actually use traditional APIs to do their work**, acting as a wrapper that translates between the MCP format and the underlying service's native interface. This means MCP and APIs are not adversaries but rather **layers in an AI stack**, with MCP providing a more AI-friendly interface on top.
+
+### Integration with RAG
+
+MCP can **supplement Retrieval-Augmented Generation (RAG)**. Instead of just providing a retriever to search a vector store or knowledge base, MCP can connect to a vector database through a server action. Searching the database as a tool, rather than passing the retriever in every LLM invocation, allows for a more strategic use of the tool and enables further tool-calling upon data retrieval.
+
+### Development and Community
+
+Anthropic introduced MCP in 2024 and is committed to building it as a **collaborative, open-source project and ecosystem**. Developers can start building and testing MCP connectors today, and pre-built MCP servers for popular enterprise systems like Google Drive, Slack, GitHub, Git, Postgres, and Puppeteer are available to help developers get started.

docs/section1/Common Terminology/RAG.md

@@ -0,0 +1,35 @@
+# Retrieval-Augmented Generation: Enhancing LLMs with External Knowledge
+
+**Retrieval-Augmented Generation (RAG)** is an Artificial Intelligence (AI) approach that **combines information retrieval with language generation** to create more accurate and contextually relevant outputs. It enhances Large Language Models (LLMs) by integrating them with external data sources.
+
+Here's a breakdown of what RAG means:
+
+- **Purpose and Goal**
+    
+    - The primary goal of RAG is to **enhance the accuracy and relevance of LLM responses** by "grounding" their outputs in up-to-date and specific real-world data.
+    - LLMs, by themselves, are limited to the knowledge contained in their training data, which can become outdated or lack domain-specific information. They can also sometimes generate plausible-sounding but incorrect information, a phenomenon known as "hallucinations," or provide vague and generic responses.
+    - RAG addresses these limitations by allowing LLMs to retrieve and incorporate current and specific information from various external sources, such as databases, documentation, and APIs. This process helps minimize contradictions and inconsistencies in the generated text, improving its quality and reliability.
+- **How RAG Works** RAG operates by following a series of steps to enhance generative AI outputs:
+    
+    1. **Data Collection:** First, all necessary external data for the application is gathered. This could include user manuals, product databases, or FAQs.
+    2. **Data Chunking:** The gathered data is then broken down into smaller, more manageable pieces or "chunks".
+    3. **Embedding and Indexing:** Each of these text chunks is converted into **numerical representations called vector embeddings**. Embeddings are dense, continuous vectors in a lower-dimensional space that capture semantic relationships, meaning similar data points are located closer together in this space. These embeddings are then stored and indexed in a **vector database**.
+    4. **Retrieval:** When a user poses a query, it is also converted into a vector embedding. The vector database then uses algorithms to quickly find the most similar stored vectors (i.e., relevant data chunks) to the query vector. This process is known as **vector search** or **semantic search**, as it looks for meaning rather than just keywords.
+    5. **Generation:** The retrieved text chunks, along with the original user query, are then fed into the LLM. The LLM uses this augmented context to generate a coherent, precise, and contextually relevant response. This effectively instructs the generative model to prioritise the retrieved information before giving its response, making it less likely to hallucinate or rely solely on its internal training data.
+- **Benefits of Using RAG**
+    
+    - **Access to Fresh Information:** RAG overcomes the limitation of LLMs being confined to their pre-trained data by providing them with up-to-date information.
+    - **Factual Grounding:** It helps LLMs remain factually accurate, even when dealing with complex or specific domain knowledge.
+    - **Improved Relevance, Accuracy, and Quality:** By grounding the LLM in relevant external knowledge, RAG significantly enhances the quality of generated text, making it more reliable and useful.
+    - **Efficient Inference:** RAG allows for faster and more efficient inference because the LLM doesn't need to re-process an entire massive dataset for every query. Once embeddings are created and stored, retrieval is quick.
+- **Role in LLMs and AI Agents**
+    
+    - RAG is a key technique for extending LLMs and providing them with "external memory" beyond their core training dataset.
+    - It is particularly useful for conversational AI, recommendation engines, and chatbots, allowing them to provide real-time, contextual answers.
+    - RAG also enhances the capabilities of **AI agents** by providing them with dynamic access to external context, which is crucial for agents to perform tasks, solve problems, and adapt to changing circumstances.
+- **Challenges**
+    
+    - Implementing RAG systems can involve **integration complexity**, especially when dealing with multiple external data sources in various formats.
+    - The **quality of the input data is critical**; poor or biased source content will lead to inaccurate responses. Human oversight is crucial for data curation to ensure reliability.
+
+In essence, RAG acts as a bridge, combining the broad language capabilities of LLMs with specific, accurate, and current information from external sources, making AI systems more reliable and effective for real-world applications.

docs/section1/Common Terminology/_category_.json

@@ -0,0 +1,7 @@
+{
+  "position": 4,
+  "link": {
+    "type": "generated-index",
+    "description": "5 minutes to learn the most important Docusaurus concepts."
+  }
+}