GenAI Interview Questions — RAG and Agents

The Core Mechanics of RAG

Retrieval-Augmented Generation works by addressing the inherent limitation of static model weights: they are frozen in time and lack access to private, proprietary datasets. By decoupling the knowledge base from the language model, RAG allows a system to query a vector database for relevant information before generating a response. When a user asks a question, the system first transforms that query into a semantic vector, then performs a similarity search to retrieve the most pertinent document chunks. This context is then injected into the prompt as a source of truth, effectively grounding the model's output. The genius of this approach lies in reducing hallucinations, as the model is forced to adhere to the provided retrieved documents rather than relying solely on its internal, potentially outdated training parameters. Understanding this retrieval loop is critical for troubleshooting low-precision results in production systems.

# Simple retrieval logic using cosine similarity on document embeddings
import numpy as np

def retrieve_context(query_vec, document_vecs, documents, top_k=2):
    # Calculate cosine similarity between query and all stored documents
    scores = np.dot(document_vecs, query_vec) / (np.linalg.norm(document_vecs, axis=1) * np.linalg.norm(query_vec))
    # Get indices of the top k highest scoring documents
    top_indices = np.argsort(scores)[-top_k:][::-1]
    return [documents[i] for i in top_indices]

Vector Stores and Semantic Search

Vector databases are the storage engines that make large-scale retrieval feasible within milliseconds. Instead of traditional keyword matching, these stores utilize high-dimensional vector spaces where semantic similarity translates to proximity in that space. When you embed text, you transform linguistic structures into numerical sequences that represent meaning. During search, the database performs a nearest-neighbor calculation to find vectors close to the query vector. The reason this works better than simple keyword lookup is that it handles synonymy and context; a query about 'monetary policy' can successfully retrieve documents about 'interest rate adjustments' even if the exact keywords do not overlap. Maintaining a healthy vector index requires careful consideration of chunking strategies, as the granularity of your text chunks dictates the precision of the information provided to the model during the prompt generation phase.

# Mock indexing for a vector database
class VectorStore:
    def __init__(self): self.index = []
    def add(self, vector, metadata):
        # Stores the vector embedding with associated document source
        self.index.append({'vec': vector, 'meta': metadata})
    def search(self, query_vec):
        # Return documents based on proximity to the query
        return sorted(self.index, key=lambda x: np.dot(x['vec'], query_vec), reverse=True)

Prompt Engineering for Contextual Grounding

Once the relevant information is retrieved, the challenge shifts to how you integrate it into the model's prompt. The goal is to enforce a strict constraint where the model prioritizes the retrieved context over its pre-trained memory. This is achieved through systematic prompt engineering, where you define clear delimiters for the context and explicitly instruct the model to answer only based on provided materials. By structuring the prompt with a clear 'context' block and a 'question' block, you minimize the risk of the model blending facts. Furthermore, specifying the output format—such as asking for citations from the context—creates a feedback mechanism that encourages the model to 'show its work'. If the model ignores instructions, it is often because the retrieval step returned irrelevant data, which demonstrates that the bottleneck is often the quality of the search rather than the prompt itself.

# Formatting retrieved documents into a grounded system prompt
def construct_prompt(query, context_list):
    # Injecting context with clear demarcations to reduce hallucination
    context_str = "\n".join([f"Source {i}: {doc}" for i, doc in enumerate(context_list)])
    return f"Context:\n{context_str}\n\nQuestion: {query}\nAnswer based strictly on the context above."

Introduction to AI Agents

Agents represent an evolution from passive text generation to active task execution. While a RAG system provides facts, an Agent possesses a loop of perception, reasoning, and action. The Agent is given a goal, a set of available tools, and a scratchpad to track its thought process. It breaks a complex query into a sequence of sub-tasks, deciding which tool—such as a calculator, a database query function, or a web search API—is needed to progress. This works because the language model acts as an internal controller, parsing its own outputs to select the next action. The primary challenge in building Agents is managing the 'reasoning loop,' where the agent must determine when it has sufficient information to formulate a final answer, preventing infinite tool-calling cycles that waste resources and degrade response quality.

# A basic ReAct (Reasoning and Acting) loop pattern
def agent_step(task, tools):
    thought = "I need to check the inventory to answer this request."
    # Selecting the tool based on the model's logical reasoning
    tool_call = tools['inventory_lookup'].run(task)
    return f"Thought: {thought}\nResult: {tool_call}"

Tool Orchestration and Error Handling

Orchestrating tools effectively requires robust error handling and input validation. Because Agents operate by calling functions defined in code, the model might occasionally pass malformed arguments or attempt to use tools that do not exist. To mitigate this, developers should implement a 'guardrail' layer that sanitizes model output before it executes a function. This acts as a safety gate, ensuring that the model's autonomous decision-making does not lead to unauthorized system changes or memory overflows. Additionally, logging the agent's thought chain is critical for debugging; since agents make non-deterministic decisions, you need to see exactly why it chose a particular tool path. By wrapping each tool execution in a try-except block, you allow the agent to receive an error message and attempt a corrective action, which creates a self-healing loop that is essential for reliable autonomous workflows.

# Tool wrapper with safety guardrails
def execute_tool(tool_name, tool_func, params):
    try:
        # Validate parameters before execution to prevent system crashes
        if not isinstance(params, dict): raise ValueError("Invalid input format")
        return tool_func(**params)
    except Exception as e:
        # Return error to model so it can re-plan its task
        return f"Tool execution failed: {str(e)}. Attempt a different strategy."

Interview questions

All Generative AI interview questions →

Check yourself

Take the full Generative AI quiz →