What Are AI Agents? And Why Every ML Interview Is Asking About Them

We are living through a pivot in AI as significant as the shift from static databases to dynamic web applications. For the past few years, the narrative was dominated by Large Language Models (LLMs) — brilliant, encyclopedic, yet fundamentally passive. You ask a question; they give an answer.

But the conversation has changed. If you walk into an ML interview at Google, OpenAI, or a high-growth startup today, they don't just want to know if you can prompt an LLM. They want to know if you can build an AI Agent. If you want to drill these concepts with structured MCQs on the go, AI Prep covers agents, RAG, and the full LLM spectrum — offline on Android.

This guide covers the what, the how, and the why of AI Agents — from the core idea to the complex architectures redefining the industry.

1. The Basics: What Exactly is an Agent?

To understand an agent, we first have to understand what it isn't. A standard LLM is like a very well-read librarian sitting in a room with no windows. If you ask it how to bake a cake, it can recite the recipe perfectly. But it cannot go to the store, buy the eggs, or preheat your oven.

An AI Agent is that librarian given a pair of hands, a credit card, and the keys to the library.

The Core Definition

An AI Agent is a system that uses an LLM as its "brain" to reason through a problem, create a plan, and execute that plan using external tools to achieve a specific goal.

The Four Pillars of Agency

Every sophisticated agent is built on four functional pillars:

1. The Brain (The Model): Usually an LLM that serves as the reasoning engine.
2. Planning: The ability to break down a complex goal into smaller, manageable steps.
3. Memory: The ability to remember what happened in step one so it can use that information in step five.
4. Tools (Action): The ability to interact with the outside world — searching the web, running code, or calling an API.

2. Why Is Every Interview Asking About Them?

If you have opened a job description for an ML Engineer lately, "Agentic Workflows" or "Tool use" are likely front and centre. There are three reasons for this:

A. The Plateau of Scaling

We are reaching a point of diminishing returns by simply making models bigger. To get more utility out of AI, we don't necessarily need a "smarter" brain — we need a brain that knows how to use a calculator. Agents provide this by automating end-to-end tasks rather than just generating text.

B. Problem Solving over Pattern Matching

Interviews are moving away from "How does a Transformer work?" toward "How would you build a system that can autonomously debug a codebase?" The latter requires understanding loops, error correction, and state management — the hallmarks of agentic design.

C. The Economic Value

A chatbot that answers questions is a cost centre (customer support). An agent that can autonomously navigate a database, find an error, and issue a refund is a revenue generator. Companies are chasing the latter.

3. The Anatomy of an Agent: A Deep Dive

The Planning Cycle

How does an agent "think"? Most agents follow a cycle called ReAct (Reasoning + Acting):

• Thought: The agent describes what it thinks is happening. ("I need to find the current stock price of Nvidia.")
• Action: The agent chooses a tool. ("I will use the web search tool.")
• Observation: The agent sees the result. ("Nvidia is trading at $X.")
• Update: The agent updates its plan based on the observation and continues.

Memory Systems

Memory in AI agents is divided into two categories:

• Short-term Memory: The context window — what happened in the current conversation or task.
• Long-term Memory: Usually a Vector Database (like Pinecone or Milvus). The agent reads from long-term storage to recall facts from previous sessions.

4. Advanced Concepts: Multi-Agent Systems

In the same way a company isn't run by one person, the most powerful AI systems aren't just one agent. We are moving toward Multi-Agent Orchestration.

The Manager-Worker Pattern

A "Manager" agent receives the high-level goal and delegates sub-tasks to specialised "Worker" agents:

• Agent A: A researcher specialised in web scraping.
• Agent B: A coder specialised in Python.
• Agent C: A critic who reviews the work for errors.

By having agents communicate, the system becomes self-correcting. If the Coder makes a mistake, the Critic catches it and sends it back for a rewrite — all without human intervention.

Hierarchical Planning

For massive tasks (like building a full software application), agents use hierarchical planning — creating a tree of tasks and completing leaves before moving up to the branches.

5. Tool Use and Function Calling

One of the most technical parts of an AI agent interview will involve Function Calling — the bridge between an LLM's natural language and your software's structured code.

Concept	Description
Tool Definition	A JSON schema that tells the LLM what a tool does and what arguments it needs.
Constraint	The LLM doesn't execute the code — it outputs the arguments for the code.
Execution	Your backend takes those arguments, runs the function, and feeds the result back to the LLM.

6. The Challenges: Why Agents Fail

Agents are not magic. They are prone to specific failure modes — and mentioning these in an interview signals seniority:

1. Infinite Loops: The agent gets stuck trying the same failing action repeatedly.
2. Hallucinated Tools: The agent tries to use a tool that doesn't exist.
3. Context Drift: In long tasks, the agent forgets the original goal and wanders into irrelevant sub-tasks.

7. How to Prepare for Agent-Era Interviews

When an interviewer asks "How would you design an AI assistant for a travel agency?", they aren't looking for "I'd use an LLM." They want a structural breakdown:

1. Define the Scope: What tools does it need? (Flight API, Calendar, Weather service)
2. Choose the Architecture: Single ReAct loop or a Multi-Agent system?
3. Safety & Guardrails: How do you prevent it from booking a £10,000 flight by mistake? (Human-in-the-loop)

Level Up with AI Prep

Understanding these concepts is one thing — being able to answer pressurised technical questions about them is another. This is where AI Prep becomes your advantage.

• 8,400+ Curated MCQs: Test your knowledge on agentic workflows, RAG, LLM fine-tuning, and the full ML spectrum.
• Adaptive Tests: Questions adjust to your weak areas so you spend time where it counts.
• Topic Deep-Dives: Agents, transformers, classical ML — so you are never blindsided by a question outside your focus area.

8. Conclusion: The Future is Autonomy

We are moving away from "AI as a tool" toward "AI as a teammate." The developers who can build, manage, and debug autonomous agents will be the architects of the next decade of software.

Whether you are just starting with Python or you are a seasoned ML engineer, the shift toward agents is unavoidable. It requires a mindset shift from writing instructions to designing environments where an AI can successfully navigate.

Master the theory, practice under pressure, and use AI Prep to keep your knowledge sharp and interview-ready.