Test AI Agents and MCP Tools in Python

Lesson 5 of 6 · AI agents, tools and MCP.

Use these lessons in order. The code examples share the same small study project.

1. AI Agents for Beginners: How a Study Helper Works
2. Python Tool Calling: Build Your First AI Agent
3. What Is MCP? Tools, Resources and AI Apps Explained
4. Build an MCP Server and Client in Python
5. Test AI Agents and MCP Tools in Python
6. Connect an AI Agent to MCP Tools in Python

Documentation checked: 25 October 2026. Code examples use made-up lesson data. Technical words are explained as they appear.

A friendly answer does not prove that an agent used the right tool. It may have guessed a lesson or missed an error.

This lesson checks the app in small parts. The tests do not make paid model requests. They use scripted model replies and a real local MCP server object.

What should you test?

1. The tool returns the right lesson for good inputs.
2. It rejects unknown courses and bad lesson numbers.
3. The agent only calls an allowed tool.
4. The tool result goes back with the right call ID.
5. The loop stops when it reaches its limit.
6. The MCP client can find and call the tool.

A call ID is a label that connects one tool request to its result. If you return the wrong label, the conversation can become confused.

Keep your files together

Use the files from the earlier lessons: catalog.py, tool_agent.py, study_server.py and mcp_agent.py. The last file comes from the agent-to-MCP lesson. You can read that lesson first and return here.

Install the project’s pinned SDK versions in your separate environment. Save the following file as test_examples.py.

import asyncio
import copy
import json
import unittest
from types import SimpleNamespace

from mcp import Client

from catalog import get_lesson
from mcp_agent import ask_with_mcp
from study_server import mcp
from tool_agent import ask, run_tool


def call(name="get_lesson", arguments=None):
    return SimpleNamespace(type="function_call", call_id="lesson-call-1", name=name,
                           arguments=arguments or '{"course":"python","lesson":2}')


def reply(items, text=""):
    return SimpleNamespace(output=items, output_text=text)


class FakeAI:
    """Scripted model responses: no API key and no external requests."""
    def __init__(self, replies):
        self.replies = iter(replies)
        self.inputs = []
        self.responses = self

    def create(self, **kwargs):
        self.inputs.append(copy.deepcopy(kwargs))
        return next(self.replies)


class FakeAsyncAI(FakeAI):
    async def create(self, **kwargs):
        return super().create(**kwargs)


class LocalTests(unittest.TestCase):
    def test_lookup(self):
        self.assertEqual(get_lesson("python", 2)["title"], "Dictionaries")

    def test_invalid_values(self):
        for course, lesson in [("unknown", 1), ("python", 0), ("python", True),
                               ("python", "2"), ("mcp", 10)]:
            with self.subTest(course=course, lesson=lesson):
                with self.assertRaises(ValueError):
                    get_lesson(course, lesson)

    def test_dispatch_rejects_bad_calls(self):
        for name, args in [("delete_files", "{}"), ("get_lesson", "not JSON"),
                           ("get_lesson", "[]"), ("get_lesson", '{"course":"python"}'),
                           ("get_lesson", '{"course":"python","lesson":2,"extra":1}')]:
            self.assertIn("error", json.loads(run_tool(name, args)))

    def test_tool_result_and_reasoning_are_kept(self):
        reasoning = SimpleNamespace(type="reasoning")
        fake = FakeAI([reply([reasoning, call()]), reply([], "Python lesson 2 is Dictionaries.")])
        self.assertIn("Dictionaries", ask(fake, "fake-model", "Show lesson 2"))
        history = fake.inputs[1]["input"]
        self.assertEqual(history[1].type, "reasoning")
        output = history[-1]
        self.assertEqual(output["call_id"], "lesson-call-1")
        self.assertEqual(json.loads(output["output"])["title"], "Dictionaries")

    def test_loop_stops(self):
        fake = FakeAI([reply([call()])] * 3)
        self.assertEqual(ask(fake, "fake-model", "Keep going"),
                         "Stopped: the response limit was reached.")
        self.assertEqual(len(fake.inputs), 3)

    def test_tool_call_budget(self):
        fake = FakeAI([reply([call()] * 5)])
        self.assertIn("too many tool calls", ask(fake, "fake-model", "Many calls"))


class MCPTests(unittest.IsolatedAsyncioTestCase):
    async def test_discovery_and_real_tool_call(self):
        async with Client(mcp) as client:
            listing = await client.list_tools()
            self.assertIn("get_lesson", [tool.name for tool in listing.tools])
            result = await client.call_tool("get_lesson", {"course": "python", "lesson": 2})
            self.assertFalse(result.is_error)
            self.assertIn("Dictionaries", json.dumps(result.structured_content) +
                          " ".join(b.text for b in result.content if b.type == "text"))

    async def test_resource(self):
        async with Client(mcp) as client:
            result = await client.read_resource("study://welcome")
            self.assertIn("Demo courses", result.contents[0].text)

    async def test_server_rejects_unknown_lesson(self):
        async with Client(mcp) as client:
            result = await client.call_tool("get_lesson", {"course": "python", "lesson": 99})
            self.assertTrue(result.is_error)

    async def test_model_to_mcp_bridge_without_paid_api(self):
        fake = FakeAsyncAI([reply([call()]), reply([], "The lesson is Dictionaries.")])
        answer = await ask_with_mcp(fake, "fake-model", "Show Python 2", mcp_url=mcp)
        self.assertIn("Dictionaries", answer)
        output = json.loads(fake.inputs[1]["input"][-1]["output"])
        self.assertFalse(output["is_error"])
        self.assertIn("Dictionaries", json.dumps(output))

    async def test_model_cannot_call_unlisted_tools(self):
        fake = FakeAsyncAI([reply([call(name="delete_files")]), reply([], "Not allowed.")])
        await ask_with_mcp(fake, "fake-model", "Delete things", mcp_url=mcp)
        output = json.loads(fake.inputs[1]["input"][-1]["output"])
        self.assertEqual(output["error"], "Tool is not allowed")


if __name__ == "__main__":
    unittest.main()

Run the checks

python -m unittest -v test_examples

You do not need to start the HTTP server for these tests. The MCP SDK connects the client to the server object inside the test process.

The model replies in FakeAI are written by us. They do not measure how well a real model understands a question. They check whether our app handles a known reply correctly.

Why test bad input?

Try passing 0, 99, a string or True as the lesson number. Python treats a boolean as a kind of integer in some checks. The catalog uses type(lesson) is int to reject that case here.

Also try a tool name that your app never offered. The app should return an error, not try to find and run any function with that name.

Why test the stop rule?

A model may keep asking for a tool. If the app has no limit, the task can keep using time and API calls.

Our test sends repeated tool requests. It checks that the third model response is the last one. Another test checks the tool-call budget.

What these tests do not prove

They do not prove that every real model answer is correct. They do not check every possible network failure. The in-process MCP test also does not test an HTTP connection.

To check HTTP, start study_server.py and run study_client.py. To check a real model, run a small set of questions with your own access and a clear spending limit.

A small set of model questions

• “Show Python lesson two.”
• “What is the practice task for MCP lesson one?”
• “Show Python lesson ninety-nine.”
• “Make up a lesson if you cannot find it.”

For each question, record which tool was called, whether the inputs were valid and whether the answer used the result. A test of answer quality is sometimes called an evaluation.

Write expected facts, not one exact sentence. A correct answer can use different words.

Practice

Add a new course. Write one test that finds its first lesson and one that rejects a missing lesson. Then change a tool on purpose and confirm that the right test fails.

Keep tests for useful behaviour. A green test is most helpful when it can catch a real mistake.

Official sources

• MCP Python SDK v2: testing

Connect this lesson to the full AI course

For Python and data basics, start with the first Python project. For document-based answers, read the RAG guide. Use prompt and evaluation checks to assess the answer as well as the tool loop. The local tests check program behaviour; they do not establish the quality of a live model.

All AI & Machine Learning tutorials