Name: My MCP Server Code Works, but the Agent Fails: The Case for MCP-specific Evaluations - Calum Murray & Wesley Chun, Red Hat
Start: 2026-04-02T15:35:00-0400
End: 2026-04-02T16:00:00-0400

April 2-3, 2026
New York, NY
View More Details & Registration

The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for MCP Dev Summit North America to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration..

IMPORTANT NOTE: Timing of sessions and room locations are subject to change.

My MCP Server Code Works, but the Agent Fails: The Case for MCP-specific Evaluations - Calum Murray & Wesley Chun, Red Hat

Thursday April 2, 2026 3:35pm - 4:00pm EDT

Marquis Ballroom (9th Floor)

You can write a perfect MCP server (clean code, typed schemas, 100% code coverage), yet agent interactions still fail. This is the probabilistic gap: your server is deterministic, but its user (the agent) is stochastic.

Standard “Agent Evals” are often the wrong tool to fix this. They judge the final outcome (was the answer good?), not the process. They struggle to provide useful insights into how the agent understands and uses your MCP server, instead focusing on providing insights into the agent itself.

In this session, we introduce mcpchecker, an open source framework for MCP server evaluations. We will show how to build integration tests specifically for the agent-MCP server interface, allowing you to isolate and debug these interactions.

Stop guessing why agents fail. Learn to test your server’s semantic interface and prove that agents can actually understand it.

Speakers

Calum Murray

Software Engineer, Red Hat

I am a Software Engineer at Red Hat, where I work on Applied AI projects with a focus on MCP and Agents. I also work on Serverless with the Knative community.

I am a CNCF ambassador, where I present about new and exciting technologies in the AI/Serverless as well as mentor new contributors... Read More →

Wesley Chun

Technical Program Manager - AI, Red Hat

WESLEY CHUN, MSCS, is a Google Developer Expert (GDE) in Google Cloud (GCP) & Google Workspace (GWS), author of Prentice Hall's bestselling "Core Python" series (corepython.com), co-author of "Python Web Development with Django", and has written for Linux Journal & CNET. He's currently... Read More →

mcpchecker mcpdevsummit pdf

Thursday April 2, 2026 3:35pm - 4:00pm EDT
Marquis Ballroom (9th Floor)

MCP Best Practices

Audience Experience Level Intermediate
Session Slides Yes

MCP Dev Summit North America

Calum Murray

Wesley Chun

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Get help with the event