GENAI110

Overview

Agent Development Kit (ADK) is a modular and extensible open-source framework for building AI agents. While ADK provides its own built-in evaluation module, this lab demonstrates how to use the Agent Platform Generative AI Evaluation Service to assess the performance of an ADK-based agent. This approach offers a broader, explainable, and quality-controlled toolkit to evaluate generative models or applications using custom metrics and human-aligned benchmarks.

In this lab, you will walk through a step-by-step guide to evaluate your ADK agent using Agent Platform Gen AI Evaluation.

This lab is based on this notebook from the GoogleCloudPlatform/generative-ai repo.

Objective

In this lab, you learn how to perform the following tasks:

• Build and run a local ADK agent.
• Create and format an agent evaluation dataset.
• Evaluate agent performance using:
  • Single tool usage evaluation.
  • Trajectory-based evaluation.
  • Response quality evaluation.
• Use Agent Platform’s Evaluation Service to generate explainable metrics and benchmark results.

Setup and requirements

Before you click the Start Lab button

Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources are made available to you.

This hands-on lab lets you do the lab activities in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials you use to sign in and access Google Cloud for the duration of the lab.

To complete this lab, you need:

• Access to a standard internet browser (Chrome browser recommended).

• Time to complete the lab—remember, once you start, you cannot pause a lab.

How to start your lab and sign in to the Google Cloud console

1. Click the Start Lab button. If you need to pay for the lab, a dialog opens for you to select your payment method. On the right is the Lab setup pane with the following:

   • The Open Google Cloud console button
   • The temporary credentials that you must use for this lab
   • Other information, if needed, to step through this lab

2. In the Lab Setup section, right-click Open Google Cloud console and select Open link in new window (or alternatively right-click and select Open Link in Incognito Window if you are running the Chrome browser).

   The lab spins up resources, and then opens another tab that shows the Sign in page.

   Tip: Arrange the tabs in separate windows, side-by-side.

   Note: If you see the Choose an account dialog, click Use Another Account.

3. If necessary, copy the Username below and paste it into the Sign in dialog.

   {{{user_0.username | "Username"}}}

   You can also find the Username in the Lab Setup pane.

4. Click Next.

5. Copy the Password below and paste it into the Welcome dialog.

   {{{user_0.password | "Password"}}}

   You can also find the Password in the Lab Setup pane.

6. Click Next.

   Important: You must use the credentials the lab provides you. Do not use your Google Cloud account credentials. Note: Using your own Google Cloud account for this lab may incur extra charges.

7. Click through the subsequent pages:

   • Accept the terms and conditions.
   • Do not add recovery options or two-factor authentication (because this is a temporary account).
   • Do not sign up for free trials.

After a few moments, the Google Cloud console opens in this tab.

Note: To access Google Cloud products and services, click the Navigation menu or type the service or product name in the Search field.

Task 1. Prepare the environment in Agent Platform Workbench

1. In the Google Cloud console, from the Navigation menu (), select Agent Platform > Notebooks > Workbench.

2. Click Enable APIs, and then click Enable to confirm.

3. Under Instances, click on Open JupyterLab next to your vertex-ai-jupyterlab instance. JupyterLab will launch in a new tab.

4. On the JupyterLab Launcher page, click Terminal to open a Shell.

5. Download a notebook and helper file for this lab with the command:

   gcloud storage cp gs://{{{project_0.project_id| YOUR_GCP_PROJECT_ID}}}-bucket/* .

6. Open the file.

7. In the Select Kernel dialog, choose Python 3 (Local) from the list of available kernels.

8. Run the first code cell under the header Install Agent Development Kit (ADK) and other required packages to install dependencies. To run a cell, click the play button at the top or highlight the cell and press SHIFT+ENTER on your keyboard.

9. To use the newly installed packages in this Jupyter runtime, you must restart the kernel. Wait for the [*] beside the cell to change to [1] to show that the cell has completed, then in the Jupyter Lab menus, select Kernel > Restart Kernel and Clear Outputs of All Cells....

10. When prompted to confirm, select Restart.

11. Once the kernel has restarted, run the next cell under Set Google Cloud project information to set Google Cloud project information and initialize Agent Platform.

12. Run the next cell in the Import libraries section of the notebook.

Click Check my progress to verify the objectives. Set Google Cloud project information

Task 2. Read and run the rest of the notebook

Read and run through the cells of the notebook. In its sections, you will:

• Set tools to define get_product_details() and get_product_price() functions that your agent will later use as tools.
• Set the Model to define the Gemini model your agent will use.
• Assemble the agent by defining a function that builds an Agent, providing it the tools you've defined above, and uses it to generate a response to a query.
• Test the Agent to see a couple of example responses, including Markdown-formatted output citing which functions were called.

Click Check my progress to verify the objectives. Build ADK agent

• Learn about Evaluating an ADK agent with Agent Platform Gen AI Evaluation.
• Prepare an Agent Evaluation Dataset consisting of a set of prompts and their expected tool calls (reference trajectory).
• Run Single tool usage evaluation to determine if the agent is choosing the correct single tool for a given task.
• Run Trajectory Evaluation to assess whether the agent not only chooses the right tools but also utilizes them in your intended order.
• Learn to Visualize Evaluation Results.
• Learn to Define a custom metric and evaluate on it.

Click Check my progress to verify the objectives. Evaluating an ADK agent with Agent Platform Gen AI Evaluation

• A bonus Bring-Your-Own-Dataset (BYOD) section shows you how to evaluate agent responses generated elsewhere by providing an evaluation dataset containing those responses.

Click Check my progress to verify the objectives. Perform BYOD evaluation of LangGraph agents using GenAI

Click Check my progress to verify the objectives. Clean up the experiment

Congratulations!

You have successfully evaluated an ADK agent using Agent Platform Generative AI Evaluation. You built and executed the agent locally, prepared a custom evaluation dataset, and assessed the agent’s tool usage, action trajectory, and final response quality using built-in evaluation tools.

You evaluated the agent using the following components:

• ADK (Agent Development Kit) for building and running the agent
• A structured evaluation dataset with expected tool usage and references
• Agent Platform Gen AI Evaluation Service
• Tool usage evaluation metrics
• Trajectory matching for step-by-step accuracy
• Evaluation on a custom metric

Manual Last Updated August 28, 2026

Lab Last Tested August 28, 2026

Copyright 2026 Google LLC. All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.