{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 追踪与可观测性\n\nAutoGen 内置了[追踪功能支持](https://microsoft.github.io/autogen/dev/user-guide/core-user-guide/framework/telemetry.html)，可收集应用程序执行的全面记录。此功能对调试、性能分析和理解应用流程非常有用。\n\n该能力由[OpenTelemetry](https://opentelemetry.io/)库提供支持，这意味着您可以使用任何兼容OpenTelemetry的后端来收集和分析追踪数据。\n\n## 设置\n\n首先需要安装OpenTelemetry Python包。可通过pip安装：\n\n```bash\npip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-grpc\n```\n\n安装SDK后，在AutoGen中设置追踪的最简单方法是：\n\n1. 配置OpenTelemetry追踪提供程序\n2. 设置导出器将追踪数据发送到后端\n3. 将追踪提供程序连接到AutoGen运行时\n\n## 遥测后端\n\n要收集和查看追踪数据，需要设置遥测后端。有多个开源选项可用，包括Jaeger、Zipkin。本示例我们将使用Jaeger作为遥测后端。\n\n快速启动方式是通过Docker本地运行Jaeger：\n\n```bash\ndocker run -d --name jaeger \\\n -e COLLECTOR_OTLP_ENABLED=true \\\n -p 16686:16686 \\\n -p 4317:4317 \\\n -p 4318:4318 \\\n jaegertracing/all-in-one:latest\n```\n\n此命令启动的Jaeger实例会在16686端口监听Jaeger UI，4317端口监听OpenTelemetry收集器。可通过`http://localhost:16686`访问Jaeger UI。\n\n## 为AgentChat团队添加追踪\n\n接下来我们将了解如何为AutoGen GroupChat团队启用追踪。AutoGen运行时已支持OpenTelemetry（自动记录消息元数据）。首先创建一个追踪服务用于检测AutoGen运行时。\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from opentelemetry import trace\n", "from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter\n", "from opentelemetry.sdk.resources import Resource\n", "from opentelemetry.sdk.trace import TracerProvider\n", "from opentelemetry.sdk.trace.export import BatchSpanProcessor\n", "\n", "otel_exporter = OTLPSpanExporter(endpoint=\"http://localhost:4317\", insecure=True)\n", "tracer_provider = TracerProvider(resource=Resource({\"service.name\": \"autogen-test-agentchat\"}))\n", "span_processor = BatchSpanProcessor(otel_exporter)\n", "tracer_provider.add_span_processor(span_processor)\n", "trace.set_tracer_provider(tracer_provider)\n", "\n", "# 稍后我们将通过服务名引用此追踪器 tracer = trace.get_tracer(\"autogen-test-agentchat\")\n", "# " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "创建[团队](./tutorial/teams.ipynb)的所有代码您应该都已熟悉。需要注意的是，所有AgentChat代理和团队都使用AutoGen核心API运行时运行。而该运行时已内置检测功能，可记录[运行时消息事件（元数据）](https://github.com/microsoft/autogen/blob/main/python/packages/autogen-core/src/autogen_core/_telemetry/_tracing_config.py)，包括：\n\n- **create**：消息创建时\n- **send**：消息发送时\n- **publish**：消息发布时\n- **receive**：消息接收时\n- **intercept**：消息拦截时\n- **process**：消息处理时\n- **ack**：消息确认时\n" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from autogen_agentchat.agents import AssistantAgent\n", "from autogen_agentchat.conditions import MaxMessageTermination, TextMentionTermination\n", "from autogen_agentchat.teams import SelectorGroupChat\n", "from autogen_agentchat.ui import Console\n", "from autogen_core import SingleThreadedAgentRuntime\n", "from autogen_ext.models.openai import OpenAIChatCompletionClient\n", "\n", "\n", "def search_web_tool(query: str) -> str:\n", " if \"2006-2007\" in query:\n", " return \"\"\"Here are the total points scored by Miami Heat players in the 2006-2007 season:\n", " Udonis Haslem: 844 points\n", " Dwayne Wade: 1397 points\n", " James Posey: 550 points\n", " ...\n", " \"\"\"\n", " elif \"2007-2008\" in query:\n", " return \"The number of total rebounds for Dwayne Wade in the Miami Heat season 2007-2008 is 214.\"\n", " elif \"2008-2009\" in query:\n", " return \"The number of total rebounds for Dwayne Wade in the Miami Heat season 2008-2009 is 398.\"\n", " return \"No data found.\"\n", "\n", "\n", "def percentage_change_tool(start: float, end: float) -> float:\n", " return ((end - start) / start) * 100\n", "\n", "\n", "async def main() -> None:\n", " model_client = OpenAIChatCompletionClient(model=\"gpt-4o\")\n", "\n", " planning_agent = AssistantAgent(\n", " \"PlanningAgent\",\n", " description=\"An agent for planning tasks, this agent should be the first to engage when given a new task.\",\n", " model_client=model_client,\n", " system_message=\"\"\"\n", " You are a planning agent.\n", " Your job is to break down complex tasks into smaller, manageable subtasks.\n", " Your team members are:\n", " WebSearchAgent: Searches for information\n", " DataAnalystAgent: Performs calculations\n", "\n", " You only plan and delegate tasks - you do not execute them yourself.\n", "\n", " When assigning tasks, use this format:\n", " 1. : \n", "\n", " After all tasks are complete, summarize the findings and end with \"TERMINATE\".\n", " \"\"\",\n", " )\n", "\n", " web_search_agent = AssistantAgent(\n", " \"WebSearchAgent\",\n", " description=\"An agent for searching information on the web.\",\n", " tools=[search_web_tool],\n", " model_client=model_client,\n", " system_message=\"\"\"\n", " You are a web search agent.\n", " Your only tool is search_tool - use it to find information.\n", " You make only one search call at a time.\n", " Once you have the results, you never do calculations based on them.\n", " \"\"\",\n", " )\n", "\n", " data_analyst_agent = AssistantAgent(\n", " \"DataAnalystAgent\",\n", " description=\"An agent for performing calculations.\",\n", " model_client=model_client,\n", " tools=[percentage_change_tool],\n", " system_message=\"\"\"\n", " You are a data analyst.\n", " Given the tasks you have been assigned, you should analyze the data and provide results using the tools provided.\n", " If you have not seen the data, ask for it.\n", " \"\"\",\n", " )\n", "\n", " text_mention_termination = TextMentionTermination(\"TERMINATE\")\n", " max_messages_termination = MaxMessageTermination(max_messages=25)\n", " termination = text_mention_termination | max_messages_termination\n", "\n", " selector_prompt = \"\"\"Select an agent to perform task.\n", "\n", " {roles}\n", "\n", " Current conversation context:\n", " {history}\n", "\n", " Read the above conversation, then select an agent from {participants} to perform the next task.\n", " Make sure the planner agent has assigned tasks before other agents start working.\n", " Only select one agent.\n", " \"\"\"\n", "\n", " task = \"Who was the Miami Heat player with the highest points in the 2006-2007 season, and what was the percentage change in his total rebounds between the 2007-2008 and 2008-2009 seasons?\"\n", "\n", " tracer = trace.get_tracer(\"autogen-test-agentchat\")\n", " with tracer.start_as_current_span(\"runtime\"):\n", " team = SelectorGroupChat(\n", " [planning_agent, web_search_agent, data_analyst_agent],\n", " model_client=model_client,\n", " termination_condition=termination,\n", " selector_prompt=selector_prompt,\n", " allow_repeated_speaker=True,\n", " )\n", " await Console(team.run_stream(task=task))\n", "\n", " await model_client.close()\n", "\n", "\n", "# asyncio.run(main())" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "---------- user ----------\n", "Who was the Miami Heat player with the highest points in the 2006-2007 season, and what was the percentage change in his total rebounds between the 2007-2008 and 2008-2009 seasons?\n", "---------- PlanningAgent ----------\n", "To accomplish this, we can break down the tasks as follows:\n", "\n", "1. WebSearchAgent: Search for the Miami Heat player with the highest points during the 2006-2007 NBA season.\n", "2. WebSearchAgent: Find the total rebounds for the identified player in both the 2007-2008 and 2008-2009 NBA seasons.\n", "3. DataAnalystAgent: Calculate the percentage change in total rebounds for the player between the 2007-2008 and 2008-2009 seasons.\n", "\n", "Once these tasks are complete, I will summarize the findings.\n", "---------- WebSearchAgent ----------\n", "[FunctionCall(id='call_PUhxZyR0CTlWCY4uwd5Zh3WO', arguments='{\"query\":\"Miami Heat highest points scorer 2006-2007 season\"}', name='search_web_tool')]\n", "---------- WebSearchAgent ----------\n", "[FunctionExecutionResult(content='Here are the total points scored by Miami Heat players in the 2006-2007 season:\\n Udonis Haslem: 844 points\\n Dwayne Wade: 1397 points\\n James Posey: 550 points\\n ...\\n ', name='search_web_tool', call_id='call_PUhxZyR0CTlWCY4uwd5Zh3WO', is_error=False)]\n", "---------- WebSearchAgent ----------\n", "Here are the total points scored by Miami Heat players in the 2006-2007 season:\n", " Udonis Haslem: 844 points\n", " Dwayne Wade: 1397 points\n", " James Posey: 550 points\n", " ...\n", " \n", "---------- WebSearchAgent ----------\n", "Dwyane Wade was the Miami Heat player with the highest points in the 2006-2007 season, scoring 1,397 points. Now, let's find his total rebounds for the 2007-2008 and 2008-2009 NBA seasons.\n", "---------- WebSearchAgent ----------\n", "[FunctionCall(id='call_GL7KkWKj9ejIM8FfpgXe2dPk', arguments='{\"query\": \"Dwyane Wade total rebounds 2007-2008 season\"}', name='search_web_tool'), FunctionCall(id='call_X81huZoiA30zIjSAIDgb8ebe', arguments='{\"query\": \"Dwyane Wade total rebounds 2008-2009 season\"}', name='search_web_tool')]\n", "---------- WebSearchAgent ----------\n", "[FunctionExecutionResult(content='The number of total rebounds for Dwayne Wade in the Miami Heat season 2007-2008 is 214.', name='search_web_tool', call_id='call_GL7KkWKj9ejIM8FfpgXe2dPk', is_error=False), FunctionExecutionResult(content='The number of total rebounds for Dwayne Wade in the Miami Heat season 2008-2009 is 398.', name='search_web_tool', call_id='call_X81huZoiA30zIjSAIDgb8ebe', is_error=False)]\n", "---------- WebSearchAgent ----------\n", "The number of total rebounds for Dwayne Wade in the Miami Heat season 2007-2008 is 214.\n", "The number of total rebounds for Dwayne Wade in the Miami Heat season 2008-2009 is 398.\n", "---------- DataAnalystAgent ----------\n", "[FunctionCall(id='call_kB50RkFVqHptA7FOf0lL2RS8', arguments='{\"start\":214,\"end\":398}', name='percentage_change_tool')]\n", "---------- DataAnalystAgent ----------\n", "[FunctionExecutionResult(content='85.98130841121495', name='percentage_change_tool', call_id='call_kB50RkFVqHptA7FOf0lL2RS8', is_error=False)]\n", "---------- DataAnalystAgent ----------\n", "85.98130841121495\n", "---------- PlanningAgent ----------\n", "The Miami Heat player with the highest points during the 2006-2007 NBA season was Dwayne Wade, who scored 1,397 points. The percentage increase in his total rebounds from the 2007-2008 season (214 rebounds) to the 2008-2009 season (398 rebounds) was approximately 86%.\n", "\n", "TERMINATE\n" ] } ], "source": [ "await main()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "然后您可以使用Jaeger UI查看从上述应用运行中收集的追踪数据。\n\n![Jaeger界面](jaeger.png)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 自定义追踪\n\n到目前为止，我们仅记录了由AutoGen运行时生成的默认事件（消息创建、发布等）。但您也可以创建自定义跨度来记录应用程序中的特定事件。\n\n在下面的示例中，我们将展示如何通过围绕团队添加自定义跨度来记录运行时事件，以及记录团队生成的消息跨度，从而记录来自`RoundRobinGroupChat`团队的消息。\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "-- primary_agent -- : Leaves cascade like gold, \n", "Whispering winds cool the earth.\n", "primary_agent: Leaves cascade like gold, \n", "Whispering winds cool the earth.\n", "\n", "-- critic_agent -- : Your haiku beautifully captures the essence of the fall season with vivid imagery. However, it appears to have six syllables in the second line, which should traditionally be five. Here's a revised version keeping the 5-7-5 syllable structure:\n", "\n", "Leaves cascade like gold, \n", "Whispering winds cool the air. \n", "\n", "Please adjust the second line to reflect a five-syllable count. Thank you!\n", "critic_agent: Your haiku beautifully captures the essence of the fall season with vivid imagery. However, it appears to have six syllables in the second line, which should traditionally be five. Here's a revised version keeping the 5-7-5 syllable structure:\n", "\n", "Leaves cascade like gold, \n", "Whispering winds cool the air. \n", "\n", "Please adjust the second line to reflect a five-syllable count. Thank you!\n", "\n", "-- primary_agent -- : Leaves cascade like gold, \n", "Whispering winds cool the air.\n", "primary_agent: Leaves cascade like gold, \n", "Whispering winds cool the air.\n", "\n", "-- critic_agent -- : APPROVE\n", "critic_agent: APPROVE\n" ] } ], "source": [ "from autogen_agentchat.base import TaskResult\n", "from autogen_agentchat.conditions import ExternalTermination\n", "from autogen_agentchat.teams import RoundRobinGroupChat\n", "from autogen_core import CancellationToken\n", "\n", "\n", "async def run_agents() -> None:\n", " # 创建一个OpenAI模型客户端。\n", " model_client = OpenAIChatCompletionClient(model=\"gpt-4o-2024-08-06\")\n", "\n", " # 创建主代理。\n", " primary_agent = AssistantAgent(\n", " \"primary_agent\",\n", " model_client=model_client,\n", " system_message=\"You are a helpful AI assistant.\",\n", " )\n", "\n", " # 创建评论代理。\n", " critic_agent = AssistantAgent(\n", " \"critic_agent\",\n", " model_client=model_client,\n", " system_message=\"Provide constructive feedback. Respond with 'APPROVE' to when your feedbacks are addressed.\",\n", " )\n", "\n", " # 定义一个终止条件，当评论者批准时停止任务。\n", " text_termination = TextMentionTermination(\"APPROVE\")\n", "\n", " tracer = trace.get_tracer(\"autogen-test-agentchat\")\n", " with tracer.start_as_current_span(\"runtime_round_robin_events\"):\n", " team = RoundRobinGroupChat([primary_agent, critic_agent], termination_condition=text_termination)\n", "\n", " response_stream = team.run_stream(task=\"Write a 2 line haiku about the fall season\")\n", " async for response in response_stream:\n", " async for response in response_stream:\n", " if not isinstance(response, TaskResult):\n", " print(f\"\\n-- {response.source} -- : {response.to_text()}\")\n", " with tracer.start_as_current_span(f\"agent_message.{response.source}\") as message_span:\n", " message_span.set_attribute(\"agent.name\", response.source)\n", " message_span.set_attribute(\"message.content\", response.to_text())\n", " print(f\"{response.source}: {response.to_text()}\")\n", "\n", " await model_client.close()\n", "\n", "\n", "await run_agents()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "在上面的代码中，我们为代理发送的每条消息创建了一个新的跨度。我们在跨度上设置了属性，包括代理名称和消息内容。这使我们能够追踪消息在应用程序中的流动，并理解它们是如何被处理的。\n" ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 2 }