{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# 模型客户端\n\nAutoGen 提供了一套内置模型客户端用于调用 ChatCompletion API。\n所有模型客户端都实现了 {py:class}`~autogen_core.models.ChatCompletionClient` 协议类。\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "目前我们支持以下内置模型客户端：\n* {py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient`：用于 OpenAI 模型及兼容 OpenAI API 的模型（如 Gemini）。\n* {py:class}`~autogen_ext.models.openai.AzureOpenAIChatCompletionClient`：用于 Azure OpenAI 模型。\n* {py:class}`~autogen_ext.models.azure.AzureAIChatCompletionClient`：用于 GitHub 模型及托管在 Azure 上的模型。\n* {py:class}`~autogen_ext.models.ollama.OllamaChatCompletionClient`（实验性）：用于托管在 Ollama 上的本地模型。\n* {py:class}`~autogen_ext.models.anthropic.AnthropicChatCompletionClient`（实验性）：用于托管在 Anthropic 上的模型。\n* {py:class}`~autogen_ext.models.semantic_kernel.SKChatCompletionAdapter`：用于 Semantic Kernel AI 连接器的适配器。\n\n有关如何使用这些模型客户端的更多信息，请参阅每个客户端的文档。\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 记录模型调用\n\nAutoGen 使用标准 Python 日志模块来记录模型调用和响应等事件。\n日志记录器名称为 {py:attr}`autogen_core.EVENT_LOGGER_NAME`，事件类型为 `LLMCall`。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import logging\n",
        "\n",
        "from autogen_core import EVENT_LOGGER_NAME\n",
        "\n",
        "logging.basicConfig(level=logging.WARNING)\n",
        "logger = logging.getLogger(EVENT_LOGGER_NAME)\n",
        "logger.addHandler(logging.StreamHandler())\n",
        "logger.setLevel(logging.INFO)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 调用模型客户端\n\n要调用模型客户端，可以使用 {py:meth}`~autogen_core.models.ChatCompletionClient.create` 方法。\n此示例使用 {py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient` 来调用 OpenAI 模型。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "finish_reason='stop' content='The capital of France is Paris.' usage=RequestUsage(prompt_tokens=15, completion_tokens=8) cached=False logprobs=None thought=None\n"
          ]
        }
      ],
      "source": [
        "from autogen_core.models import UserMessage\n",
        "from autogen_ext.models.openai import OpenAIChatCompletionClient\n",
        "\n",
        "model_client = OpenAIChatCompletionClient(\n",
        "    model=\"gpt-4\", temperature=0.3\n",
        ")  # 假设环境变量中已设置 OPENAI_API_KEY。\n",
        "\n",
        "result = await model_client.create([UserMessage(content=\"What is the capital of France?\", source=\"user\")])\n",
        "print(result)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 流式令牌\n\n你可以使用 {py:meth}`~autogen_core.models.ChatCompletionClient.create_stream` 方法来创建一个\n支持流式令牌分块的聊天补全请求。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Streamed responses:\n",
            "In the heart of an ancient forest, beneath the shadow of snow-capped peaks, a dragon named Elara lived secretly for centuries. Elara was unlike any dragon from the old tales; her scales shimmered with a deep emerald hue, each scale engraved with symbols of lost wisdom. The villagers in the nearby valley spoke of mysterious lights dancing across the night sky, but none dared venture close enough to solve the enigma.\n",
            "\n",
            "One cold winter's eve, a young girl named Lira, brimming with curiosity and armed with the innocence of youth, wandered into Elara’s domain. Instead of fire and fury, she found warmth and a gentle gaze. The dragon shared stories of a world long forgotten and in return, Lira gifted her simple stories of human life, rich in laughter and scent of earth.\n",
            "\n",
            "From that night on, the villagers noticed subtle changes—the crops grew taller, and the air seemed sweeter. Elara had infused the valley with ancient magic, a guardian of balance, watching quietly as her new friend thrived under the stars. And so, Lira and Elara’s bond marked the beginning of a timeless friendship that spun tales of hope whispered through the leaves of the ever-verdant forest.\n",
            "\n",
            "------------\n",
            "\n",
            "The complete response:\n",
            "In the heart of an ancient forest, beneath the shadow of snow-capped peaks, a dragon named Elara lived secretly for centuries. Elara was unlike any dragon from the old tales; her scales shimmered with a deep emerald hue, each scale engraved with symbols of lost wisdom. The villagers in the nearby valley spoke of mysterious lights dancing across the night sky, but none dared venture close enough to solve the enigma.\n",
            "\n",
            "One cold winter's eve, a young girl named Lira, brimming with curiosity and armed with the innocence of youth, wandered into Elara’s domain. Instead of fire and fury, she found warmth and a gentle gaze. The dragon shared stories of a world long forgotten and in return, Lira gifted her simple stories of human life, rich in laughter and scent of earth.\n",
            "\n",
            "From that night on, the villagers noticed subtle changes—the crops grew taller, and the air seemed sweeter. Elara had infused the valley with ancient magic, a guardian of balance, watching quietly as her new friend thrived under the stars. And so, Lira and Elara’s bond marked the beginning of a timeless friendship that spun tales of hope whispered through the leaves of the ever-verdant forest.\n",
            "\n",
            "\n",
            "------------\n",
            "\n",
            "The token usage was:\n",
            "RequestUsage(prompt_tokens=0, completion_tokens=0)\n"
          ]
        }
      ],
      "source": [
        "from autogen_core.models import CreateResult, UserMessage\n",
        "from autogen_ext.models.openai import OpenAIChatCompletionClient\n",
        "\n",
        "model_client = OpenAIChatCompletionClient(model=\"gpt-4o\")  # 假设环境变量中已设置 OPENAI_API_KEY。\n",
        "\n",
        "messages = [\n",
        "    UserMessage(content=\"Write a very short story about a dragon.\", source=\"user\"),\n",
        "]\n",
        "\n",
        "# 创建流式连接。\n",
        "stream = model_client.create_stream(messages=messages)\n",
        "\n",
        "# 遍历流并打印响应内容。\n",
        "print(\"Streamed responses:\")\n",
        "async for chunk in stream:  # type: ignore\n",
        "    if isinstance(chunk, str):\n",
        "        # 数据块是一个字符串。\n",
        "        print(chunk, flush=True, end=\"\")\n",
        "    else:\n",
        "        # 最终数据块是一个CreateResult对象。\n",
        "        assert isinstance(chunk, CreateResult) and isinstance(chunk.content, str)\n",
        "        # 最后一个响应是包含完整消息的CreateResult对象。\n",
        "        print(\"\\n\\n------------\\n\")\n",
        "        print(\"The complete response:\", flush=True)\n",
        "        print(chunk.content, flush=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "```{note}\n流式响应中的最后一个响应始终是{py:class}`~autogen_core.models.CreateResult`类型的最终响应。\n```\n\n```{note}\n默认的用量响应返回零值。要启用用量统计，\n请参阅{py:meth}`~autogen_ext.models.openai.BaseOpenAIChatCompletionClient.create_stream`\n获取更多详情。\n```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 结构化输出\n\n通过在{py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient`和{py:class}`~autogen_ext.models.openai.AzureOpenAIChatCompletionClient`中\n将`response_format`字段设置为[Pydantic BaseModel](https://docs.pydantic.dev/latest/concepts/models/)类，可以启用结构化输出。\n\n```{note}\n结构化输出仅适用于支持该功能的模型。同时\n需要模型客户端也支持结构化输出。\n目前，{py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient`\n和{py:class}`~autogen_ext.models.openai.AzureOpenAIChatCompletionClient`\n支持结构化输出。\n```\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "I'm glad to hear that you're feeling happy! It's such a great emotion that can brighten your whole day. Is there anything in particular that's bringing you joy today? 😊\n",
            "happy\n"
          ]
        }
      ],
      "source": [
        "from typing import Literal\n",
        "\n",
        "from pydantic import BaseModel\n",
        "\n",
        "\n",
        "# 代理的响应格式作为Pydantic基础模型。\n",
        "class AgentResponse(BaseModel):\n",
        "    thoughts: str\n",
        "    response: Literal[\"happy\", \"sad\", \"neutral\"]\n",
        "\n",
        "\n",
        "# 创建一个使用OpenAI GPT-4o模型并带有自定义响应格式的代理。\n",
        "model_client = OpenAIChatCompletionClient(\n",
        "    model=\"gpt-4o\",\n",
        "    response_format=AgentResponse,  # type: ignore\n",
        ")\n",
        "\n",
        "# 向模型发送消息列表并等待响应。\n",
        "messages = [\n",
        "    UserMessage(content=\"I am happy.\", source=\"user\"),\n",
        "]\n",
        "response = await model_client.create(messages=messages)\n",
        "assert isinstance(response.content, str)\n",
        "parsed_response = AgentResponse.model_validate_json(response.content)\n",
        "print(parsed_response.thoughts)\n",
        "print(parsed_response.response)\n",
        "\n",
        "# 关闭与模型客户端的连接。\n",
        "await model_client.close()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "你还可以在 {py:meth}`~autogen_ext.models.openai.BaseOpenAIChatCompletionClient.create` 方法中使用 `extra_create_args` 参数\n来设置 `response_format` 字段，从而为每个请求配置结构化输出。\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 缓存模型响应\n\n`autogen_ext` 实现了 {py:class}`~autogen_ext.models.cache.ChatCompletionCache`，它可以包装任何 {py:class}`~autogen_core.models.ChatCompletionClient`。使用这个包装器可以避免在多次使用相同提示查询底层客户端时产生令牌消耗。\n\n{py:class}`~autogen_core.models.ChatCompletionCache` 使用 {py:class}`~autogen_core.CacheStore` 协议。我们已经实现了一些有用的 {py:class}`~autogen_core.CacheStore` 变体，包括 {py:class}`~autogen_ext.cache_store.diskcache.DiskCacheStore` 和 {py:class}`~autogen_ext.cache_store.redis.RedisStore`。\n\n以下是使用 `diskcache` 进行本地缓存的示例：\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "vscode": {
          "languageId": "shellscript"
        }
      },
      "outputs": [],
      "source": [
        "# pip install -U \"autogen-ext[openai, diskcache]\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "True\n"
          ]
        }
      ],
      "source": [
        "import asyncio\n",
        "import tempfile\n",
        "\n",
        "from autogen_core.models import UserMessage\n",
        "from autogen_ext.cache_store.diskcache import DiskCacheStore\n",
        "from autogen_ext.models.cache import CHAT_CACHE_VALUE_TYPE, ChatCompletionCache\n",
        "from autogen_ext.models.openai import OpenAIChatCompletionClient\n",
        "from diskcache import Cache\n",
        "\n",
        "\n",
        "async def main() -> None:\n",
        "    with tempfile.TemporaryDirectory() as tmpdirname:\n",
        "        # 初始化原始客户端\n",
        "        openai_model_client = OpenAIChatCompletionClient(model=\"gpt-4o\")\n",
        "\n",
        "        # 然后初始化 CacheStore，这里使用 diskcache.Cache\n",
        "        # 。你也可以像这样使用 redis：from autogen_ext.cache_store.redis\n",
        "        # import RedisStore import redis redis_instance\n",
        "        # = redis.Redis() cache_store = RedisCacheStore[CHAT_CACHE_VALUE_TYPE](redis_instance)\n",
        "        # \n",
        "        # \n",
        "        cache_store = DiskCacheStore[CHAT_CACHE_VALUE_TYPE](Cache(tmpdirname))\n",
        "        cache_client = ChatCompletionCache(openai_model_client, cache_store)\n",
        "\n",
        "        response = await cache_client.create([UserMessage(content=\"Hello, how are you?\", source=\"user\")])\n",
        "        print(response)  # 应打印来自OpenAI的响应\n",
        "        response = await cache_client.create([UserMessage(content=\"Hello, how are you?\", source=\"user\")])\n",
        "        print(response)  # 应打印缓存的响应\n",
        "\n",
        "        await openai_model_client.close()\n",
        "        await cache_client.close()\n",
        "\n",
        "\n",
        "asyncio.run(main())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "检查`cached_client.total_usage()`（或`model_client.total_usage()`）在缓存响应前后的值应该得到相同的计数。\n\n注意缓存机制对提供给`cached_client.create`或`cached_client.create_stream`的精确参数非常敏感，因此更改`tools`或`json_output`参数可能导致缓存未命中。\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 使用模型客户端构建代理\n\n让我们创建一个简单的AI代理，该代理可以使用ChatCompletion API响应消息。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from dataclasses import dataclass\n",
        "\n",
        "from autogen_core import MessageContext, RoutedAgent, SingleThreadedAgentRuntime, message_handler\n",
        "from autogen_core.models import ChatCompletionClient, SystemMessage, UserMessage\n",
        "from autogen_ext.models.openai import OpenAIChatCompletionClient\n",
        "\n",
        "\n",
        "@dataclass\n",
        "class Message:\n",
        "    content: str\n",
        "\n",
        "\n",
        "class SimpleAgent(RoutedAgent):\n",
        "    def __init__(self, model_client: ChatCompletionClient) -> None:\n",
        "        super().__init__(\"A simple agent\")\n",
        "        self._system_messages = [SystemMessage(content=\"You are a helpful AI assistant.\")]\n",
        "        self._model_client = model_client\n",
        "\n",
        "    @message_handler\n",
        "    async def handle_user_message(self, message: Message, ctx: MessageContext) -> Message:\n",
        "        # 准备聊天补全模型的输入\n",
        "        user_message = UserMessage(content=message.content, source=\"user\")\n",
        "        response = await self._model_client.create(\n",
        "            self._system_messages + [user_message], cancellation_token=ctx.cancellation_token\n",
        "        )\n",
        "        # 返回模型的响应。\n",
        "        assert isinstance(response.content, str)\n",
        "        return Message(content=response.content)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "`SimpleAgent` 类是 `autogen_core.RoutedAgent` 的子类，\n用于方便地自动将消息路由到相应的处理程序。\n它有一个处理程序 `handle_user_message`，用于处理来自用户的消息。它使用 `ChatCompletionClient` 生成对消息的响应。\n然后按照直接通信模型将响应返回给用户。\n\n```{note}\n类型为 `autogen_core.CancellationToken` 的 `cancellation_token` 用于取消\n异步操作。它与消息处理程序内部的异步调用相关联，\n调用方可以使用它来取消处理程序。\n```\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Seattle is a vibrant city with a wide range of activities and attractions. Here are some fun things to do in Seattle:\n",
            "\n",
            "1. **Space Needle**: Visit this iconic observation tower for stunning views of the city and surrounding mountains.\n",
            "\n",
            "2. **Pike Place Market**: Explore this historic market where you can see the famous fish toss, buy local produce, and find unique crafts and eateries.\n",
            "\n",
            "3. **Museum of Pop Culture (MoPOP)**: Dive into the world of contemporary culture, music, and science fiction at this interactive museum.\n",
            "\n",
            "4. **Chihuly Garden and Glass**: Marvel at the beautiful glass art installations by artist Dale Chihuly, located right next to the Space Needle.\n",
            "\n",
            "5. **Seattle Aquarium**: Discover the diverse marine life of the Pacific Northwest at this engaging aquarium.\n",
            "\n",
            "6. **Seattle Art Museum**: Explore a vast collection of art from around the world, including contemporary and indigenous art.\n",
            "\n",
            "7. **Kerry Park**: For one of the best views of the Seattle skyline, head to this small park on Queen Anne Hill.\n",
            "\n",
            "8. **Ballard Locks**: Watch boats pass through the locks and observe the salmon ladder to see salmon migrating.\n",
            "\n",
            "9. **Ferry to Bainbridge Island**: Take a scenic ferry ride across Puget Sound to enjoy charming shops, restaurants, and beautiful natural scenery.\n",
            "\n",
            "10. **Olympic Sculpture Park**: Stroll through this outdoor park with large-scale sculptures and stunning views of the waterfront and mountains.\n",
            "\n",
            "11. **Underground Tour**: Discover Seattle's history on this quirky tour of the city's underground passageways in Pioneer Square.\n",
            "\n",
            "12. **Seattle Waterfront**: Enjoy the shops, restaurants, and attractions along the waterfront, including the Seattle Great Wheel and the aquarium.\n",
            "\n",
            "13. **Discovery Park**: Explore the largest green space in Seattle, featuring trails, beaches, and views of Puget Sound.\n",
            "\n",
            "14. **Food Tours**: Try out Seattle’s diverse culinary scene, including fresh seafood, international cuisines, and coffee culture (don’t miss the original Starbucks!).\n",
            "\n",
            "15. **Attend a Sports Game**: Catch a Seahawks (NFL), Mariners (MLB), or Sounders (MLS) game for a lively local experience.\n",
            "\n",
            "Whether you're interested in culture, nature, food, or history, Seattle has something for everyone to enjoy!\n"
          ]
        }
      ],
      "source": [
        "# 创建运行时并注册代理。\n",
        "from autogen_core import AgentId\n",
        "\n",
        "model_client = OpenAIChatCompletionClient(\n",
        "    model=\"gpt-4o-mini\",\n",
        "    # api_key=\"sk-...\", # 可选，如果你在环境变量中设置了 OPENAI_API_KEY\n",
        ")\n",
        "\n",
        "runtime = SingleThreadedAgentRuntime()\n",
        "await SimpleAgent.register(\n",
        "    runtime,\n",
        "    \"simple_agent\",\n",
        "    lambda: SimpleAgent(model_client=model_client),\n",
        ")\n",
        "# 启动运行时处理消息。\n",
        "runtime.start()\n",
        "# 向代理发送消息并获取响应。\n",
        "message = Message(\"Hello, what are some fun things to do in Seattle?\")\n",
        "response = await runtime.send_message(message, AgentId(\"simple_agent\", \"default\"))\n",
        "print(response.content)\n",
        "# 停止运行时处理消息。\n",
        "await runtime.stop()\n",
        "await model_client.close()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "上述 `SimpleAgent` 总是返回一个仅包含系统消息和最新用户消息的全新上下文。\n我们可以使用来自 {py:mod}`autogen_core.model_context` 的模型上下文类\n来让代理\"记住\"之前的对话。\n更多详情请参阅[模型上下文](./model-context.ipynb)页面。\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 从环境变量获取API密钥\n\n在上面的示例中，我们展示了可以通过 `api_key` 参数提供API密钥。重要的是，OpenAI和Azure OpenAI客户端使用了[openai包](https://github.com/openai/openai-python/blob/3f8d8205ae41c389541e125336b0ae0c5e437661/src/openai/__init__.py#L260)，如果没有提供密钥，它会自动从环境变量中读取API密钥。\n\n- 对于OpenAI，可以设置 `OPENAI_API_KEY` 环境变量。\n- 对于Azure OpenAI，可以设置 `AZURE_OPENAI_API_KEY` 环境变量。\n\n此外，对于Gemini(Beta)，可以设置 `GEMINI_API_KEY` 环境变量。\n\n这是一个值得采用的良好实践，因为它可以避免在代码中包含敏感的API密钥。\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": ".venv",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.12.7"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}