跳到主要内容

ai-request-rewrite

ai-request-rewrite 插件处理客户端请求的方式是：在将请求中继到上游服务之前，先将其转发给 LLM 服务进行转换。这使得能够进行 LLM 驱动的修改，例如数据编辑、内容丰富或重新格式化。该插件支持与 OpenAI、DeepSeek、Gemini、Vertex AI、Anthropic、OpenRouter 以及其他 OpenAI 兼容的 API 集成。

示例​

以下示例展示了如何在不同场景下配置 ai-request-rewrite。

这些示例将使用 OpenAI 作为 LLM 服务。要继续操作，请获取 OpenAI API Key 并将其保存到环境变量中：

export OPENAI_API_KEY=YOUR_OPENAI_API_KEY   # 替换为你的 API Key

编辑敏感信息​

以下示例展示了如何使用 ai-request-rewrite 插件在请求到达上游服务之前编辑敏感信息。

创建一个路由并按如下方式配置 ai-request-rewrite 插件：

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
  -H "X-API-KEY: ${ADMIN_API_KEY}" \
  -d '{
    "id": "ai-request-rewrite-route",
    "uri": "/anything",
    "methods": ["POST"],
    "plugins": {
      "ai-request-rewrite": {
        "provider": "openai",
        "auth": {
          "header": {
            "Authorization": "Bearer '"$OPENAI_API_KEY"'"
          }
        },
        "options":{
          "model": "gpt-4"
        },
        "prompt": "Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\''s license numbers). Replace detected sensitive values with a masked format (e.g., \"*** **** **** 1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
      }
    },
    "upstream": {
      "type": "roundrobin",
      "nodes": {
        "httpbin.org:80": 1
      }
    }
  }'

❶ 指定提供商为 openai。

❷ 在 Authorization 请求头中附带 OpenAI API Key。

❸ 指定模型名称。

❹ 指定在请求到达上游服务之前要编辑的信息。

发送一个包含一些个人身份信息的 POST 请求到该路由：

curl "http://127.0.0.1:9080/anything" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "content": "John said his debit card number is 4111 1111 1111 1111 and SIN is 123-45-6789."
  }'

你应该会收到类似以下的响应：

{
  "args": {},
  "data": "{\n    \"content\": \"John said his debit card number is **** **** **** 1111 and SIN is ***-**-****.\"\n  }"
  ...,
  "json": {
    "messages": [
      {
        "content": "Client information from customer service calls",
        "role": "system"
      }, 
      {
        "content": "John said his debit card number is **** **** **** 1111 and SIN is ***-**-****."
        "role": "user"
      }
    ], 
    "model": "openai"
  }, 
  "method": "POST", 
  "origin": "192.168.97.1, 103.97.2.170", 
  "url": "http://127.0.0.1/anything"
}

重新格式化数据​

以下示例展示了如何使用 ai-request-rewrite 插件在请求到达上游服务之前重新格式化数据。

创建一个路由并按如下方式配置 ai-request-rewrite 插件：

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
  -H "X-API-KEY: ${ADMIN_API_KEY}" \
  -d '{
    "id": "ai-request-rewrite-route",
    "uri": "/anything",
    "methods": ["POST"],
    "plugins": {
      "ai-request-rewrite": {
        "provider": "openai",
        "auth": {
          "header": {
            "Authorization": "Bearer '"$OPENAI_API_KEY"'"
          }
        },
        "options":{
          "model": "gpt-4"
        },
        "prompt": "Convert natural language queries into structured JSON format with intent and extracted parameters."
      }
    },
    "upstream": {
      "type": "roundrobin",
      "nodes": {
        "httpbin.org:80": 1
      }
    }
  }'

❶ 指定提供商为 openai。

❷ 在 Authorization 请求头中附带 OpenAI API Key。

❸ 指定模型名称。

❹ 指定在请求到达上游服务之前如何重新格式化。

发送一个包含一些个人身份信息的 POST 请求到该路由：

curl "http://127.0.0.1:9080/anything" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Book a flight from NYC to LA on April 10, 2022."
  }'

你应该会收到类似以下的响应：

{
  "args": {},
  "data": "{\n  \"intent\": \"BookFlight\",\n  \"parameters\": {\n    \"origin\": \"NYC\",\n    \"destination\": \"LA\",\n    \"date\": \"2022-04-10\"\n  }\n}", 
  ...,
  "json": {
    "intent": "BookFlight", 
    "parameters": {
      "date": "2022-04-10", 
      "destination": "LA", 
      "origin": "NYC"
    }
  },
  "method": "POST", 
  "origin": "192.168.97.1, 103.97.2.167", 
  "url": "http://127.0.0.1/anything"
}

总结信息​

以下示例展示了如何使用 ai-request-rewrite 插件在请求到达上游服务之前总结信息。

创建一个路由并按如下方式配置 ai-request-rewrite 插件：

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
  -H "X-API-KEY: ${ADMIN_API_KEY}" \
  -d '{
    "id": "ai-request-rewrite-route",
    "uri": "/anything",
    "methods": ["POST"],
    "plugins": {
      "ai-request-rewrite": {
        "provider": "openai",
        "auth": {
          "header": {
            "Authorization": "Bearer '"$OPENAI_API_KEY"'"
          }
        },
        "options":{
          "model": "gpt-4"
        },
        "prompt": "Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative."
      }
    },
    "upstream": {
      "type": "roundrobin",
      "nodes": {
        "httpbin.org:80": 1
      }
    }
  }'

❶ 指定提供商为 openai。

❷ 在 Authorization 请求头中附带 OpenAI API Key。

❸ 指定模型名称。

❹ 指定在请求到达上游服务之前总结信息的要求。

发送一个包含一些个人身份信息的 POST 请求到该路由：

curl "http://127.0.0.1:9080/anything" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Hey! So, I’m planning a trip to Japan next spring for about three weeks, and I want to visit Tokyo, Kyoto, and Osaka, but I’m not sure how to split my time between them. I really love history and cultural sites, so temples and shrines are a must. I’m also a big foodie, especially into ramen and sushi, so I’d love recommendations on the best spots. I prefer quieter areas for accommodation, but I don’t mind traveling into busy areas for sightseeing. Oh, and I’d also like to do a day trip somewhere outside these cities—maybe Hakone or Nara? I heard the cherry blossoms might still be in bloom in early April, so I’d love to catch them if possible. Also, what’s the best way to get around—should I get a JR Pass, or would individual tickets be better? Thanks!"
  }'

你应该会收到类似以下的响应：

{
  "args": {},
  "data": "The individual is planning a three-week trip to Japan in the spring, looking to visit Tokyo, Kyoto, and Osaka. They are interested in history, culture, temples, and shrines. They love ramen and sushi, so are seeking food recommendations. Accommodation should be in quieter areas, but they are open to busy sites for sightseeing. Along with these cities, they plan to make a day trip to either Hakone or Nara, hoping to see the cherry blossoms in early April. The best transport method between buying the JR Pass or individual tickets is also a query.", 
  ...,
  "method": "POST", 
  "origin": "192.168.97.1, 103.97.2.171", 
  "url": "http://127.0.0.1/anything"
}