配置故障注入
混沌工程对于构建可靠系统至关重要。fault-injection插件可以在请求到达上游服务 之前,在网关层模拟故障和延迟,从而测试应用的韧性。
前置条件
- API7 企业版实例正在运行。
- 已创建网关组且网关实例正在运行。
- 一个 Dashboard Token。
配置 HTTP 中止
可以对一定比例的请求返回指定 HTTP 状态码,以模拟服务故障。
- Admin API
- ADC
curl -k "https://localhost:7443/apisix/admin/services/fault-injection-service?gateway_group_id={gateway_group_id}" -X PUT \
-H "X-API-KEY: ${API_KEY}" \
-H "Content-Type: application/json" \
-d '{
"name": "fault-injection-service",
"upstream": {
"type": "roundrobin",
"scheme": "http",
"nodes": [
{
"host": "httpbin.org",
"port": 80,
"weight": 100
}
]
}
}'
curl -k "https://localhost:7443/apisix/admin/routes/fault-injection-route?gateway_group_id={gateway_group_id}" -X PUT \
-H "X-API-KEY: ${API_KEY}" \
-H "Content-Type: application/json" \
-d '{
"name": "fault-injection-route",
"paths": ["/get"],
"service_id": "fault-injection-service",
"plugins": {
"fault-injection": {
"abort": {
"http_status": 503,
"body": "Service Temporarily Unavailable",
"percentage": 10
}
}
}
}'
adc.yaml
services:
- name: fault-injection-service
upstream:
nodes:
- host: httpbin.org
port: 80
weight: 1
routes:
- name: fault-injection-route
uris:
- /get
plugins:
fault-injection:
abort:
http_status: 503
body: "Service Temporarily Unavailable"
percentage: 10
adc sync -f adc.yaml
配置响应延迟
还可以延迟响应,以模拟缓慢的网络状况或后端延迟。
- Admin API
- ADC
curl -k "https://localhost:7443/apisix/admin/services/delay-fault-injection-service?gateway_group_id={gateway_group_id}" -X PUT \
-H "X-API-KEY: ${API_KEY}" \
-H "Content-Type: application/json" \
-d '{
"name": "delay-fault-injection-service",
"upstream": {
"type": "roundrobin",
"scheme": "http",
"nodes": [
{
"host": "httpbin.org",
"port": 80,
"weight": 100
}
]
}
}'
curl -k "https://localhost:7443/apisix/admin/routes/delay-fault-injection-route?gateway_group_id={gateway_group_id}" -X PUT \
-H "X-API-KEY: ${API_KEY}" \
-H "Content-Type: application/json" \
-d '{
"name": "delay-fault-injection-route",
"paths": ["/delay/5"],
"service_id": "delay-fault-injection-service",
"plugins": {
"fault-injection": {
"delay": {
"duration": 5,
"percentage": 20
}
}
}
}'
adc.yaml
services:
- name: delay-fault-injection-service
upstream:
nodes:
- host: httpbin.org
port: 80
weight: 1
routes:
- name: delay-fault-injection-route
uris:
- /delay/5
plugins:
fault-injection:
delay:
duration: 5
percentage: 20
adc sync -f adc.yaml
验证配置
要验证注入比例为 10% 的中止配置,请发送多个请求:
for i in {1..20}; do curl -i "http://127.0.0.1:9080/get"; done
应有部分请求收到注入的 503 Service Temporarily Unavailable 响应。对于延迟,当故障被注入时,curl 命令应等待配置的 duration 时长。
要验证延迟示例,请测量对延迟路由发出的几个请求:
time curl -i "http://127.0.0.1:9080/delay/5"
如果应用配置后路由立即返回 404,请等待几秒,让最新配置下发到网关后重试。