ua-restriction
ua-restriction 插件支持通过配置 User-Agent 的白名单或黑名单来限制对上游资源的访问。常见的用例是防止网络爬虫过载上游资源并导致服务降级。
示例
下面的示例展示了如何在不同场景下配置 ua-restriction 插件。
拒绝网络爬虫并自定义错误消息
以下示例展示了如何配置插件来抵御不需要的网络爬虫并自定义拒绝消息。
- Admin API
- ADC
- Ingress Controller
创建一个路由如下：
curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
-H "X-API-KEY: ${ADMIN_API_KEY}" \
-d '{
"id": "ua-restriction-route",
"uri": "/anything",
"plugins": {
"ua-restriction": {
"bypass_missing": false,
"denylist": [
"(Baiduspider)/(\\d+)\\.(\\d+)",
"bad-bot-1"
],
"message": "Access denied"
}
},
"upstream": {
"type": "roundrobin",
"nodes": {
"httpbin.org:80": 1
}
}
}'
adc.yaml
services:
- name: ua-restriction-service
routes:
- name: ua-restriction-route
uris:
- /anything
plugins:
ua-restriction:
bypass_missing: false
denylist:
- "(Baiduspider)/(\\d+)\\.(\\d+)"
- "bad-bot-1"
message: "Access denied"
upstream:
type: roundrobin
nodes:
- host: httpbin.org
port: 80
weight: 1
将配置同步到网关：
adc sync -f adc.yaml
- Gateway API
- APISIX CRD
ua-restriction-ic.yaml
apiVersion: v1
kind: Service
metadata:
namespace: aic
name: httpbin-external-domain
spec:
type: ExternalName
externalName: httpbin.org
---
apiVersion: apisix.apache.org/v1alpha1
kind: PluginConfig
metadata:
namespace: aic
name: ua-restriction-plugin-config
spec:
plugins:
- name: ua-restriction
config:
bypass_missing: false
denylist:
- "(Baiduspider)/(\\d+)\\.(\\d+)"
- "bad-bot-1"
message: "Access denied"
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
namespace: aic
name: ua-restriction-route
spec:
parentRefs:
- name: apisix
rules:
- matches:
- path:
type: Exact
value: /anything
filters:
- type: ExtensionRef
extensionRef:
group: apisix.apache.org
kind: PluginConfig
name: ua-restriction-plugin-config
backendRefs:
- name: httpbin-external-domain
port: 80
将配置应用到集群：
kubectl apply -f ua-restriction-ic.yaml
ua-restriction-ic.yaml
apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
namespace: aic
name: httpbin-external-domain
spec:
ingressClassName: apisix
externalNodes:
- type: Domain
name: httpbin.org
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
namespace: aic
name: ua-restriction-route
spec:
ingressClassName: apisix
http:
- name: ua-restriction-route
match:
paths:
- /anything
upstreams:
- name: httpbin-external-domain
plugins:
- name: ua-restriction
enable: true
config:
bypass_missing: false
denylist:
- "(Baiduspider)/(\\d+)\\.(\\d+)"
- "bad-bot-1"
message: "Access denied"
将配置应用到集群：
kubectl apply -f ua-restriction-ic.yaml
❶ 不允许绕过 UA 限制规则。
❷ 配置不应访问上游资源的 User-Agent。
❸ 自定义拒绝访问时的错误消息。
发送请求到该路由：
curl -i "http://127.0.0.1:9080/anything"
你应该收到 HTTP/1.1 200 OK 响应。
使用被禁止的 User-Agent 发送另一个请求到该路由：
curl -i "http://127.0.0.1:9080/anything" -H 'User-Agent: Baiduspider/5.0'
你应该收到 HTTP/1.1 403 Forbidden 响 应以及以下消息：
{"message":"Access denied"}
绕过 UA 限制检查
以下示例展示了如何配置插件以允许特定 User-Agent 的请求绕过 UA 限制。
- Admin API
- ADC
- Ingress Controller
创建一个路由如下：
curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
-H "X-API-KEY: ${ADMIN_API_KEY}" \
-d '{
"id": "ua-restriction-route",
"uri": "/anything",
"plugins": {
"ua-restriction": {
"bypass_missing": true,
"allowlist": [
"good-bot-1"
],
"message": "Access denied"
}
},
"upstream": {
"type": "roundrobin",
"nodes": {
"httpbin.org:80": 1
}
}
}'
adc.yaml
services:
- name: ua-restriction-service
routes:
- name: ua-restriction-route
uris:
- /anything
plugins:
ua-restriction:
bypass_missing: true
allowlist:
- "good-bot-1"
message: "Access denied"
upstream:
type: roundrobin
nodes:
- host: httpbin.org
port: 80
weight: 1
将配置同步到网关：
adc sync -f adc.yaml
- Gateway API
- APISIX CRD
ua-restriction-ic.yaml
apiVersion: v1
kind: Service
metadata:
namespace: aic
name: httpbin-external-domain
spec:
type: ExternalName
externalName: httpbin.org
---
apiVersion: apisix.apache.org/v1alpha1
kind: PluginConfig
metadata:
namespace: aic
name: ua-restriction-allowlist-plugin-config
spec:
plugins:
- name: ua-restriction
config:
bypass_missing: true
allowlist:
- "good-bot-1"
message: "Access denied"
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
namespace: aic
name: ua-restriction-route
spec:
parentRefs:
- name: apisix
rules:
- matches:
- path:
type: Exact
value: /anything
filters:
- type: ExtensionRef
extensionRef:
group: apisix.apache.org
kind: PluginConfig
name: ua-restriction-allowlist-plugin-config
backendRefs:
- name: httpbin-external-domain
port: 80
将配置应用到集群：
kubectl apply -f ua-restriction-ic.yaml
ua-restriction-ic.yaml
apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
namespace: aic
name: httpbin-external-domain
spec:
ingressClassName: apisix
externalNodes:
- type: Domain
name: httpbin.org
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
namespace: aic
name: ua-restriction-route
spec:
ingressClassName: apisix
http:
- name: ua-restriction-route
match:
paths:
- /anything
upstreams:
- name: httpbin-external-domain
plugins:
- name: ua-restriction
enable: true
config:
bypass_missing: true
allowlist:
- "good-bot-1"
message: "Access denied"
将配置应用到集群：
kubectl apply -f ua-restriction-ic.yaml
❶ 允许绕过 UA 限制规则。
❷ 配置应允许访问上游资源的 User-Agent。
发送未修改 User-Agent 的请求到该路由：
curl -i "http://127.0.0.1:9080/anything"
你应该收到 HTTP/1.1 403 Forbidden 响应以及以下消息：
{"message":"Access denied"}
发送另一个使用空 User-Agent 的请求到该路由：
curl -i "http://127.0.0.1:9080/anything" -H 'User-Agent: '
你应该收到 HTTP/1.1 200 OK 响应。