data-mask
data-mask 插件在使用日志记录插件时,会屏蔽请求头、请求体和 URL 查询中的敏感信息。请注意,它不会修改实际的请求或响应流量。
要对网关访问日志中的敏感信息进行脱敏,请参阅配置数据脱敏。
该插件可以在路由、服务或全局插件上配置。但是,请注意全局插件始终在路由或服务级插件之前执行,因此数据屏蔽可能会在日志记录之后发生。
例如,如果全局配置了日志记录插件,而 data-mask 应用于路由级别,则请求将在屏蔽发生之前被记录,敏感数据将以明文形式出现。
为了确保预期的行为,建议在同一级别配置两个插件:
- 均在全局级别(如果适合你的用例,推荐)
- 均在路由或服务级别
示例
以下示例展示了如何在不同场景下使用 data-mask 插件。
虽然所有示例都使用 file-logger 插件进行日志记录,但该插件仅用于演示数据屏蔽的结果。请选择最适合你环境的日志记录插件。
屏蔽 URL 查询中的敏感信息
以下示例演示了如何在 file-logger 插件将请求记录到本地文件之前,屏蔽请求 URL 查询中的敏感信息。
创建一个带有 file-logger 插件(用于记录请求)和 data-mask 插件(带有三个数据屏蔽规则)的路由:
- Admin API
- ADC
- Ingress Controller
curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
-H "X-API-KEY: ${ADMIN_API_KEY}" \
-d '{
"id": "data-mask-route",
"uri": "/anything",
"plugins": {
"data-mask": {
"request": [
{
"action": "remove",
"name": "password",
"type": "query"
},
{
"action": "replace",
"name": "token",
"type": "query",
"value": "*****"
},
{
"action": "regex",
"name": "card",
"regex": "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)",
"type": "query",
"value": "$1-****-****-$2"
}
]
},
"file-logger": {
"path": "/tmp/mask-query.log"
}
},
"upstream": {
"type": "roundrobin",
"nodes": {
"httpbin.org:80": 1
}
}
}'
services:
- name: data-mask-service
routes:
- name: data-mask-route
uris:
- /anything
plugins:
data-mask:
request:
- action: remove
name: password
type: query
- action: replace
name: token
type: query
value: "*****"
- action: regex
name: card
regex: "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)"
type: query
value: "$1-****-****-$2"
file-logger:
path: /tmp/mask-query.log
upstream:
type: roundrobin
nodes:
- host: httpbin.org
port: 80
weight: 1
将配置同步到网关:
adc sync -f adc.yaml
- Gateway API
- APISIX CRD
apiVersion: v1
kind: Service
metadata:
namespace: aic
name: httpbin-external-domain
spec:
type: ExternalName
externalName: httpbin.org
---
apiVersion: apisix.apache.org/v1alpha1
kind: PluginConfig
metadata:
namespace: aic
name: data-mask-query-plugin-config
spec:
plugins:
- name: data-mask
config:
request:
- action: remove
name: password
type: query
- action: replace
name: token
type: query
value: "*****"
- action: regex
name: card
regex: "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)"
type: query
value: "$1-****-****-$2"
- name: file-logger
config:
path: /tmp/mask-query.log
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
namespace: aic
name: data-mask-route
spec:
parentRefs:
- name: apisix
rules:
- matches:
- path:
type: Exact
value: /anything
filters:
- type: ExtensionRef
extensionRef:
group: apisix.apache.org
kind: PluginConfig
name: data-mask-query-plugin-config
backendRefs:
- name: httpbin-external-domain
port: 80
将配置应用到集群:
kubectl apply -f data-mask-query-ic.yaml
apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
namespace: aic
name: httpbin-external-domain
spec:
ingressClassName: apisix
externalNodes:
- type: Domain
name: httpbin.org
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
namespace: aic
name: data-mask-route
spec:
ingressClassName: apisix
http:
- name: data-mask-route
match:
paths:
- /anything
upstreams:
- name: httpbin-external-domain
plugins:
- name: data-mask
enable: true
config:
request:
- action: remove
name: password
type: query
- action: replace
name: token
type: query
value: "*****"
- action: regex
name: card
regex: "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)"
type: query
value: "$1-****-****-$2"
- name: file-logger
enable: true
config:
path: /tmp/mask-query.log
将配置应用到集群:
kubectl apply -f data-mask-query-ic.yaml
❶ 从请求中删除 password URL 查询的数据屏蔽规则。
❷ 将 token URL 查询的值替换为 ***** 的数据屏蔽规则。
❸ 使用正则表达式匹配 URL 查询中的卡号并屏蔽卡号中间部分的数据屏蔽规则。
❹ 文件系统中保存日志的日志文件路径。
向路由发送带有 URL 查询中敏感信息的请求:
curl -i "http://127.0.0.1:9080/anything?password=abc&token=xyz&card=1234-1234-1234-1234"
你应该收到 HTTP/1.1 200 OK 响应。
导航到 /tmp/mask-query.log 文件并检查日志内容,你应该看到类似以下的日志条目:
{
"request": {
"uri": "/anything?token=*****&card=1234-****-****-1234",
"method": "GET",
"url": "http://127.0.0.1:9080/anything?token=*****&card=1234-****-****-1234",
"querystring": {
"token": "*****",
"card": "1234-****-****-1234"
}
}
}
屏蔽请求头中的敏感信息
以下示例演示了如何在 file-logger 插件将请求记录到本地文件之前,屏蔽请求头中的敏感信息。
创建一个带有 file-logger 插件(用于记录请求)和 data-mask 插件(带有三个数据屏蔽规则)的路由:
- Admin API
- ADC
- Ingress Controller
curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
-H "X-API-KEY: ${ADMIN_API_KEY}" \
-d '{
"id": "data-mask-route",
"uri": "/anything",
"plugins": {
"data-mask": {
"request": [
{
"action": "remove",
"name": "password",
"type": "header"
},
{
"action": "replace",
"name": "token",
"type": "header",
"value": "*****"
},
{
"action": "regex",
"name": "card",
"regex": "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)",
"type": "header",
"value": "$1-****-****-$2"
}
]
},
"file-logger": {
"path": "/tmp/mask-header.log"
}
},
"upstream": {
"type": "roundrobin",
"nodes": {
"httpbin.org:80": 1
}
}
}'
services:
- name: data-mask-service
routes:
- name: data-mask-route
uris:
- /anything
plugins:
data-mask:
request:
- action: remove
name: password
type: header
- action: replace
name: token
type: header
value: "*****"
- action: regex
name: card
regex: "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)"
type: header
value: "$1-****-****-$2"
file-logger:
path: /tmp/mask-header.log
upstream:
type: roundrobin
nodes:
- host: httpbin.org
port: 80
weight: 1
将配置同步到网关:
adc sync -f adc.yaml
- Gateway API
- APISIX CRD
apiVersion: v1
kind: Service
metadata:
namespace: aic
name: httpbin-external-domain
spec:
type: ExternalName
externalName: httpbin.org
---
apiVersion: apisix.apache.org/v1alpha1
kind: PluginConfig
metadata:
namespace: aic
name: data-mask-header-plugin-config
spec:
plugins:
- name: data-mask
config:
request:
- action: remove
name: password
type: header
- action: replace
name: token
type: header
value: "*****"
- action: regex
name: card
regex: "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)"
type: header
value: "$1-****-****-$2"
- name: file-logger
config:
path: /tmp/mask-header.log
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
namespace: aic
name: data-mask-route
spec:
parentRefs:
- name: apisix
rules:
- matches:
- path:
type: Exact
value: /anything
filters:
- type: ExtensionRef
extensionRef:
group: apisix.apache.org
kind: PluginConfig
name: data-mask-header-plugin-config
backendRefs:
- name: httpbin-external-domain
port: 80
将配置应用到集群:
kubectl apply -f data-mask-header-ic.yaml
apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
namespace: aic
name: httpbin-external-domain
spec:
ingressClassName: apisix
externalNodes:
- type: Domain
name: httpbin.org
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
namespace: aic
name: data-mask-route
spec:
ingressClassName: apisix
http:
- name: data-mask-route
match:
paths:
- /anything
upstreams:
- name: httpbin-external-domain
plugins:
- name: data-mask
enable: true
config:
request:
- action: remove
name: password
type: header
- action: replace
name: token
type: header
value: "*****"
- action: regex
name: card
regex: "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)"
type: header
value: "$1-****-****-$2"
- name: file-logger
enable: true
config:
path: /tmp/mask-header.log
将配置应用到集群:
kubectl apply -f data-mask-header-ic.yaml
❶ 从请求中删除 password 头的数据屏蔽规则。
❷ 将 token 请求头的值替换为 ***** 的数据屏蔽规则。
❸ 使用正则表达式匹配请求头中的卡号并屏蔽卡号中间部分的数据屏蔽规则。
❹ 文件系统中保存日志的日志文件路径。
向路由发送带有头中敏感信息的 POST 请求:
curl -i "http://127.0.0.1:9080/anything" -X POST \
-H "password: abc" \
-H "token: xyz" \
-H "card: 1234-1234-1234-1234"
你应该收到 HTTP/1.1 200 OK 响应。
导航到 /tmp/mask-header.log 文件并检查日志内容,你应该看到类似以下的日志条目:
{
"request": {
"uri": "/anything",
"method": "GET",
"url": "http://127.0.0.1:9080/anything",
"headers": {
"user-agent": "curl/8.6.0",
"token": "*****",
"card": "1234-****-****-1234"
}
}
}
屏蔽 URL 编码请求体中的敏感信息
以下示例演示了如何在 file-logger 插件将请求记录到本地文件之前,屏蔽 URL 编码请求体中的敏感信息。
创建一个带有 file-logger 插件(用于记录请求)和 data-mask 插件(带有三个数据屏蔽规则)的路由:
- Admin API
- ADC
- Ingress Controller
curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
-H "X-API-KEY: ${ADMIN_API_KEY}" \
-d '{
"id": "data-mask-route",
"uri": "/anything",
"plugins": {
"data-mask": {
"request": [
{
"action": "remove",
"body_format": "urlencoded",
"name": "password",
"type": "body"
},
{
"action": "replace",
"body_format": "urlencoded",
"name": "token",
"type": "body",
"value": "*****"
},
{
"action": "regex",
"body_format": "urlencoded",
"name": "card",
"regex": "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)",
"type": "body",
"value": "$1-****-****-$2"
}
]
},
"file-logger": {
"include_req_body": true,
"path": "/tmp/mask-urlencoded-body.log"
}
},
"upstream": {
"type": "roundrobin",
"nodes": {
"httpbin.org:80": 1
}
}
}'
services:
- name: data-mask-service
routes:
- name: data-mask-route
uris:
- /anything
plugins:
data-mask:
request:
- action: remove
body_format: urlencoded
name: password
type: body
- action: replace
body_format: urlencoded
name: token
type: body
value: "*****"
- action: regex
body_format: urlencoded
name: card
regex: "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)"
type: body
value: "$1-****-****-$2"
file-logger:
include_req_body: true
path: /tmp/mask-urlencoded-body.log
upstream:
type: roundrobin
nodes:
- host: httpbin.org
port: 80
weight: 1
将配置同步到网关:
adc sync -f adc.yaml
- Gateway API
- APISIX CRD
apiVersion: v1
kind: Service
metadata:
namespace: aic
name: httpbin-external-domain
spec:
type: ExternalName
externalName: httpbin.org
---
apiVersion: apisix.apache.org/v1alpha1
kind: PluginConfig
metadata:
namespace: aic
name: data-mask-urlencoded-plugin-config
spec:
plugins:
- name: data-mask
config:
request:
- action: remove
body_format: urlencoded
name: password
type: body
- action: replace
body_format: urlencoded
name: token
type: body
value: "*****"
- action: regex
body_format: urlencoded
name: card
regex: "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)"
type: body
value: "$1-****-****-$2"
- name: file-logger
config:
include_req_body: true
path: /tmp/mask-urlencoded-body.log
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
namespace: aic
name: data-mask-route
spec:
parentRefs:
- name: apisix
rules:
- matches:
- path:
type: Exact
value: /anything
filters:
- type: ExtensionRef
extensionRef:
group: apisix.apache.org
kind: PluginConfig
name: data-mask-urlencoded-plugin-config
backendRefs:
- name: httpbin-external-domain
port: 80
将配置应用到集群:
kubectl apply -f data-mask-urlencoded-ic.yaml
apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
namespace: aic
name: httpbin-external-domain
spec:
ingressClassName: apisix
externalNodes:
- type: Domain
name: httpbin.org
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
namespace: aic
name: data-mask-route
spec:
ingressClassName: apisix
http:
- name: data-mask-route
match:
paths:
- /anything
upstreams:
- name: httpbin-external-domain
plugins:
- name: data-mask
enable: true
config:
request:
- action: remove
body_format: urlencoded
name: password
type: body
- action: replace
body_format: urlencoded
name: token
type: body
value: "*****"
- action: regex
body_format: urlencoded
name: card
regex: "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)"
type: body
value: "$1-****-****-$2"
- name: file-logger
enable: true
config:
include_req_body: true
path: /tmp/mask-urlencoded-body.log
将配置应用到集群:
kubectl apply -f data-mask-urlencoded-ic.yaml
❶ 从请求体中删除 password 信息的数据屏蔽规则。
❷ 将请求体中的 token 信息替换为 ***** 的数据屏蔽规则。
❸ 使用正则表达式匹配请求体中的卡号并屏蔽卡号中间部分的数据屏蔽规则。
❹ 文件系统中保存日志的日志文件路径。
向路由发送请求:
curl -i "http://127.0.0.1:9080/anything" \
--data-urlencode "password=abc" \
--data-urlencode "token=xyz" \
--data-urlencode "card=1234-1234-1234-1234"
你应该收到 HTTP/1.1 200 OK 响应。
导航到 /tmp/mask-urlencoded-body.log 文件并检查日志内容,你应该看到类似以下的日志条目:
{
"request": {
"uri": "/anything",
"body": "token=*****&card=1234-****-****-1234",
"method": "POST",
"url": "http://127.0.0.1:9080/anything"
}
}
屏蔽 JSON 编码请求体中的敏感信息
以下示例展示了如何在 file-logger 插件将请求记录到本地文件之前,使用插件中的 JSONPath 语法查找目标字段,并对 JSON 编码请求体中的敏感信息进行脱敏。
创建一个带有 file-logger 插件(用于记录请求)和 data-mask 插件(带有三个数据屏蔽规则)的路由:
- Admin API
- ADC
- Ingress Controller
curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
-H "X-API-KEY: ${ADMIN_API_KEY}" \
-d '{
"id": "data-mask-route",
"uri": "/anything",
"plugins": {
"data-mask": {
"request": [
{
"action": "remove",
"body_format": "json",
"name": "$.password",
"type": "body"
},
{
"action": "replace",
"body_format": "json",
"name": "users[*].token",
"type": "body",
"value": "*****"
},
{
"action": "regex",
"body_format": "json",
"name": "$.users[*].credit.card",
"regex": "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)",
"type": "body",
"value": "$1-****-****-$2"
}
]
},
"file-logger": {
"include_req_body": true,
"path": "/tmp/mask-json-body.log"
}
},
"upstream": {
"type": "roundrobin",
"nodes": {
"httpbin.org:80": 1
}
}
}'
services:
- name: data-mask-service
routes:
- name: data-mask-route
uris:
- /anything
plugins:
data-mask:
request:
- action: remove
body_format: json
name: "$.password"
type: body
- action: replace
body_format: json
name: "users[*].token"
type: body
value: "*****"
- action: regex
body_format: json
name: "$.users[*].credit.card"
regex: "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)"
type: body
value: "$1-****-****-$2"
file-logger:
include_req_body: true
path: /tmp/mask-json-body.log
upstream:
type: roundrobin
nodes:
- host: httpbin.org
port: 80
weight: 1
将配置同步到网关:
adc sync -f adc.yaml
- Gateway API
- APISIX CRD
apiVersion: v1
kind: Service
metadata:
namespace: aic
name: httpbin-external-domain
spec:
type: ExternalName
externalName: httpbin.org
---
apiVersion: apisix.apache.org/v1alpha1
kind: PluginConfig
metadata:
namespace: aic
name: data-mask-json-plugin-config
spec:
plugins:
- name: data-mask
config:
request:
- action: remove
body_format: json
name: "$.password"
type: body
- action: replace
body_format: json
name: "users[*].token"
type: body
value: "*****"
- action: regex
body_format: json
name: "$.users[*].credit.card"
regex: "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)"
type: body
value: "$1-****-****-$2"
- name: file-logger
config:
include_req_body: true
path: /tmp/mask-json-body.log
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
namespace: aic
name: data-mask-route
spec:
parentRefs:
- name: apisix
rules:
- matches:
- path:
type: Exact
value: /anything
filters:
- type: ExtensionRef
extensionRef:
group: apisix.apache.org
kind: PluginConfig
name: data-mask-json-plugin-config
backendRefs:
- name: httpbin-external-domain
port: 80
将配置应用到集群:
kubectl apply -f data-mask-json-ic.yaml
apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
namespace: aic
name: httpbin-external-domain
spec:
ingressClassName: apisix
externalNodes:
- type: Domain
name: httpbin.org
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
namespace: aic
name: data-mask-route
spec:
ingressClassName: apisix
http:
- name: data-mask-route
match:
paths:
- /anything
upstreams:
- name: httpbin-external-domain
plugins:
- name: data-mask
enable: true
config:
request:
- action: remove
body_format: json
name: "$.password"
type: body
- action: replace
body_format: json
name: "users[*].token"
type: body
value: "*****"
- action: regex
body_format: json
name: "$.users[*].credit.card"
regex: "(\\d+)\\-\\d+\\-\\d+\\-(\\d+)"
type: body
value: "$1-****-****-$2"
- name: file-logger
enable: true
config:
include_req_body: true
path: /tmp/mask-json-body.log
将配置应用到集群:
kubectl apply -f data-mask-json-ic.yaml
❶ 从请求体中删除 password 信息的数据屏蔽规则。
❷ 将请求体中的 token 信息替换为 ***** 的数据屏蔽规则。
❸ 使用正则表达式匹配请求体中的卡号并屏蔽卡号中间部分的数据屏蔽规则。
❹ 文件系统中保存日志的日志文件路径。
向路由发送带有请求体中敏感信息的请求:
curl -i "http://127.0.0.1:9080/anything" -X POST -d '
{
"password": "abc",
"users": [
{
"token": "xyz",
"credit": {
"card": "1234-1234-1234-1234"
}
},
{
"token": "xyz",
"credit": {
"card": "1234-1234-1234-1234"
}
}
]
}'
你应该收到 HTTP/1.1 200 OK 响应。
导航到 /tmp/mask-json-body.log 文件并检查日志内容,你应该看到类似以下的日志条目:
{
"request": {
"uri": "/anything",
"body": "{\"users\":[{\"token\":\"*****\",\"credit\":{\"card\":\"1234-****-****-1234\"}},{\"token\":\"*****\",\"credit\":{\"card\":\"1234-****-****-1234\"}}]}",
"method": "POST",
"url": "http://127.0.0.1:9080/anything"
}
}