安全

准备

需要先开启sidecar的自动注入或手动注入

自动注入手动注入

bash

# namespace下的所有服务都会被自动注入sidecar
$ kubectl label namespace <your_namespace> istio-injection=enabled

bash

# 只有yaml中的pod会被注入sidecar
$ kubectl apply -f <(istioctl kube-inject -f <yaml>) -n <namespace>

根据下面的命令，预先创建sleep.yaml和httpbin.yaml。

sleep.yamlhttpbin.yaml

bash

$ cat <<EOF> ./sleep.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sleep
---
apiVersion: v1
kind: Service
metadata:
  name: sleep
  labels:
    app: sleep
spec:
  ports:
    - port: 80
      name: http
  selector:
    app: sleep
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep
  template:
    metadata:
      labels:
        app: sleep
    spec:
      terminationGracePeriodSeconds: 0
      serviceAccountName: sleep
      containers:
        - name: sleep
          image: curlimages/curl
          command: ["/bin/sleep", "infinity"]
          imagePullPolicy: IfNotPresent
EOF

bash

$ cat <<EOF> ./httpbin.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: httpbin
---
apiVersion: v1
kind: Service
metadata:
  name: httpbin
  labels:
    app: httpbin
    service: httpbin
spec:
  ports:
    - name: http
      port: 8000
      targetPort: 80
  selector:
    app: httpbin
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpbin
spec:
  replicas: 1
  selector:
    matchLabels:
      app: httpbin
      version: v1
  template:
    metadata:
      labels:
        app: httpbin
        version: v1
    spec:
      serviceAccountName: httpbin
      containers:
        - image: docker.io/kong/httpbin
          imagePullPolicy: IfNotPresent
          name: httpbin
          ports:
            - containerPort: 80
EOF

分别在命名空间default、test下创建对应的sleep和httpbin服务。

defaulttest

bash

# default下的sleep服务默认不注入sidecar
$ kubectl apply -f ./sleep.yaml)

bash

# 若ns不存在，需要预创建
$ kubectl create ns test
$ kubectl apply -f <(istioctl kube-inject -f ./httpbin.yaml) -n test
$ kubectl apply -f <(istioctl kube-inject -f ./sleep.yaml) -n test

对等认证

未开启mTLS认证

准备步骤中我们在default下创建了没有sidecar的sleep服务，在test下创建了有sidecar的sleep和httpbin服务。

测试未开启mTLS下的服务间访问：

bash

# 从default下的sleep发起http请求（没有注入envoy代理）
$ kubectl exec -it $(kubectl get po -l app=sleep -o jsonpath={.items[0].metadata.name}) -c sleep -- curl http://httpbin.test:8000/ip
{
  "origin": "127.0.0.6"
}
# 从test下的sleep发起http请求
$ kubectl -n test exec -it $(kubectl -n test get po -l app=sleep -o jsonpath={.items[0].metadata.name}) -c sleep -- curl http://httpbin.test:8000/ip
{
  "origin": "127.0.0.6"
}

开启mTLS认证

配置PeerAuthentication来开启test下服务的mTLS认证。

bash

$ kubectl -n test apply -f - <<EOF
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: peer-policy
spec:
  mtls:
    mode: STRICT
EOF

没有定义selector配置，所以默认匹配test下的所有服务。
STRICT表示所有服务都要求mTLS访问。

测试开启mTLS后的服务间的访问

bash

# default下的sleep服务无法访问test下的httpbin
$ kubectl exec -it $(kubectl get po -l app=sleep -o jsonpath={.items[0].metadata.name}) -c sleep -- curl http://httpbin.test:8000/ip
curl: (56) Recv failure: Connection reset by peer
command terminated with exit code 56
# test下的sleep服务仍旧能够访问test下的httpbin
$ kubectl -n test exec -it $(kubectl -n test get po -l app=sleep -o jsonpath={.items[0].metadata.name}) -c sleep -- curl http://httpbin.test:8000/ip
{
  "origin": "127.0.0.6"
}

对等认证开启后，没有代理的sleep服务无法访问httpbin服务，而具有代理的sleep服务则可以访问。

手动给default下的sleep服务注入sidecar

bash

$ kubectl apply -f <(istioctl kube-inject -f ./sleep.yaml)
$ kubectl exec -it $(kubectl get po -l app=sleep -o jsonpath={.items[0].metadata.name}) -c sleep -- curl http://httpbin.test:8000/ip
{
  "origin": "127.0.0.6"
}

给default下的服务注入sidecar后，服务间的访问就能够正常响应了。

请求认证

根据下面的命令，分别创建Gateway、VirtualService和RequestAuthentication三种类型的资源，用于实现集群内外的请求认证。

GatewayVirtualServiceRequestAuthentication

bash

$ kubectl -n test apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: httpbin-gw
spec:
  selector:
    istio: ingressgateway # 绑定istio-system下的istio的默认网关istio-ingressgateway
  servers:
    - port:
        name: http
        number: 80
        protocol: HTTP
      hosts:
        - httpbin.example.com
EOF

bash

$ kubectl -n test apply -f - <<EOF
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: httpbin-vs
spec:
  gateways:
  - httpbin-gw # 对应Gateway资源的名称
  hosts:
  - httpbin.example.com
  http:
  - match:
    - uri:
        prefix: / # 匹配所有的uri
    route:
    - destination:
        host: httpbin
        port:
          number: 8000
EOF

bash

$ kubectl -n istio-system apply -f - <<EOF 
apiVersion: security.istio.io/v1
kind: RequestAuthentication
metadata:
  name: httpbin-ra
spec:
  selector:
    matchLabels:
      istio: ingressgateway # 绑定istio的ingressgateway网关
  jwtRules:
    - issuer: "xiaokexiang@aliyun.com" # 指定访问的issuer
      jwks: | # 由jwt工具生成，jwks对应文本，jwksUri指向配置地址
        {
          "keys": [
            {
              "alg": "RS256",
              "e": "AQAB",
              "kty": "RSA",
              "n": "rW1v_J3cg_fp1kd86AVJi1Z7QM5UzoXqT7JMrxW6dHQ2U22ibU-sdsCryIqGjMkUSNx22FQE0zKNBfmtFcNzGTy0B5pfniy6K9bPWVHQUVde0uSGYumQDqKS9e0hzbVXCAkX1TO3uibkQ6onrR8--rYUNv-fa9thxkJ5Ps7j5kbpWMnpzQ45QzZyevYqls73TYXaVfaKPgD3BRBSE7kKFGk1T6PF70Ch4cm3r_idJG74ZG35LSB9hBqq8rBI-rG3_y_qfsQdqnWxk0ObsXjE1e1MMZsIOWxpfd8wqL0vZmFxwcdPwnUSc2h7lDSMh1O8-FQpVjOBe4JOq68PXr81iw",
              "use": "sig"
            }
          ]
        }
EOF

参考网关访问配置环境变量，并通过网关访问httpbin服务

携带token✔️不携带token✔️

bash

$ curl --resolve "httpbin.example.com:$INGRESS_PORT:$INGRESS_HOST" --header "Authorization: Bearer $TOKEN" httpbin.example.com:$INGRESS_PORT/ip

bash

$ curl --resolve "httpbin.example.com:$INGRESS_PORT:$INGRESS_HOST" httpbin.example.com:$INGRESS_PORT/ip

在没有配置AuthorizationPolicy的前提下，只配置RequestAuthentication即使不携带token也能够访问。
通过jwt-tools来生成公私钥、jwt token和jwks配置。也可以使用jwt在线令牌验证来验证令牌是否合法。
bash
# 生成公私钥
$ ./jwt cert
# 生成token，--iss的值就是ra的yaml中允许访问的名称，不匹配会无法访问
$ ./jwt enc --iss=xiaokexiang@aliyun.com
# 生成token，指定iss和sub
$ ./jwt enc --iss=xiaokexiang@aliyun.com --sub=test-jwt
# 生成jwk配置(其实对应着公钥)
$ ./jwt jwk
1
2
3
4
5
6
7
8

禁止不携带token的访问

bash

$ kubectl -n istio-system apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
 name: refuse-without-token
spec:
 selector:
   matchLabels:
     istio: ingressgateway # 匹配istio默认网关
 action: DENY
 rules:
 - from:
   - source:
       notRequestPrincipals: ["*"] # 表示不携带请求token
EOF

再次测试：通过网关访问服务

携带token✔️不携带token❌️

bash

$ curl --resolve "httpbin.example.com:$INGRESS_PORT:$INGRESS_HOST" --header "Authorization: Bearer $TOKEN" httpbin.example.com:$INGRESS_PORT/ip

bash

$ curl --resolve "httpbin.example.com:$INGRESS_PORT:$INGRESS_HOST" httpbin.example.com:$INGRESS_PORT/ip

复制JWT声明到HTTP头

bash

$ kubectl -n istio-system apply -f - <<EOF 
apiVersion: security.istio.io/v1
kind: RequestAuthentication
metadata:
  name: httpbin-ra
spec:
  selector:
    matchLabels:
      istio: ingressgateway
  jwtRules:
    - issuer: "xiaokexiang@aliyun.com"
      outputClaimToHeaders:
        - header: "X-Jwt-Claim-iss"# 输出的请求头key
          claim: "iss"# 来源jwt的payload.iss
      jwks: |
        {
          "keys": [
            {
              "alg": "RS256",
              "e": "AQAB",
              "kty": "RSA",
              "n": "rW1v_J3cg_fp1kd86AVJi1Z7QM5UzoXqT7JMrxW6dHQ2U22ibU-sdsCryIqGjMkUSNx22FQE0zKNBfmtFcNzGTy0B5pfniy6K9bPWVHQUVde0uSGYumQDqKS9e0hzbVXCAkX1TO3uibkQ6onrR8--rYUNv-fa9thxkJ5Ps7j5kbpWMnpzQ45QzZyevYqls73TYXaVfaKPgD3BRBSE7kKFGk1T6PF70Ch4cm3r_idJG74ZG35LSB9hBqq8rBI-rG3_y_qfsQdqnWxk0ObsXjE1e1MMZsIOWxpfd8wqL0vZmFxwcdPwnUSc2h7lDSMh1O8-FQpVjOBe4JOq68PXr81iw",
              "use": "sig"
            }
          ]
        }
EOF

再次通过istio的入口网关访问httpbin服务的headers接口（以JSON格式返回请求的所有标头信息）

bash

# 查看请求httpbin的headers有哪些
$ curl --resolve "httpbin.example.com:$INGRESS_PORT:$INGRESS_HOST" --header "Authorization: Bearer $TOKEN" httpbin.example.com:$INGRESS_PORT/headers

{
  "headers": {
    "Accept": "*/*", 
    "Host": "httpbin.example.com:30001", 
    "User-Agent": "curl/7.29.0", 
    "X-B3-Parentspanid": "01ec2a461bd75008", 
    "X-B3-Sampled": "0", 
    "X-B3-Spanid": "f1845822abfaa73a", 
    "X-B3-Traceid": "641928c28c78f1cd01ec2a461bd75008", 
    "X-Envoy-Attempt-Count": "1", 
    "X-Envoy-Internal": "true", 
    "X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/test/sa/httpbin;Hash=1dfdb0756685af3ca8ee65884c4747e689add87f401089816fd7b5c53db93fba;Subject=\"\";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account", 
    "X-Jwt-Claim-Iss": "xiaokexiang@aliyun.com"
  }
}

授权

提示

授权(AuthorizaionPolicy)示例基于准备和请求认证章节的配置文件，请先参考配置文件部署资源。

HTTP流量

配置授权的allow-nothing策略，对服务间的httpbin的访问进行控制。

bash

# allow-nothing
$ kubectl -n test apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: allow-nothing
spec:
  action: ALLOW
EOF

通过不定义selector，根据授权规则无ALLOW策略匹配，最终会拒绝来实现allow-nothing策略。

此时无论是同namespace还是跨namespace的服务间访问都是被拒绝。

同namespace❌️跨namespace❌️

bash

$ kubectl -n test exec -it $(kubectl -n test get po -l app=sleep -o jsonpath={.items[0].metadata.name}) -c sleep -- curl http://httpbin.test:8000/ip

bash

$ kubectl -n default exec -it $(kubectl -n default get po -l app=sleep -o jsonpath={.items[0].metadata.name}) -c sleep -- curl http://httpbin.test:8000/ip

添加授权策略以实现：同namespace下的服务可以访问httpbin的ip接口。

bash

$ kubectl -n test apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: allow-ns-test
spec:
  selector:
    matchLabels:
      app: httpbin # 绑定test下的httpbin服务
  action: ALLOW
  rules:
    - from:
        - source:
            namespaces: ["test"] # 匹配test
      to:
        - operation:
            paths: [ "/ip" ] # 拒绝了非ip接口的访问
            methods: [ "GET" ]
EOF

同namespace✔️跨namespace❌️

bash

$ kubectl -n test exec -it $(kubectl -n test get po -l app=sleep -o jsonpath={.items[0].metadata.name}) -c sleep -- curl http://httpbin.test:8000/ip

bash

$ kubectl -n default exec -it $(kubectl -n default get po -l app=sleep -o jsonpath={.items[0].metadata.name}) -c sleep -- curl http://httpbin.test:8000/ip

添加授权策略以实现：持有SA: sleep的服务请求携带标头version:v1即可访问httpbin的ip接口。

bash

$ kubectl -n test apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: allow-ns-test
spec:
  selector:
    matchLabels:
      app: httpbin # 绑定test下的httpbin服务
  action: ALLOW
  rules:
    - from:
        - source:
            principals: ["cluster.local/ns/test/sa/sleep"] # 持有sa: sleep
      to:
        - operation:
            paths: [ "/ip" ] # 拒绝了非ip接口的访问
            methods: [ "GET" ]
      when:
        - key: request.headers[version] # 只有在请求头携带version且值为v1的时候规则才会生效
          values: ["v1"]
EOF

同namespace❌️跨namespace❌️同namespace带标头✔️

bash

$ kubectl -n test exec -it $(kubectl -n test get po -l app=sleep -o jsonpath={.items[0].metadata.name}) -c sleep -- curl -H "version: v1" http://httpbin.test:8000/ip

bash

$ kubectl -n default exec -it $(kubectl -n default get po -l app=sleep -o jsonpath={.items[0].metadata.name}) -c sleep -- curl http://httpbin.test:8000/ip

bash

$ kubectl -n test exec -it $(kubectl -n test get po -l app=sleep -o jsonpath={.items[0].metadata.name}) -c sleep -- curl -H "version: v1" http://httpbin.test:8000/ip

添加授权策略以实现：任何请求只要携带标头version: v1都可以访问httpbin的ip接口。

bash

$ kubectl -n test apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: allow-ns-test
spec:
  selector:
    matchLabels:
      app: httpbin # 绑定test下的httpbin服务
  action: ALLOW
  rules:
    - to:
        - operation:
            paths: [ "/ip" ] # 拒绝了非ip接口的访问
            methods: [ "GET" ]
      when:
        - key: request.headers[version] # 只有在请求头携带version且值为v1的时候规则才会生效
          values: ["v1"]
EOF

网关访问✔️同namespace✔️跨namespace✔️

bash

$ curl --resolve "httpbin.example.com:$INGRESS_PORT:$INGRESS_HOST" -H "Authorization: Bearer $TOKEN" -H "version:v1" httpbin.example.com:$INGRESS_PORT/ip

bash

$ kubectl -n test exec -it $(kubectl -n test get po -l app=sleep -o jsonpath={.items[0].metadata.name}) -c sleep -- curl -H "version:v1" http://httpbin:8000/ip

bash

$ kubectl -n default exec -it $(kubectl -n default get po -l app=sleep -o jsonpath={.items[0].metadata.name}) -c sleep -- curl -H "version:v1" http://httpbin.test:8000/ip

TCP流量

新建namespace: foo并开启sidecar自动注入

bash

$ kubectl create ns foo
$ kubectl label foo istio-injection=enabled

根据下面的配置部署服务

sleeptcp-echo

bash

$ kubectl -n foo apply -f - <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sleep
---
apiVersion: v1
kind: Service
metadata:
  name: sleep
  labels:
    app: sleep
    service: sleep
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: sleep
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep
  template:
    metadata:
      labels:
        app: sleep
    spec:
      terminationGracePeriodSeconds: 0
      serviceAccountName: sleep
      containers:
      - name: sleep
        image: curlimages/curl
        command: ["/bin/sleep", "infinity"]
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - mountPath: /etc/sleep/tls
          name: secret-volume
      volumes:
      - name: secret-volume
        secret:
          secretName: sleep-secret
          optional: true
---
EOF

bash

$ kubectl -n foo apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: tcp-echo
  labels:
    app: tcp-echo
    service: tcp-echo
spec:
  ports:
  - name: tcp
    port: 9000
  - name: tcp-other
    port: 9001
  selector:
    app: tcp-echo
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tcp-echo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tcp-echo
      version: v1
  template:
    metadata:
      labels:
        app: tcp-echo
        version: v1
    spec:
      containers:
      - name: tcp-echo
        image: docker.io/istio/tcp-echo-server:1.2
        imagePullPolicy: IfNotPresent
        args: [ "9000,9001,9002", "hello" ]
        ports:
        - containerPort: 9000
        - containerPort: 9001
EOF

tcp-echo用于处理TCP请求，会将请求的参数添加hello并返回。例如传递world，若返回hello world则表示服务正常响应。

测试tcp-echo服务的9000、9001是否正常处理请求

9000✔️9001✔️

bash

$ kubectl -n foo exec -it "$(kubectl -n foo get po -l app=sleep -o jsonpath={.items..metadata.name})" -c sleep -- sh -c \
'echo "port 9000" | nc tcp-echo 9000' | grep "hello" && echo 'connection succeeded' || echo 'connection rejected'

bash

$ kubectl -n foo exec -it "$(kubectl -n foo get po -l app=sleep -o jsonpath={.items..metadata.name})" -c sleep -- sh -c \
'echo "port 9001" | nc tcp-echo 9001' | grep "hello" && echo 'connection succeeded' || echo 'connection rejected'

配置授权策略以实现：只允许9000的端口访问，拒绝9001端口

bash

$ kubectl -n foo apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: tcp-policy
spec:
  selector:
    matchLabels:
      app: tcp-echo
  action: ALLOW
  rules:
  - to:
    - operation:
        ports: ["9000"]
EOF

9000✔️9001❌️

bash

$ kubectl -n foo exec -it "$(kubectl -n foo get po -l app=sleep -o jsonpath={.items..metadata.name})" -c sleep -- sh -c \
'echo "port 9000" | nc tcp-echo 9000' | grep "hello" && echo 'connection succeeded' || echo 'connection rejected'

bash

$ kubectl -n foo exec -it "$(kubectl -n foo get po -l app=sleep -o jsonpath={.items..metadata.name})" -c sleep -- sh -c \
'echo "port 9001" | nc tcp-echo 9001' | grep "hello" && echo 'connection succeeded' || echo 'connection rejected'

配置授权策略以实现：设置HTTP-ONLY的参数的ALLOW策略实现拒绝9000、9001端口的请求访问。

bash

$ kubectl -n foo apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: tcp-policy
spec:
  selector:
    matchLabels:
      app: tcp-echo
  action: ALLOW
  rules:
  - to:
    - operation:
        ports: ["9000"]
        methods: ["GET"]
EOF

对TCP流量使用了HTTP-ONLY字段（methods），这会导致ALLOW规则无效且请求与任何ALLOW规则都无法匹配（与DENY规则不同）。最终该请求被拒绝。

9000❌️9001❌️

bash

$ kubectl -n foo exec -it "$(kubectl -n foo get po -l app=sleep -o jsonpath={.items..metadata.name})" -c sleep -- sh -c \
'echo "port 9000" | nc tcp-echo 9000' | grep "hello" && echo 'connection succeeded' || echo 'connection rejected'

bash

$ kubectl -n foo exec -it "$(kubectl -n foo get po -l app=sleep -o jsonpath={.items..metadata.name})" -c sleep -- sh -c \
'echo "port 9001" | nc tcp-echo 9001' | grep "hello" && echo 'connection succeeded' || echo 'connection rejected'

配置授权策略以实现：设置HTTP-ONLY的参数的DENY策略实现拒绝9000、9001端口的请求访问。

bash

$ kubectl -n foo apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: tcp-policy
spec:
  selector:
    matchLabels:
      app: tcp-echo
  action: DENY
  rules:
  - to:
    - operation:
        methods: ["GET"]
EOF

因为TCP流量不理解HTTP-ONLY字段，所以拒绝所有到TCP端口的流量，这里就是全部拒绝。

9000❌️9001❌️

bash

$ kubectl -n foo exec -it "$(kubectl -n foo get po -l app=sleep -o jsonpath={.items..metadata.name})" -c sleep -- sh -c \
'echo "port 9000" | nc tcp-echo 9000' | grep "hello" && echo 'connection succeeded' || echo 'connection rejected'

bash

$ kubectl -n foo exec -it "$(kubectl -n foo get po -l app=sleep -o jsonpath={.items..metadata.name})" -c sleep -- sh -c \
'echo "port 9001" | nc tcp-echo 9001' | grep "hello" && echo 'connection succeeded' || echo 'connection rejected'

配置授权策略以实现：9000的端口访问只允许HTTP的GET访问，其他端口不受限制。

bash

$ kubectl -n foo apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: tcp-policy
spec:
  selector:
    matchLabels:
      app: tcp-echo
  action: DENY
  rules:
  - to:
    - operation:
        ports: ["9000"]
        methods: ["GET"]
EOF

与ALLOW规则不同的是，DENY类型的规则匹配9000端口（虽然不识别methods），所以会拒绝9000端口，允许90001端口的访问。

9000❌️9001✔️

bash

$ kubectl -n foo exec -it "$(kubectl -n foo get po -l app=sleep -o jsonpath={.items..metadata.name})" -c sleep -- sh -c \
'echo "port 9000" | nc tcp-echo 9000' | grep "hello" && echo 'connection succeeded' || echo 'connection rejected'

bash

$ kubectl -n foo exec -it "$(kubectl -n foo get po -l app=sleep -o jsonpath={.items..metadata.name})" -c sleep -- sh -c \
'echo "port 9001" | nc tcp-echo 9001' | grep "hello" && echo 'connection succeeded' || echo 'connection rejected'

JWT令牌

在禁止不携带token的访问中，我们通过给istio的入口网关设置授权策略，来拒绝任何不携带token的请求。同样，我们可以通过给服务设置授权策略，来实现访问服务的令牌必须在身份列表中。

bash

$ kubectl -n test apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: httpbin-with-token
spec:
  selector:
    matchLabels:
      app: httpbin
  action: ALLOW
  rules:
  - from:
    - source:
       requestPrincipals: ["xiaokexiang@aliyun.com"]
EOF

授权策略会校验TOKEN中的iss和sub是否与requestPrincipals的参数对应。若匹配则允许访问。

错误排查

在访问某个服务的过程中，出现了预期外的授权问题，我们可以通过开启对应授权服务的debug日志来排查问题。

单服务授权策略

先部署授权策略，只允许namespace: test下的服务访问

bash

$ kubectl -n test apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: httpbin-with-token
spec:
  selector:
    matchLabels:
      app: httpbin
  action: ALLOW
  rules:
  - from:
    - source:
       principals: ["default"]
EOF

开启test下httpbin服务RBAC的debug日志

bash

# 开启test下httpbin服务的RBAC的debug日志
$ istioctl proxy-config log $(kubectl -n test get po -l app=httpbin -o jsonpath={.items[0].metadata.name}) -n test  --level "rbac:debug" | grep rbac

日志开启后，通过下述命令，查看httpbin的istio-proxy的RBAC日志

bash

$ kubectl -n test logs -f --tail 200 -l app=httpbin -c istio-proxy

执行访问请求

bash

$ kubectl -n test exec -it $(kubectl -n test get po -l app=sleep -o jsonpath={.items[0].metadata.name}) -c sleep -- curl http://httpbin.test:8000/ip

日志结果如下:

bash

checking request: requestedServerName: outbound_.8000_._.httpbin.test.svc.cluster.local, 
sourceIP: 10.244.0.7:57262, directRemoteIP: 10.244.0.7:57262, remoteIP: 10.244.0.7:57262,
localAddress: 10.244.0.8:80, ssl: uriSanPeerCertificate: 
spiffe://cluster.local/ns/test/sa/sleep, dnsSanPeerCertificate: , 
subjectPeerCertificate: , headers: ':authority', 'httpbin.test:8000'
':path', '/ip' # 请求地址
':method', 'GET' # 请求方法
':scheme', 'http' # 请求方法
'user-agent', 'curl/8.5.0'
'accept', '*/*'
'x-forwarded-proto', 'http'
'x-request-id', 'c1fdae55-dc33-407e-a5e7-839d1581064d'
'x-envoy-attempt-count', '1'
'x-b3-traceid', '4328ba9e7b86417d7f11b6f743e91e3e'
'x-b3-spanid', '7f11b6f743e91e3e'
'x-b3-sampled', '0'
'x-forwarded-client-cert', 
'By=spiffe://cluster.local/ns/test/sa/httpbin;
Hash=f0212791e47be48a49dd51d93e42f625777d23eb658ce3f140cad0d3246da39b;Subject="";
URI=spiffe://cluster.local/ns/test/sa/sleep' # 请求证书，用来判断请求来源和请求目标
, dynamicMetadata:      thread=29
enforced denied, matched policy none thread=29 # 授权策略是否匹配，是允许还是拒绝

6-8行表明了请求的路径，请求的方法、请求的方式。
17-20行表明了请求的来源和请求的目标
22行表示了授权策略是否匹配，被允许了还是拒绝（因为只允许default下的服务访问，所以此处为拒绝）

多服务授权策略

我们修改授权策略，允许istio-system下的服务访问，用来测试流量通过入口网关的到httpbin服务的多服务授权链路排查。

bash

$ kubectl -n test apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: httpbin-with-token
spec:
  selector:
    matchLabels:
      app: httpbin
  action: ALLOW
  rules:
  - from:
    - source:
       namespaces: ["default"]
       namespaces: ["istio-system"]
EOF

提示

前文已经给入口网关配置了RequestAuthentication请求认证、AuthorizationPolicy授权策略。

开启入口网关RBAC的debug日志

bash

# 开启test下httpbin服务的RBAC的debug日志
$ istioctl proxy-config log $(kubectl -n istio-system get po -l istio=ingressgateway -o jsonpath={.items[0].metadata.name}) -n istio-system  --level "rbac:debug" | grep rbac

查看ingressgateway的istio-proxy的RBAC日志

bash

$ kubectl -n istio-system logs -f --tail 200 -l istio=ingressgateway -c istio-proxy

执行访问请求

bash

$ curl --resolve "httpbin.example.com:$INGRESS_PORT:$INGRESS_HOST" --header "Authorization: Bearer $TOKEN" httpbin.example.com:$INGRESS_PORT/ip

最终ingressgateway和httpbin的istio-proxy都打印了RBAC的debug日志。

ingressgateway ->httpbin

bash

checking request: requestedServerName: , sourceIP: 10.244.0.1:53362, directRemoteIP: 10.244.0.1:53362, remoteIP: 10.244.0.1:53362,localAddress: 10.244.0.6:8080, ssl: none, headers: ':authority', 'httpbin.example.com:30001'
':path', '/ip'
':method', 'GET'
':scheme', 'http'
'user-agent', 'curl/7.29.0'
'accept', '*/*'
'x-forwarded-for', '10.244.0.1'
'x-forwarded-proto', 'http'
'x-envoy-internal', 'true'
'x-request-id', 'c2af173e-8680-4663-9bd4-f05fc7a3dd01'
'x-envoy-decorator-operation', 'httpbin.test.svc.cluster.local:8000/*'
'x-envoy-peer-metadata-id', 'router~10.244.0.6~istio-ingressgateway-7ccdc6cf88-5sf9f.istio-system~istio-system.svc.cluster.local'
'x-envoy-peer-metadata', 'ChoKCkNMVVNURVJfSUQSDBoKS3ViZXJuZXRlcwocCgxJTlNUQU5DRV9JUFMSDBoKMTAuMjQ0LjAuNgoZCg1JU1RJT19WRVJTSU9OEggaBjEuMjAuMQqcAwoGTEFCRUxTEpEDKo4DCh0KA2FwcBIWGhRpc3Rpby1pbmdyZXNzZ2F0ZXdheQoTCgVjaGFydBIKGghnYXRld2F5cwoUCghoZXJpdGFnZRIIGgZUaWxsZXIKNgopaW5zdGFsbC5vcGVyYXRvci5pc3Rpby5pby9vd25pbmctcmVzb3VyY2USCRoHdW5rbm93bgoZCgVpc3RpbxIQGg5pbmdyZXNzZ2F0ZXdheQoZCgxpc3Rpby5pby9yZXYSCRoHZGVmYXVsdAowChtvcGVyYXRvci5pc3Rpby5pby9jb21wb25lbnQSERoPSW5ncmVzc0dhdGV3YXlzChIKB3JlbGVhc2USBxoFaXN0aW8KOQofc2VydmljZS5pc3Rpby5pby9jYW5vbmljYWwtbmFtZRIWGhRpc3Rpby1pbmdyZXNzZ2F0ZXdheQovCiNzZXJ2aWNlLmlzdGlvLmlvL2Nhbm9uaWNhbC1yZXZpc2lvbhIIGgZsYXRlc3QKIgoXc2lkZWNhci5pc3Rpby5pby9pbmplY3QSBxoFZmFsc2UKGgoHTUVTSF9JRBIPGg1jbHVzdGVyLmxvY2FsCi8KBE5BTUUSJxolaXN0aW8taW5ncmVzc2dhdGV3YXktN2NjZGM2Y2Y4OC01c2Y5ZgobCglOQU1FU1BBQ0USDhoMaXN0aW8tc3lzdGVtCl0KBU9XTkVSElQaUmt1YmVybmV0ZXM6Ly9hcGlzL2FwcHMvdjEvbmFtZXNwYWNlcy9pc3Rpby1zeXN0ZW0vZGVwbG95bWVudHMvaXN0aW8taW5ncmVzc2dhdGV3YXkKJwoNV09SS0xPQURfTkFNRRIWGhRpc3Rpby1pbmdyZXNzZ2F0ZXdheQ=='
'x-jwt-claim-iss', 'xiaokexiang@aliyun.com'
, dynamicMetadata: filter_metadata {
  key: "envoy.filters.http.jwt_authn"
  value {
    fields {
      key: "xiaokexiang@aliyun.com"
      value {
        struct_value {
          fields {
            key: "exp"
            value {
              number_value: 1705992075
            }
          }
          fields {
            key: "iss"
            value {
              string_value: "xiaokexiang@aliyun.com"
            }
          }
          fields {
            key: "sub"
            value {
              string_value: "httpbin"
            }
          }
        }
      }
    }
  }
}
filter_metadata {
  key: "istio_authn"
  value {
    fields {
      key: "request.auth.claims"
      value {
        struct_value {
          fields {
            key: "iss"
            value {
              list_value {
                values {
                  string_value: "xiaokexiang@aliyun.com"
                }
              }
            }
          }
          fields {
            key: "sub"
            value {
              list_value {
                values {
                  string_value: "httpbin"
                }
              }
            }
          }
        }
      }
    }
    fields {
      key: "request.auth.principal"
      value {
        string_value: "xiaokexiang@aliyun.com/httpbin"
      }
    }
    fields {
      key: "request.auth.raw_claims"
      value {
        string_value: "{\"iss\":\"xiaokexiang@aliyun.com\",\"sub\":\"httpbin\",\"exp\":1705992075}"
      }
    }
  }
}
enforced allowed, matched policy nonethread=38

bash

checking request: requestedServerName: outbound_.8000_._.httpbin.test.svc.cluster.local, sourceIP: 10.244.0.6:58366, directRemoteIP: 10.244.0.6:58366, remoteIP: 10.244.0.1:0,localAddress: 10.244.0.8:80, ssl: uriSanPeerCertificate: spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account, dnsSanPeerCertificate: , subjectPeerCertificate: , headers: ':authority', 'httpbin.example.com:30001'
':path', '/ip'
':method', 'GET'
':scheme', 'http'
'user-agent', 'curl/7.29.0'
'accept', '*/*'
'x-forwarded-for', '10.244.0.1'
'x-forwarded-proto', 'http'
'x-request-id', 'c2af173e-8680-4663-9bd4-f05fc7a3dd01'
'x-jwt-claim-iss', 'xiaokexiang@aliyun.com'
'x-envoy-attempt-count', '1'
'x-b3-traceid', 'a954349a3f0e03a3b07ff42621241ccf'
'x-b3-spanid', 'b07ff42621241ccf'
'x-b3-sampled', '0'
'x-envoy-internal', 'true'
'x-forwarded-client-cert', 'By=spiffe://cluster.local/ns/test/sa/httpbin;Hash=e040d0ebb454fe0d7c5521bf0a424b30d17dd7f81d50edf93e1ca14679ea0401;Subject="";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account'
, dynamicMetadata:      thread=29
enforced allowed, matched policy ns[test]-policy[httpbin-with-token]-rule[0]  thread=29

ingressgateway和httpbin的istio-proxy都输出了授权允许的RBAC日志信息。
相比httpbin，ingressgateway因为有请求认证，所以额外输出了解密后JWT相关信息。

模拟运行

bash

$ kubectl -n test apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: httpbin-with-token
  annotations:
    "istio.io/dry-run": "true"
spec:
  selector:
    matchLabels:
      app: httpbin
  action: ALLOW
  rules:
  - from:
    - source:
       namespaces: ["istio-system"]
EOF

持有"istio.io/dry-run": "true"注解的授权策略会被创建，但授权策略并不会真正生效。

通过Zipkin，更好的理解流量的执行链路，同时避免不正确的授权引起生产流量中断。

bash

for i in {1..20}; do curl --resolve "httpbin.example.com:$INGRESS_PORT:$INGRESS_HOST" -H "Authorization: Bearer $TOKEN" -H "X-B3-Sampled: 1" httpbin.example.com:$INGRESS_PORT/ip  -s -o /dev/null -w "%{http_code}\n"; sleep 1; done

X-B3-Sampled: 1表示zipkin对数据进行了采样追踪，因为这里还没有通过TelemetryApi启用链路追踪，所以需要手动传入请求头来记录追踪。

链路追踪如下所示：

zipkin_1

安全 ​

准备 ​

对等认证 ​

未开启mTLS认证 ​

开启mTLS认证 ​

请求认证 ​

禁止不携带token的访问 ​

复制JWT声明到HTTP头 ​

授权 ​

HTTP流量 ​

TCP流量 ​

JWT令牌 ​

错误排查 ​

单服务授权策略 ​

多服务授权策略 ​

模拟运行 ​

安全

准备

对等认证

未开启mTLS认证

开启mTLS认证

请求认证

禁止不携带token的访问

复制JWT声明到HTTP头

授权

HTTP流量

TCP流量

JWT令牌

错误排查

单服务授权策略

多服务授权策略

模拟运行