科学上网--k8s版

昨天折腾了一下，在自建的aks集群上搭好了出国的机场，用cert-manager自动签发和续期证书、域名动态的解析到service public ip，即使ip被封禁了也无所谓，毕竟云原生🤣。。

原文地址：https://github.com/cvvz/k8s-playground/tree/master/gost#科学上网-k8s版

pre-requisite

  1. 购买域名
  2. 准备好以下命令行工具: envsubst, cmctl, kubectl, az, helm

step 1: 配置DNS

step 1.1: 创建 azure dns zone

1export AZURE_DEFAULTS_GROUP=your-resource-group
2export DOMAIN_NAME=your-domain-name 
3az network dns zone create --name $DOMAIN_NAME

step 1.2: 在域名提供商的控制台中设置域名DNS的NS records为Azure authoritative DNS servers

执行以下命令获取 azure authoritative DNS servers 列表：

1az network dns zone show --name $DOMAIN_NAME --query nameServers -o tsv

step 1.3: 等待ns record 传播完成，可能需要几个小时。

通过以下命令验证是否能成功解析到ns record：

1dig $DOMAIN_NAME ns +trace +nodnssec

step 2: Enable workload identity feature

cert-manager 需要调用azure api，使用workload identity进行鉴权

1az extension add --name aks-preview
2
3az feature register --namespace "Microsoft.ContainerService" --name "EnableWorkloadIdentityPreview"
4
5# 执行以下命令，直到状态变为 Registered：
6az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/EnableWorkloadIdentityPreview')].{Name:name,State:properties.state}"
7
8az provider register --namespace Microsoft.ContainerService

step 3： 创建aks集群

 1export CLUSTER=your-aks-cluster-name
 2# region建议选择east asia，香港机房，网络延迟相对更小
 3export AZURE_DEFAULTS_LOCATION=eastasia
 4
 5# 创建集群
 6az aks create -n ${CLUSTER} \
 7--enable-oidc-issuer \
 8--enable-workload-identity 
 9
10az aks get-credentials -n ${CLUSTER}

step 4: 部署cert-manager

 1cat <<EOF > /tmp/values.yaml
 2podLabels:
 3  azure.workload.identity/use: "true"
 4serviceAccount:
 5  labels:
 6    azure.workload.identity/use: "true"
 7EOF
 8
 9helm repo add jetstack https://charts.jetstack.io
10helm repo update
11helm upgrade cert-manager jetstack/cert-manager \
12    --install \
13    --create-namespace \
14    --wait \
15    --namespace cert-manager \
16    --set installCRDs=true \
17    --reuse-values \
18    --values /tmp/values.yaml

step 5: 为cert-manager配置federated workload identity

eastasia region不支持 workload identity，参考：https://learn.microsoft.com/en-us/azure/active-directory/workload-identities/workload-identity-federation-considerations#unsupported-regions-user-assigned-managed-identities

选择一个支持的region（比如japaneast）中创建workload identity

 1export USER_ASSIGNED_IDENTITY_NAME=your-cert-manager-identity-name
 2export IDENTITY_RG=your-identity-group
 3export IDENTITY_RG_LOCATION=your-identity-group-location
 4
 5az group create -n ${IDENTITY_RG} -l ${IDENTITY_RG_LOCATION}
 6az identity create --name "${USER_ASSIGNED_IDENTITY_NAME}" -g ${IDENTITY_RG} -l ${IDENTITY_RG_LOCATION}
 7
 8export USER_ASSIGNED_IDENTITY_CLIENT_ID=$(az identity show --name "${USER_ASSIGNED_IDENTITY_NAME}" --query 'clientId' -o tsv -g ${IDENTITY_RG})
 9az role assignment create \
10    --role "DNS Zone Contributor" \
11    --assignee $USER_ASSIGNED_IDENTITY_CLIENT_ID \
12    --scope $(az network dns zone show --name $DOMAIN_NAME -o tsv --query id)
13
14# cert-manager的service account和namespace
15export SERVICE_ACCOUNT_NAME=cert-manager 
16export SERVICE_ACCOUNT_NAMESPACE=cert-manager 
17
18export SERVICE_ACCOUNT_ISSUER=$(az aks show --name $CLUSTER --query "oidcIssuerProfile.issuerUrl" -o tsv)
19az identity federated-credential create \
20  --name "cert-manager" \
21  --identity-name "${USER_ASSIGNED_IDENTITY_NAME}" \
22  --issuer "${SERVICE_ACCOUNT_ISSUER}" \
23  --subject "system:serviceaccount:${SERVICE_ACCOUNT_NAMESPACE}:${SERVICE_ACCOUNT_NAME}" \
24  -g ${IDENTITY_RG}

step 6: 生成证书

 1export GOST_NS=gost
 2kubectl create ns $GOST_NS
 3
 4# 创建issuer
 5export EMAIL_ADDRESS=<email-address> 
 6export AZURE_SUBSCRIPTION_ID=<your-subscription-id>  
 7wget https://raw.githubusercontent.com/cvvz/k8s-playground/master/gost/clusterissuer-lets-encrypt.yaml 
 8envsubst < clusterissuer-lets-encrypt.yaml | kubectl apply -f  -
 9kubectl describe clusterissuer letsencrypt-production
10
11# 创建Certificate
12wget https://raw.githubusercontent.com/cvvz/k8s-playground/master/gost/certificate.yaml 
13envsubst < certificate.yaml | kubectl apply -f -
14
15# 验证证书状态
16cmctl status certificate www -n $GOST_NS
17cmctl inspect secret www-tls -n $GOST_NS

step 7: 部署gost服务

 1export AZURE_LOADBALANCER_DNS_LABEL_NAME=lb-$(uuidgen) 
 2export USER=your-user-name
 3export PASSWORD=your-password
 4
 5# deployment
 6wget https://raw.githubusercontent.com/cvvz/k8s-playground/master/gost/deployment.yaml
 7envsubst < deployment.yaml | kubectl apply -f -
 8# service
 9wget https://raw.githubusercontent.com/cvvz/k8s-playground/master/gost/service.yaml
10envsubst < service.yaml | kubectl apply -f -

step 8: 设置dns record

1# 设置www A record
2az network dns record-set cname set-record \
3    --zone-name $DOMAIN_NAME \
4    --cname $AZURE_LOADBALANCER_DNS_LABEL_NAME.$AZURE_DEFAULTS_LOCATION.cloudapp.azure.com \
5    --record-set-name www
6
7# 验证可以解析到service external ip
8dig www.$DOMAIN_NAME A

不管pod和service ip怎么变化，只要$AZURE_LOADBALANCER_DNS_LABEL_NAME不变，域名始终会解析到service的public ip。

所以就算ip被封禁，重新创建一个service生成新的public ip就行了。

step 9：验证

1curl -v "https://www.google.com" --proxy "https://www.$DOMAIN_NAME" --proxy-user $USER:$PASSWORD