Kubernetes借助vpn打通本地和集群内网

1 搭建集群

搭建Kubernetes集群,可以参考《Ubuntu 18.04快速部署Kubernetes集群
  网络类型选择calico。

2 基本配置

执行如下脚本 config_openvpn_test.sh

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
#!/bin/bash
VOLUME="$HOME/openvpn"
vpn_ip="vpn.coder4.com"
# init for first time only
rm -rf $VOLUME
mkdir -p $VOLUME
docker run -v $VOLUME:/etc/openvpn --rm kylemanna/openvpn ovpn_genconfig -u udp://$vpn_ip -s 10.4.0.0/24
docker run -v $VOLUME:/etc/openvpn --rm -it kylemanna/openvpn ovpn_initpki
#!/bin/bash VOLUME="$HOME/openvpn" vpn_ip="vpn.coder4.com" # init for first time only rm -rf $VOLUME mkdir -p $VOLUME docker run -v $VOLUME:/etc/openvpn --rm kylemanna/openvpn ovpn_genconfig -u udp://$vpn_ip -s 10.4.0.0/24 docker run -v $VOLUME:/etc/openvpn --rm -it kylemanna/openvpn ovpn_initpki
#!/bin/bash
VOLUME="$HOME/openvpn"
vpn_ip="vpn.coder4.com"
# init for first time only
rm -rf $VOLUME
mkdir -p $VOLUME 
docker run -v $VOLUME:/etc/openvpn --rm kylemanna/openvpn ovpn_genconfig -u udp://$vpn_ip -s 10.4.0.0/24
docker run -v $VOLUME:/etc/openvpn --rm -it kylemanna/openvpn ovpn_initpki 

3. 创建docker

./run_openvpn_test.sh

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
#!/bin/bash
# submit to tool node
NAME="openvpn"
VOLUME="$HOME/openvpn"
dns_ip="10.96.0.10"
# stop & run server (should call init_open_vpn_test.sh before)
docker ps -q -a --filter "name=$NAME" | xargs -I {} docker rm -f {}
docker run \
--name $NAME \
--network bridge \
--dns $dns_ip \
-d \
-v $VOLUME:/etc/openvpn \
-p 1194:1194/udp \
--cap-add=NET_ADMIN \
--restart always \
kylemanna/openvpn \
ovpn_run --cipher AES-128-CBC
#!/bin/bash # submit to tool node NAME="openvpn" VOLUME="$HOME/openvpn" dns_ip="10.96.0.10" # stop & run server (should call init_open_vpn_test.sh before) docker ps -q -a --filter "name=$NAME" | xargs -I {} docker rm -f {} docker run \ --name $NAME \ --network bridge \ --dns $dns_ip \ -d \ -v $VOLUME:/etc/openvpn \ -p 1194:1194/udp \ --cap-add=NET_ADMIN \ --restart always \ kylemanna/openvpn \ ovpn_run --cipher AES-128-CBC
#!/bin/bash

# submit to tool node
NAME="openvpn"
VOLUME="$HOME/openvpn"
dns_ip="10.96.0.10"

# stop & run server (should call init_open_vpn_test.sh before) 
docker ps -q -a --filter "name=$NAME" | xargs -I {} docker rm -f {}
docker run \
    --name $NAME \
    --network bridge \
    --dns $dns_ip \
    -d \
    -v $VOLUME:/etc/openvpn \
    -p 1194:1194/udp \
    --cap-add=NET_ADMIN \
    --restart always \
    kylemanna/openvpn \
    ovpn_run --cipher AES-128-CBC

4. 生成客户端配置文件

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
#!/bin/bash
if [ x"$#" != x"1" ];then
echo "Usage: $0 <username>"
exit -1
fi
USERNAME="$1"
OVPN_FILE="$USERNAME.ovpn"
CIPHER="AES-128-CBC"
DNS_IP="10.96.0.10"
ROUTE_CMD="route 192.168.0.0 255.255.0.0"
VOLUME="$HOME/openvpn"
# generate client cert for username
docker run -v $VOLUME:/etc/openvpn --rm -it kylemanna/openvpn easyrsa build-client-full $USERNAME nopass
docker run -v $VOLUME:/etc/openvpn --rm kylemanna/openvpn ovpn_getclient $USERNAME > $OVPN_FILE
# post process
sed -i 's/redirect-gateway.*$//' $OVPN_FILE
cat >> $OVPN_FILE <<EOF
# disable lzo
comp-lzo no
# add this line, the swarm network route
$ROUTE_CMD
# dns update
dhcp-option DNS $DNS_IP
dhcp-option DOMAIN default.svc.cluster.local
dhcp-option DOMAIN svc.cluster.local
dhcp-option DOMAIN cluster.local
script-security 2
up /etc/openvpn/update-resolv-conf
down /etc/openvpn/update-resolv-conf
# security
cipher $CIPHER
EOF
#!/bin/bash if [ x"$#" != x"1" ];then echo "Usage: $0 <username>" exit -1 fi USERNAME="$1" OVPN_FILE="$USERNAME.ovpn" CIPHER="AES-128-CBC" DNS_IP="10.96.0.10" ROUTE_CMD="route 192.168.0.0 255.255.0.0" VOLUME="$HOME/openvpn" # generate client cert for username docker run -v $VOLUME:/etc/openvpn --rm -it kylemanna/openvpn easyrsa build-client-full $USERNAME nopass docker run -v $VOLUME:/etc/openvpn --rm kylemanna/openvpn ovpn_getclient $USERNAME > $OVPN_FILE # post process sed -i 's/redirect-gateway.*$//' $OVPN_FILE cat >> $OVPN_FILE <<EOF # disable lzo comp-lzo no # add this line, the swarm network route $ROUTE_CMD # dns update dhcp-option DNS $DNS_IP dhcp-option DOMAIN default.svc.cluster.local dhcp-option DOMAIN svc.cluster.local dhcp-option DOMAIN cluster.local script-security 2 up /etc/openvpn/update-resolv-conf down /etc/openvpn/update-resolv-conf # security cipher $CIPHER EOF
#!/bin/bash

if [ x"$#" != x"1" ];then
    echo "Usage: $0 <username>"
    exit -1
fi

USERNAME="$1"
OVPN_FILE="$USERNAME.ovpn"
CIPHER="AES-128-CBC"
DNS_IP="10.96.0.10"
ROUTE_CMD="route 192.168.0.0 255.255.0.0"

VOLUME="$HOME/openvpn"
# generate client cert for username 
docker run -v $VOLUME:/etc/openvpn --rm -it kylemanna/openvpn easyrsa build-client-full $USERNAME nopass
docker run -v $VOLUME:/etc/openvpn --rm kylemanna/openvpn ovpn_getclient $USERNAME > $OVPN_FILE

# post process
sed -i 's/redirect-gateway.*$//' $OVPN_FILE

cat >> $OVPN_FILE <<EOF

# disable lzo
comp-lzo no

# add this line, the swarm network route
$ROUTE_CMD

# dns update
dhcp-option DNS $DNS_IP 
dhcp-option DOMAIN default.svc.cluster.local
dhcp-option DOMAIN svc.cluster.local
dhcp-option DOMAIN cluster.local

script-security 2
up /etc/openvpn/update-resolv-conf
down /etc/openvpn/update-resolv-conf

# security
cipher $CIPHER 

EOF

然后在客户端试一下,看能不能打通内网啦~

特殊说明:

  1. Kubernetes网络模型很多,实现差别很大,上述calico + docker bridge(nat)的方式 配合才能生效,其他网络模型和组合不保证能成功
  2. 由于docker用的是bridge存在NAT性能可能不是太好,但对于一般查找问题是足够的了
  3. DNS如果要完全做到和k8s一样,还要配置local domain name,这里不再赘述
  4. 由于一些你懂的原因,如果openvpn的server和client跨国了,会被断开或者无法访问,UDP也不行,不过对于服务器应用还好,一般都是服务器部署在国内,客户端也在国内。
  5. 新版里面客户端可能要加comp-lzo no选项。

 

 

One thought on “Kubernetes借助vpn打通本地和集群内网

Leave a Reply to Anonymous Cancel reply

Your email address will not be published. Required fields are marked *