1.flannel介紹
flannel是coreos開發的容器網絡解決方案。flannel為每個host分配一個subnet，容器從此subnet中分配ip。這些ip可以在host間路由，容器間無需nat和port mapping就可以跨主機通訊。
每個subnet都是從一個更大的ip池中劃分的，flannel會在每個主機上運行一個叫flanneld得agent，其職責是從ip池中分配subnet。為了在各個主機間共享信息，flannel用etcd存放網絡配置，已分配的subnet，host的ip等信息。
數據包通過backend在主機間轉發。
flannel提供了多種backend，最常用的有vxlan和host-gw。
2.部署實驗環境
三個虛機
docker1 docker2 docker3?
etcd安裝在docker1
docker1 docker2 docker3上運行flanneld
注:為了更方便的驗證flannel和etcd所以docker1也安裝了flannel，
其實可以不用在docker1安裝
centos7自帶了軟件包，直接yum安裝即可
2.1?
安裝配置etcd
yum -y install etcd
[root@docker1 ~]# systemctl start etcd? && systemctl enable etcd
[root@docker1 ~]#測試下
[root@docker1 ~]# etcd --version
etcd Version: 3.2.18
Git SHA: eddf599
Go Version: go1.9.4
Go OS/Arch: linux/amd64
[root@docker1 ~]#
[root@docker1 ~]# etcdctl set test "a"
a
[root@docker1 ~]# etcdctl get test
a
[root@docker1 ~]#2.2?
安裝配置flannel
[root@docker1 ~]# yum -y install flannel啟動
[root@docker1 ~]# systemctl start flanneld報錯
[root@docker1 ~]# systemctl status flanneld -l
● flanneld.service - Flanneld overlay address etcd agent
?? Loaded: loaded (/usr/lib/systemd/system/flanneld.service; disabled; vendor preset: disabled)
?? Active: activating (start) since Thu 2018-06-14 02:22:26 EDT; 1min 1s ago
Main PID: 2950 (flanneld)
?? Memory: 16.6M
?? CGroup: /system.slice/flanneld.service
? ? ? ? ?? └─2950 /usr/bin/flanneld -etcd-endpoints=http://127.0.0.1:2379 -etcd-prefix=/atomic.io/network
Jun 14 02:23:18 docker1 flanneld-start[2950]: E0614 02:23:18.974351? ? 2950 network.go:102] failed to retrieve network config: 100: Key not found (/atomic.io) [11]
Jun 14 02:23:19 docker1 flanneld-start[2950]: E0614 02:23:19.977497? ? 2950 network.go:102] failed to retrieve network config: 100: Key not found (/atomic.io) [11]
Jun 14 02:23:20 docker1 flanneld-start[2950]: E0614 02:23:20.980721? ? 2950 network.go:102] failed to retrieve network config: 100: Key not found (/atomic.io) [11]
Jun 14 02:23:21 docker1 flanneld-start[2950]: E0614 02:23:21.983553? ? 2950 network.go:102] failed to retrieve network config: 100: Key not found (/atomic.io) [11]
Jun 14 02:23:22 docker1 flanneld-start[2950]: E0614 02:23:22.988446? ? 2950 network.go:102] failed to retrieve network config: 100: Key not found (/atomic.io) [11]
Jun 14 02:23:23 docker1 flanneld-start[2950]: E0614 02:23:23.992106? ? 2950 network.go:102] failed to retrieve network config: 100: Key not found (/atomic.io) [11]
Jun 14 02:23:24 docker1 flanneld-start[2950]: E0614 02:23:24.994719? ? 2950 network.go:102] failed to retrieve network config: 100: Key not found (/atomic.io) [11]
Jun 14 02:23:25 docker1 flanneld-start[2950]: E0614 02:23:25.998629? ? 2950 network.go:102] failed to retrieve network config: 100: Key not found (/atomic.io) [11]
Jun 14 02:23:27 docker1 flanneld-start[2950]: E0614 02:23:27.002486? ? 2950 network.go:102] failed to retrieve network config: 100: Key not found (/atomic.io) [11]
Jun 14 02:23:28 docker1 flanneld-start[2950]: E0614 02:23:28.006185? ? 2950 network.go:102] failed to retrieve network config: 100: Key not found (/atomic.io) [11]注意-etcd-prefix=/automic.io/network
flanel讀取的網絡配置是這個文件，這個文件是在
[root@docker1 ~]# cat /usr/lib/systemd/system/flanneld.service
[Unit]
Description=Flanneld overlay address etcd agent
After=network.target
After=network-online.target
Wants=network-online.target
After=etcd.service
Before=docker.service
[Service]
Type=notify
EnvironmentFile=/etc/sysconfig/flanneld
EnvironmentFile=-/etc/sysconfig/docker-network
ExecStart=/usr/bin/flanneld-start $FLANNEL_OPTIONS
ExecStartPost=/usr/libexec/flannel/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker
Restart=on-failure
[Install]
WantedBy=multi-user.target
RequiredBy=docker.service[root@docker1 sysconfig]# cat flanneld
# Flanneld configuration options?
# etcd url location.? Point this to the server where etcd runs
FLANNEL_ETCD_ENDPOINTS="http://127.0.0.1:2379"
# etcd config key.? This is the configuration key that flannel queries
# For address range assignment
FLANNEL_ETCD_PREFIX="/atomic.io/network"
# Any additional options that you want to pass
#FLANNEL_OPTIONS=""注意：
FLANNEL_ETCD_PREFIX="/atomic.io/network"
這個FLANNEL_ETCD_PREFIX需要etcdctl手動去建立
[root@docker1 ~]# etcdctl mk /atomic.io/network/config ‘{"Network":"172.17.0.0/16", "SubnetMin": "172.17.1.0", "SubnetMax": "172.17.254.0", "Backend":{"Type":"vxlan"}}‘再啟動flannel，啟動正常
[root@docker1 ~]# systemctl start flanneld && systemctl enable flanneld
Created symlink from /etc/systemd/system/multi-user.target.wants/flanneld.service to /usr/lib/systemd/system/flanneld.service.
Created symlink from /etc/systemd/system/docker.service.requires/flanneld.service to /usr/lib/systemd/system/flanneld.service.
[root@docker1 ~]#[root@docker1 ~]# systemctl status flanneld
● flanneld.service - Flanneld overlay address etcd agent
?? Loaded: loaded (/usr/lib/systemd/system/flanneld.service; disabled; vendor preset: disabled)
?? Active: active (running) since Thu 2018-06-14 02:47:58 EDT; 11s ago
? Process: 3513 ExecStartPost=/usr/libexec/flannel/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker (code=exited, status=0/SUCCESS)
Main PID: 3475 (flanneld)
?? Memory: 18.5M
?? CGroup: /system.slice/flanneld.service
? ? ? ? ?? └─3475 /usr/bin/flanneld -etcd-endpoints=http://127.0.0.1:2379 -etcd-prefix=/atomic.io/network
Jun 14 02:47:52 docker1 flanneld-start[3475]: E0614 02:47:52.150129? ? 3475 network.go:102] failed to retrieve network co...) [14]
Jun 14 02:47:53 docker1 flanneld-start[3475]: E0614 02:47:53.152602? ? 3475 network.go:102] failed to retrieve network co...) [14]
Jun 14 02:47:54 docker1 flanneld-start[3475]: E0614 02:47:54.155402? ? 3475 network.go:102] failed to retrieve network co...) [14]
Jun 14 02:47:55 docker1 flanneld-start[3475]: E0614 02:47:55.158612? ? 3475 network.go:102] failed to retrieve network co...) [14]
Jun 14 02:47:56 docker1 flanneld-start[3475]: E0614 02:47:56.164481? ? 3475 network.go:102] failed to retrieve network co...) [14]
Jun 14 02:47:57 docker1 flanneld-start[3475]: E0614 02:47:57.168282? ? 3475 network.go:102] failed to retrieve network co...) [14]
Jun 14 02:47:58 docker1 flanneld-start[3475]: I0614 02:47:58.179298? ? 3475 local_manager.go:179] Picking subnet in range....254.0
Jun 14 02:47:58 docker1 flanneld-start[3475]: I0614 02:47:58.261220? ? 3475 manager.go:250] Lease acquired: 172.17.21.0/24
Jun 14 02:47:58 docker1 flanneld-start[3475]: I0614 02:47:58.261993? ? 3475 network.go:98] Watching for new subnet leases
Jun 14 02:47:58 docker1 systemd[1]: Started Flanneld overlay address etcd agent.
Hint: Some lines were ellipsized, use -l to show in full.看看這個腳本
Process: 3513 ExecStartPost=/usr/libexec/flannel/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker (code=exited, status=0/SUCCESS)
flannel_env="/run/flannel/subnet.env"
docker_env="/run/docker_opts.env"
combined_opts_key="DOCKER_OPTS"
indiv_opts=false
combined_opts=false
ipmasq=true檢查下文件內容，我感覺是根據這個文件來生成網段，不確認
[root@docker1 flannel]# cat /run/flannel/subnet.env
FLANNEL_NETWORK=172.17.0.0/16
FLANNEL_SUBNET=172.17.21.1/24
FLANNEL_MTU=1472
FLANNEL_IPMASQ=false看看ip段
[root@docker1 ~]# ip a |grep flannel
11: flannel0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1472 qdisc pfifo_fast state UNKNOWN qlen 500
? ? inet 172.17.21.0/16 scope global flannel0
[root@docker1 ~]#以上都是docker1上的操作
2.3?
docker2,docker3上的操作是一樣的，我記錄docker2上的操作
[root@docker2 ~]# yum -y install flannel啟動flannel
[root@docker2 ~]# flanneld -etcd-endpoints=http://192.168.211.140:2379 -iface=ens33 -etcd-prefix=/atomic.io/network
I0614 04:28:55.785204? ? 2767 main.go:132] Installing signal handlers
I0614 04:28:55.785764? ? 2767 manager.go:149] Using interface with name ens33 and address 192.168.211.154
I0614 04:28:55.785784? ? 2767 manager.go:166] Defaulting external address to interface address (192.168.211.154)
E0614 04:28:55.786742? ? 2767 network.go:102] failed to retrieve network config: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.211.140:2379: getsockopt: no route to host
E0614 04:28:57.788671? ? 2767 network.go:102] failed to retrieve network config: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.211.140:2379: i/o timeout
E0614 04:28:59.791359? ? 2767 network.go:102] failed to retrieve network config: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.211.140:2379: i/o timeout報錯了
這個錯誤是因為etcd默認只監聽本機的2379端口
[root@docker1 ~]# cat /etc/etcd/etcd.conf
#[Member]
#ETCD_CORS=""
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_WAL_DIR=""
#ETCD_LISTEN_PEER_URLS="http://localhost:2380"
ETCD_LISTEN_CLIENT_URLS="http://localhost:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
ETCD_NAME="default"
#ETCD_SNAPSHOT_COUNT="100000"把ETCD_LISTEN_CLIENT_URLS="http://localhost:2379"改成ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
重新啟動etcd
[root@docker1 ~]# systemctl restart etcd
[root@docker1 ~]# systemctl status etcd
● etcd.service - Etcd Server
?? Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled)
?? Active: active (running) since Thu 2018-06-14 04:38:49 EDT; 1min 11s ago
Main PID: 3401 (etcd)
?? Memory: 21.5M
?? CGroup: /system.slice/etcd.service
? ? ? ? ?? └─3401 /usr/bin/etcd --name=default --data-dir=/var/lib/etcd/default.etcd --listen-client-urls=http://0.0.0.0:23...
Jun 14 04:38:48 docker1 etcd[3401]: enabled capabilities for version 3.2
Jun 14 04:38:49 docker1 etcd[3401]: 8e9e05c52164694d is starting a new election at term 9
Jun 14 04:38:49 docker1 etcd[3401]: 8e9e05c52164694d became candidate at term 10
Jun 14 04:38:49 docker1 etcd[3401]: 8e9e05c52164694d received MsgVoteResp from 8e9e05c52164694d at term 10
Jun 14 04:38:49 docker1 etcd[3401]: 8e9e05c52164694d became leader at term 10
Jun 14 04:38:49 docker1 etcd[3401]: raft.node: 8e9e05c52164694d elected leader 8e9e05c52164694d at term 10
Jun 14 04:38:49 docker1 etcd[3401]: published {Name:default ClientURLs:[http://192.168.211.140:2379]} to cluster cdf8...3a8c32
Jun 14 04:38:49 docker1 etcd[3401]: ready to serve client requests
Jun 14 04:38:49 docker1 systemd[1]: Started Etcd Server.
Jun 14 04:38:49 docker1 etcd[3401]: serving insecure client requests on [::]:2379, this is strongly discouraged!
Hint: Some lines were ellipsized, use -l to show in full.
[root@docker1 ~]#再啟動還是報錯?
[root@docker2 ~]# systemctl status flanneld -l
● flanneld.service - Flanneld overlay address etcd agent
?? Loaded: loaded (/usr/lib/systemd/system/flanneld.service; disabled; vendor preset: disabled)
?? Active: inactive (dead)
Jun 14 04:21:53 docker2 flanneld-start[2706]: E0614 04:21:53.879476? ? 2706 network.go:102] failed to retrieve network config: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:2379: getsockopt: connection refused
Jun 14 04:21:54 docker2 flanneld-start[2706]: E0614 04:21:54.880962? ? 2706 network.go:102] failed to retrieve network config: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:2379: getsockopt: connection refused
Jun 14 04:21:55 docker2 flanneld-start[2706]: E0614 04:21:55.882332? ? 2706 network.go:102] failed to retrieve network config: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:2379: getsockopt: connection refused
Jun 14 04:21:56 docker2 flanneld-start[2706]: E0614 04:21:56.887002? ? 2706 network.go:102] failed to retrieve network config: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:2379: getsockopt: connection refused
Jun 14 04:21:57 docker2 flanneld-start[2706]: E0614 04:21:57.888246? ? 2706 network.go:102] failed to retrieve network config: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:2379: getsockopt: connection refused
Jun 14 04:21:58 docker2 flanneld-start[2706]: E0614 04:21:58.889903? ? 2706 network.go:102] failed to retrieve network config: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:2379: getsockopt: connection refused
Jun 14 04:21:59 docker2 flanneld-start[2706]: E0614 04:21:59.891323? ? 2706 network.go:102] failed to retrieve network config: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:2379: getsockopt: connection refused
Jun 14 04:22:00 docker2 flanneld-start[2706]: E0614 04:22:00.892229? ? 2706 network.go:102] failed to retrieve network config: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:2379: getsockopt: connection refused
Jun 14 04:22:01 docker2 systemd[1]: Stopped Flanneld overlay address etcd agent.
Jun 14 04:22:01 docker2 flanneld-start[2706]: I0614 04:22:01.105679? ? 2706 main.go:172] Exiting...
[root@docker2 ~]#拒絕連接，應該是防火墻的問題了
關閉docker1的防火墻
[root@docker2 ~]# flanneld -etcd-endpoints=http://192.168.211.140:2379 -iface=ens33 -etcd-prefix=/atomic.io/network &
[1] 2938
[root@docker2 ~]# I0614 04:44:03.522494? ? 2938 main.go:132] Installing signal handlers
I0614 04:44:03.523151? ? 2938 manager.go:149] Using interface with name ens33 and address 192.168.211.154
I0614 04:44:03.523174? ? 2938 manager.go:166] Defaulting external address to interface address (192.168.211.154)
I0614 04:44:03.530498? ? 2938 local_manager.go:134] Found lease (172.17.41.0/24) for current IP (192.168.211.154), reusing
I0614 04:44:03.546625? ? 2938 manager.go:250] Lease acquired: 172.17.41.0/24
I0614 04:44:03.547228? ? 2938 network.go:98] Watching for new subnet leases
I0614 04:44:03.558669? ? 2938 network.go:191] Subnet added: 172.17.21.0/24
[root@docker2 ~]# ip a |grep flannel
8: flannel0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1472 qdisc pfifo_fast state UNKNOWN group default qlen 500
? ? inet 172.17.41.0/16 scope global flannel0
[root@docker2 ~]#啟動了
有點奇怪，重啟docker1后，可以進來了，不需要添加開放2379的規則
2.4?
docker3做一樣的操作
3.分析flannel網絡
?
3.1
上面把基本架構部署好了，具體如下:docker1安裝了etcd，docker1 docker2 docker3都安裝了flannel
在docker1上查看設置分配的網段和已經分配的網段
設置分配的網段
[root@docker1 ~]#? etcdctl get atomic.io/network/config
{"Network":"172.17.0.0/16", "SubnetMin": "172.17.1.0", "SubnetMax": "172.17.254.0", "Backend":{"Type":"vxlan"}}已經分配的網段
[root@docker1 ~]# etcdctl ls atomic.io/network/subnets
/atomic.io/network/subnets/172.17.21.0-24
/atomic.io/network/subnets/172.17.41.0-24
/atomic.io/network/subnets/172.17.95.0-24
[root@docker1 ~]#3.2
docker中使用flannel網絡
配置docker連接flannel，我這里用docker2和docker3
?
docker通過修改docker配置文件
/etc/systemd/system/docker.service
設置 --bip 和--mtu
連接flannel
--bip --mtu的值 ? 來自/run/flannel/subnet.env
[root@docker2 ~]#? cat /run/flannel/subnet.env
FLANNEL_NETWORK=172.17.0.0/16
FLANNEL_SUBNET=172.17.65.1/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=false--bip ?是 FLANNEL_SUBNET的值
--mtu 是 FLANNEL_MTU的值
在docker.service加上這兩個值
[root@docker2 ~]# cat /etc/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
After=network.target
[Service]
Type=notify
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver devicemapper --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=generic --bip=172.17.65.1/24 --mtu=1450
ExecReload=/bin/kill -s HUP
MountFlags=slave重啟docker
[root@docker2 ~]# systemctl daemon-reload
[root@docker2 ~]# systemctl restart docker.service3.2
簡易分析
docker連接上flannel后，網絡路由和bridge 情況，參考如下
[root@docker2 ~]# ip r
default via 192.168.211.2 dev ens33 proto dhcp metric 100
172.17.0.0/16 dev flannel.1
172.17.65.0/24 dev docker0 proto kernel scope link src 172.17.65.1
172.18.0.0/16 dev docker_gwbridge proto kernel scope link src 172.18.0.1
192.168.211.0/24 dev ens33 proto kernel scope link src 192.168.211.154 metric 100[root@docker2 ~]# brctl show
bridge name? ? ? ? bridge id? ? ? ? ? ? ? ? STP enabled? ? ? ? interfaces
docker0? ? ? ? ? ? ? ? 8000.02427f17e635? ? ? ? no? ? ? ? ? ? ? ? veth7680282
docker_gwbridge? ? ? ? ? ? ? ? 8000.024266f01344? ? ? ? no? ? ? ? ? ? ? ?
[root@docker2 ~]#沒有生成新的網橋，使用默認的docker0
172.17.65.0 同一docker主機 容器通過docker0連接
172.17.0.0 ? 不同docker主機 容器通過flannel.1轉發
3.3?
容器連接flannel網絡
[root@docker2 ~]# docker run -itd --name bbox1 busybox
dc344e4b30e48bbc5b914edc3724e43d56fa0ee7abf97246dc35ce57b6cf872c
[root@docker2 ~]# docker exec bbox1 ip r
default via 172.17.65.1 dev eth0
172.17.65.0/24 dev eth0 scope link? src 172.17.65.2
[root@docker2 ~]#
[root@docker3 ~]# docker run -itd --name bbox2 busybox
e9e824f93e8dedb498f436b169ed8dcb85bc951f4d05ae4f254aad1e3d538a8a
[root@docker3 ~]# docker exec bbox2 ip r
default via 172.17.16.1 dev eth0
172.17.16.0/24 dev eth0 scope link? src 172.17.16.2
[root@docker3 ~]#互ping
[root@docker2 ~]# docker exec bbox1 ping -c 5 172.17.16.2
PING 172.17.16.2 (172.17.16.2): 56 data bytes
64 bytes from 172.17.16.2: seq=0 ttl=60 time=3.274 ms
64 bytes from 172.17.16.2: seq=1 ttl=60 time=1.356 ms
^C
[root@docker2 ~]#
?[root@docker3 ~]# docker exec bbox2 ping -c 5 172.17.65.2
PING 172.17.65.2 (172.17.65.2): 56 data bytes
64 bytes from 172.17.65.2: seq=0 ttl=60 time=2.995 ms
64 bytes from 172.17.65.2: seq=1 ttl=60 time=1.396 ms
64 bytes from 172.17.65.2: seq=2 ttl=60 time=1.650 ms
^C
[root@docker3 ~]#分析數據流
從bbox1 ping 172.17.16.2
看看它的默認路由
[root@docker2 ~]# docker exec bbox1 ip r
default via 172.17.65.1 dev eth0
172.17.65.0/24 dev eth0 scope link? src 172.17.65.2
[root@docker2 ~]#目的地址 172.17.16.2不在直連網絡，因此數據包從default路由出。default路由的地址時 172.17.65.1，這個地址就是docker0的地址
[root@docker2 ~]# ip a |grep docker0
4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
? ? inet 172.17.65.1/24 brd 172.17.65.255 scope global docker0
24: veth63d6978@if23: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master docker0 state UP group default
[root@docker2 ~]#數據到達docker0后，發現這個數據包的地址是172.17.16.2，并不是給自己，尋找下一跳
看看node上的路由表
[root@docker2 ~]# ip r
default via 192.168.211.2 dev ens33 proto dhcp metric 100
172.17.0.0/16 dev flannel.1
172.17.65.0/24 dev docker0 proto kernel scope link src 172.17.65.1
172.18.0.0/16 dev docker_gwbridge proto kernel scope link src 172.18.0.1
192.168.211.0/24 dev ens33 proto kernel scope link src 192.168.211.154 metric 100
[root@docker2 ~]#匹配到172.17.0.0/16這條路由，這是直連路由，數據送到flannel.1上
flannel.1收到數據包，自己不是目的地，需要發數據發送出去，數據包沿著網絡協議向下流動，在二層封裝以太包，填寫目的mac地址。
這時發出arp“who is 172.17.16.2"
flannel.1是vxlan設備，vxlan并不會在二層發arp包，而是由linux kernel的 “L3 MISS"事件，將arp發到用戶空間和flanneld程序。
linux kernel的 “L3 MISS"事件參數由下面的參數設置
[root@docker2 ~]# cat /proc/sys/net/ipv4/neigh/flannel.1/app_solicit
3
[root@docker2 ~]#
flanneld程序收到“L3 MISS”內核事件以及ARP請求后，并不會向外網發送arp request，而是從etcd查找匹配該地址的子網的vtep信息。
[root@docker2 ~]#? curl -L http://192.168.211.140:2379/v2/keys/atomic.io/network/subnets/172.17.16.0-24
{"action":"get","node":{"key":"/atomic.io/network/subnets/172.17.16.0-24","value":"{\"PublicIP\":\"192.168.211.153\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"d2:72:c5:90:ff:8c\"}}","expiration":"2018-06-20T06:49:24.673562899Z","ttl":83016,"modifiedIndex":98,"createdIndex":98}}
[root@docker2 ~]#flanneld從etcd中找到答案
subnets:172.17.16.0-24
PublicIP:192.168.211.153
VtepMAC:d2:72:c5:90:ff:8cVtepMAC:d2:72:c5:90:ff:8c 這個地址誰呢？
到192.168.211.153也就是這里的docker3檢查下
[root@docker3 ~]# ip -d link show
11: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN mode DEFAULT group default
? ? link/ether d2:72:c5:90:ff:8c brd ff:ff:ff:ff:ff:ff promiscuity 0
? ? vxlan id 1 local 192.168.211.153 dev ens33 srcport 0 0 dstport 8472 nolearning ageing 300 noudpcsum noudp6zerocsumtx noudp6zerocsumrx?可以看到docker3的flannel.1的mac地址就是
找到目的地后，flanneld將查詢到的信息存入arp緩存
[root@docker2 ~]# ip n
192.168.211.254 dev ens33 lladdr 00:50:56:ea:e1:f4 STALE
172.17.16.2 dev flannel.1 lladdr d2:72:c5:90:ff:8c STALE
192.168.211.2 dev ens33 lladdr 00:50:56:fd:b3:89 STALE
192.168.211.153 dev ens33 lladdr 00:0c:29:93:2c:89 STALE
192.168.211.1 dev ens33 lladdr 00:50:56:c0:00:08 DELAY
172.17.65.2 dev docker0 lladdr 02:42:ac:11:41:02 STALE
192.168.211.140 dev ens33 lladdr 00:0c:29:f9:b7:d2 DELAY
[root@docker2 ~]#最后封裝vxlan包，發送到目的地
4.
flannel 為每個主機分配了獨立的 subnet但 flannel.1 將這些 subnet 連接起來了相互之間可以路由。本質上flannel 將各主機上相互獨立的 docker0 容器網絡組成了一個互通的大網絡實現了容器跨主機通信。flannel 沒有提供隔離。
5.
host-gw
flannel支持多種backend,host-gw是flannel的另一種backend.
與vxlan不同,host-gw不會封裝數據包.而是在主機路由表中創建到其他主機subnet的路由條目,實現容器跨主機通訊.
設置backend
修改下前面設置的,把backend改成host-gw
[root@docker1 ~]# etcdctl set atomic.io/network/config ‘{"Network":"172.17.0.0/16", "SubnetMin": "172.17.1.0", "SubnetMax": "172.17.254.0", "Backend":{"Type":"host-gw"}}‘
{"Network":"172.17.0.0/16", "SubnetMin": "172.17.1.0", "SubnetMax": "172.17.254.0", "Backend":{"Type":"host-gw"}}
[root@docker1 ~]# etcdctl get atomic.io/network/config
{"Network":"172.17.0.0/16", "SubnetMin": "172.17.1.0", "SubnetMax": "172.17.254.0", "Backend":{"Type":"host-gw"}}
[root@docker1 ~]#在docker2,docker3重新啟動flanneld進程.
[root@docker2 ~]# flanneld -etcd-endpoints=http://192.168.211.140:2379 -iface=ens33 -etcd-prefix=/atomic.io/network &檢查下路由表
[root@docker2 ~]# ip r
default via 192.168.211.2 dev ens33 proto dhcp metric 100
169.254.0.0/16 dev ens33 scope link metric 1002
172.17.16.0/24 via 192.168.211.153 dev ens33
172.17.65.0/24 dev docker0 proto kernel scope link src 172.17.65.1
172.18.0.0/16 dev docker_gwbridge proto kernel scope link src 172.18.0.1
192.168.211.0/24 dev ens33 proto kernel scope link src 192.168.211.154 metric 100
[root@docker2 ~]#
[root@docker3 ~]# ip r
default via 192.168.211.2 dev ens33 proto dhcp metric 100
169.254.0.0/16 dev ens33 scope link metric 1002
172.17.16.0/24 dev docker0 proto kernel scope link src 172.17.16.1
172.17.65.0/24 via 192.168.211.154 dev ens33
172.18.0.0/16 dev docker_gwbridge proto kernel scope link src 172.18.0.1
192.168.211.0/24 dev ens33 proto kernel scope link src 192.168.211.153 metric 100
[root@docker3 ~]#
172.17.16.0/24 via 192.168.211.153 dev ens33 ?docker2
172.17.65.0/24 via 192.168.211.154 dev ens33 ?docker3出現了相對應的路由條目
需要修改mtu并且重啟docker
mtu值前面已經有過說明，根據下面的值來修改
[root@docker3 ~]# cat /run/flannel/subnet.env
FLANNEL_NETWORK=172.17.0.0/16
FLANNEL_SUBNET=172.17.16.1/24
FLANNEL_MTU=1500
FLANNEL_IPMASQ=false
[root@docker3 ~]#
[root@docker3 ~]# sed -i ‘s/1450/1500/g‘? /etc/systemd/system/docker.service重啟docker
[root@docker3 ~]# systemctl daemon-reload
[root@docker3 ~]# systemctl restart docker
[root@docker3 ~]#通過容器數據走向分析下
[root@docker2 ~]# docker exec bbox1 ping 172.17.16.2
PING 172.17.16.2 (172.17.16.2): 56 data bytes
64 bytes from 172.17.16.2: seq=0 ttl=62 time=1.851 ms
64 bytes from 172.17.16.2: seq=1 ttl=62 time=0.946 ms
64 bytes from 172.17.16.2: seq=2 ttl=62 time=0.972 ms
^C
[root@docker2 ~]# docker exec bbox1 ip? r
default via 172.17.65.1 dev eth0
172.17.65.0/24 dev eth0 scope link? src 172.17.65.2
[root@docker2 ~]#走默認路由172.17.65.1
[root@docker2 ~]# ip a |grep docker0
4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group defaul
主站蜘蛛池模板：
亚洲免费一区二区av|
麻豆久久天天躁夜夜狠狠躁|
色狠狠综合天天综合综合|
国产成人亚洲精品狼色在线|
免费黄色大全一区二区三区|
国产稚嫩高中生呻吟激情在线视频|
99精品伊人久久久大香线蕉|
久久人人97超碰国产精品|
最新国产AV最新国产在钱|
国产美女高潮流白浆视频|
亚洲男女内射在线播放|
377P欧洲日本亚洲大胆|
免费无码黄十八禁网站|
337p粉嫩大胆色噜噜噜|
亚洲伊人久久综合成人|
亚洲精品国产一区二区三|
女高中生自慰污污网站|
北岛玲中文字幕人妻系列|
自拍日韩亚洲一区在线|
亚洲国产欧美在线看片一国产|
国产精品多p对白交换绿帽|
博客|
国产精品无码无卡在线播放|
亚洲AV永久纯肉无码精品动漫|
玩弄放荡人妻少妇系列|
国内偷自第一区二区三区|
国产精品美女久久久|
av激情亚洲男人的天堂|
我国产码在线观看av哈哈哈网站|
国产午夜亚洲精品国产成人|
五月综合网亚洲乱妇久久|
欧洲极品少妇|
精品国精品国产自在久国产应用男|
久久99九九精品久久久久蜜桃|
成人国产亚洲精品一区二|
中文字幕乱码一区二区免费|
亚洲理论在线A中文字幕|
国产人妻精品午夜福利免费|
国产品精品久久久久中文|
亚洲成人av在线综合|
亚洲AVAV天堂AV在线网阿V|