Requirements:

  • Alibaba Cloud cluster can resolve internal domain names
  • Office network resolves internal domain names + office network Internet resolution

Solution:

  • For the first problem, use Alibaba Cloud PrivateZone for resolution
  • For the second problem, configure the internal domain name zone in PrivateZone, and then synchronize it to the office network bind9 server through Alibaba Cloud synchronization tool; For the office network DNS resolution entry, use Dnsmasq to process, and directly forward the public network resolution to the public network DNS, and forward the internal domain name directly to bind9 for processing.

Some people may wonder why not use bind to directly implement all internal resolutions? The main reason here is that in actual use, it is found that bind9 has performance problems when forwarding multiple dns concurrently, and there will be occasional timeouts. This is relatively well done by dnsmasq.

1. Alibaba Cloud PrivateZone Configuration

Reference: https://help.aliyun.com/document_detail/64627.html

2. Synchronize Alibaba Cloud zone to bind9

1. Docker-compose to build bind9

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

version: '2'

services:
bind:
restart: always
image: sameersbn/bind:9.16.1-20200524
environment:
- ROOT_PASSWORD=DNS2021#
- WEBMIN_ENABLED=true
- WEBMIN_INIT_SSL_ENABLED=false
ports:
- "15353:53/tcp"
- "15353:53/udp"
- "11953:953/tcp"
- "10000:10000/tcp" #webmin management
volumes:
- ./data:/data networks: - bind9 networks: bind9: ipam: config: - subnet: 10.220.0.0/16 gateway: 10.220.0.1

2. Modify the bind configuration file

data/bind/etc/named.conf

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

include "/etc/bind/named.conf.options";
include "/etc/bind/named.conf.local";
include "/etc/bind/named.conf.default-zones";
key "rndc-key" {
    algorithm hmac-sha256;
    secret "tREasaE2Jal1GfwfL5iii3a88eRGKWui41l5h3v89OM=";
};
controls {
	inet 127.0.0.1 port 953 allow { 127.0.0.1; } keys { rndc-key; };
	};
logging {
    channel query_log {
        file "query.log" versions 10 size 50M;
        severity info;
        print-category yes;
        print-severity yes;
        print-time yes;
    };
    category queries {
        query_log;
    };
};

data/bind/etc/named.conf.options

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

options {
	directory "/var/cache/bind";
    dnssec-validation no;
    dnssec-enable no;
    recursion yes;
    allow-recursion { any;};
    allow-transfer { any; };
    allow-query-cache { any; };
    listen-on-v6 { any; };
    listen-on port 53 { any; };
    forward first;
    forwarders {
        192.168.1.211 port 53;
        192.168.1.212 port 53;
        };
    transfer-format many-answers;
    transfers-per-ns 500;
    recursive-clients 100000;
    max-transfer-time-in 5;
    transfers-in 300;
    transfers-out 300;
    querylog yes;
};

data/bind/etc/named.conf.local

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

zone "sd.com" {
	type master;
	file "/etc/bind/zones/sd.com.zone";
    allow-update { 127.0.0.1; };
	};
zone "bgt.sdi" {
	type master;
	file "/etc/bind/zones/bgt.sdi.zone";
    allow-update { 127.0.0.1; };
	};
zone "con.sdi" {
	type master;
	file "/etc/bind/zones/con.sdi.zone";
    allow-update { 127.0.0.1; };
	};
  1. Synchronize domain name zone configuration to bind9

Reference: https://help.aliyun.com/document_detail/102718.html

Write to the shell here, write the task plan, and execute batch synchronization.

1

*/5 * * * * /bin/bash /opt/bind9/update.sh

update.sh

1
2
3
4
5

#!/bin/bash
cd /opt/bind9/tools
./Zone_file_sync config.json
chown 101.101 /opt/bind9/data/bind/etc/zones/*
docker exec bind9_bind_1 bash -c "rndc -c /etc/bind/rndc.conf freeze;rndc -c /etc/bind/rndc.conf reload;rndc -c /etc/bind/rndc.conf thaw"

Configuration file tools/config.json

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

{
  "accessKeyId": "LIDD5ssssmGExzGsdfsY6sJrSqo",
  "accessKeySecret": "C5N1TTESt74KhSTsswSSSWiz2",
  "zone": [
    {
      "zoneName": "sd.com",
      "zoneId": "2a4dc4e0sdsfa5d36a3b88ab6482saf",
      "filePath": "/opt/bind9/data/bind/etc/zones/sd.com.zone"
    },
    {
      "zoneName": "bgt.sdi",
      "zoneId": "f842ca07ccsd6f35d9e294d55a0c900",
      "filePath": "/opt/bind9/data/bind/etc/zones/bgt.sdi.zone"
    },
    {
      "zoneName": "con.sdi",
      "zoneId": "beb4d911addsf2bd86425ds280e7bbf2",
      "filePath": "/opt/bind9/data/bind/etc/zones/con.sdi.zone"
    }
  ]
}

3. dnsmasq deployment

1. Install and configure dnsmasq

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

# yum -y install dnsmasq
# cat >> /etc/dnsmasq.conf << Tag
port=53
proxy-dnssec
no-hosts #Do not load local /etc/hosts
no-negcache
dns-forward-max=2000
server=114.114.114.114 #Specify upstream dns server
server=223.5.5.5 #Specify upstream dns server
server=/sd.com/127.0.0.1#15353 #Forward to the specified dns port
server=/bgt.sdi/127.0.0.1#15353
server=/con.sdi/127.0.0.1#15353
log-queries #Record dns query log
log-facility=/var/log/dnsmasq/dnsmasq.log #Specify log path
log-async=50
cache-size=100000
Tag
# systemctl start dnsmasq
# systemctl enable dnsmasq

2. Configure dnsmasq log rotation strategy:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

#cat >> /etc/logrotate.d/dnsmasq Tag
/var/log/dnsmasq/dnsmasq.log {
notifempty
daily
dateext
rotate 15
sharedscripts
postrotate
[ ! -f /var/run/dnsmasq.pid ] || kill -USR2 `cat /var/run/dnsmasq.pid`
endscript
}
Tag

Test execution

1

logrotate -vf /etc/logrotate.conf

4. What operations need to be performed to add a new zone?

The following changes are required for bind and dnsmasq:

For example: add test.com

  1. Add server=/test.com/127.0.0.1#15353 to /etc/dnsmasq.conf

  2. Add the following to /opt/bind9/data/bind/etc/named.conf.local:

1
2
3
4
5
6

zone "test.com" {
type master;
file "/etc/bind/zones/test.com.zone";
allow-update { 127.0.0.1; };
forwarders {};
};

5. How to access internal DNS within the k8s cluster

  1. Modify the cluster node /etc/resolv.conf to internal DNS
1
2
3

options timeout:1 attempts:1 rotate
nameserver 192.168.1.211
nameserver 192.168.1.212
  1. For unresolved files, coredns will forward them directly
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

Corefile: |
.:53 {
errors
health
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
upstream
fallthrough in-addr.arpa ip6.arpa
}
prometheus :9153
forward . /etc/resolv.conf
cache 30
loop
reload
loadbalance
}

At this point, the entire internal DNS resolution is complete.