DNS & BIND 9

• Issue: The /var/named directory is missing and needs to be owned by the named user.
• Create the /var/named directory

% mkdir /var/named

• Adjust the permissions on the BIND 9 home directory

% chown named: /var/named

• The BIND 9 process should now successfully load the new configuration

% rndc reconfig

• Verify the delegation of your DNS servers with

  dig text.zoneNN.dnslab.org txt
  

• the classic DNS from 1983 has not been designed with security in mind
• Attack vector: DNS cache poisoning
• Attack vector: Men-in-the-Middle data spoofing
• Attack vector: changes to the client DNS resolver configuration
• Attacks of authoritative DNS servers
• DNSSEC can help

• The DNS security extension DNSSEC secures DNS data by augmenting the data with cryptographic signatures
  • The owner (administrator) creates a pair of private and public keys for each DNS zone (asymmetric crypto)
  • The owner/administrator signs all DNS data with the private/secret key
  • The recipient of the data (DNS resolver or client operating system or application) will verify (validate) the data
    • That the data has not been changed on the server nor during transit
    • That the data comes from the owner (the owner of the private key)

• DNSSEC creates a chain of trust between the parent zone and the child zone

• Applications and DNS resolvers can follow the chain of trust to a configured trust anchor to validate the DNS data
• We will look into DNSSEC in more details later in this course

DNSSEC keys can be generated with different crypto algorithms. Some of these algorithms are obsolete and deprecated, others are not (yet) widely supported by deployed DNS software to be useful

• Every additional bit increases the strength of a DNSSEC against brute-force attacks to break the key
• Double the RSA key size results in
  • Generating signatures is up to eight times more work
  • Validation of DNSSEC signatures inside an DNS resolver up to 4 times more work
  • Size of key and signature records increases and can create operational issues

• For organizational reasons, DNSSEC splits the chain of trust inside a DNS zone onto two different keys: the Key-Signing-Key (KSK) and the Zone-Signing-Key (ZSK)

• For historical reasons, BIND 9 generates two signatures over the DNSKEY record set:
  • one signature generated with the KSK (required)
  • one signature generated with every active ZSK (optional)
• This can generate larger than good DNSKEY record answer messages
• BIND 9 can be configured to only generate a signature using the KSK

  options {
    [...]
    dnssec-dnskey-kskonly yes;
  };
  

• DNSSEC keys have an organisational lifetime (there is no technical limit on the lifetime, unlike with X.509 TLS certificates)
  • the administrator responsible for a DNSSEC-signed zone can decide when to change the DNSSEC keys
• Renewing the DNSSEC key material of a zone is called a key rollover
• Keys with weak algorithms or short key lengths should be changed (rolled) in shorter intervals
  • 1024bit RSASHA256 -> 30 days (ZSK)
  • 1536bit RSASHA256 -> 120 days (ZSK)
  • 2048bit RSASHA256 -> 360 days (KSK)
  • 2560bit RSASHA256 -> 720 days+ / 2 years+ (KSK)

• Zones with high security requirements should keep the DNSSEC keys "offline"
  • this requires manual DNSSEC signing
• Keys can be stored securely in Hardware Security Modules (HSM)
• HSM selection criteria for DNSSEC signing
  • Number of key storage slots
  • Timely support for new operating system releases (Linux distribution)
  • Timely support for new OpenSSL releases
  • Stability of the HSM-Driver
• Zones with medium to low security requirements should use a hidden-primary DNSSEC signer configuration
  • The DNSSEC signing authoritative server is not exposed to the Internet, it will not receive DNS queries
  • DNS secondary authoritative servers in the Internet will receive the DNSSEC signed zone from the hidden DNSSEC signer through DNS zone transfer
  • The DNSSEC key material is secured with operating system level permissions
  • With full automation of DNSSEC key-rollovers, the DNS server administrators don't need access to the DNSSEC keys
  • Login and access to the DNSSEC key files should be audited (Logging online and towards a remote log server)
• Zone content can be split across multiple zones or DNS server with DNS delegation
  • Every zone can have a different DNSSEC security level
  • Example: main zone example.com signed via a hidden-signer (keys not exposed to the Internet) has a sub-zone with e-mail address hashes (OPENPGPKEY and SMIMEA) in the zone securemail.example.com) which is secured with NSEC3-NARROW scheme and keys that are online on every authoritative server

• Since BIND 9.6 zones can be signed manually (BIND 9.6 was the first version to support the new DNSSEC version)
• Benefits:
  • The signing can be done offline on a machine not connected to any network, BIND 9 does not need to be running on the signing machine
  • The DNSSEC private keys can be stored on security devices such as Hardware Security Modules (HSM) and can be encrypted
  • The generated DNSSEC signed zone files are universal and can be used in any DNS server that supports DNSSEC signed zones
  • The parameters of the signing process can be tuned

• Drawbacks:
  • Manually signed zones need to be refreshed (re-signed) periodically so that the signatures do not expire
  • Any automation must be done through custom scripts

• These are the steps to manually sign a zone for BIND 9
• For older BIND 9 versions, enable DNSSEC in the named.conf configuration file

  options {
     directory "/var/named";
  };
  

• Create a Zone-Signing-Key (ZSK)
  • This requires good real entropy (randomness) in the operating system. If the key creation process takes too long (more than a minute for a key), there might not be enough entropy in the system. In such case a hardware random number generator or a software entropy gathering daemon (such as haveged) can help

  % dnssec-keygen -a RSASHA256 -b 1536 zoneNN.dnslab.org
  Generating key pair.........++++ ............................
  KzoneNN.dnslab.org.+008+16239
  
  % more KzoneNN.dnslab.org.+008+16239.key
  ; This is a zone-signing key, keyid 16239, for zoneNN.dnslab.org.
  ; Created: 20160202121320 (Tue Feb  2 13:13:20 2016)
  ; Publish: 20160202121320 (Tue Feb  2 13:13:20 2016)
  ; Activate: 20160202121320 (Tue Feb  2 13:13:20 2016)
  zoneNN.dnslab.org. IN DNSKEY 256 3 8 AwEAAc1xFtt40wPEx4TVB7h8Ac7HvMZuF1LIqESU/0HUUzDT2rkujMdL
  z0fgJJQVStYIbb1fXN0/PmKayEpj5ScbT7WU9Bef6b49uG1PwhsaftRr
  /udr3DEA6MTEdRqkl8K+E3P9hFj4XKxus45MYVSPaXZg3TcIQK3xpXC8
  sKISny43cQaJpm12oBtKsANlA25KRJC8soP1s/GqLSnArWDMN/YGqvs0
  QECulpm2Nh1uULZfzwga8515xizyx5yAl/sgWQ==
  

• Generate a Key-Signing-Key with the same DNSSEC algorithm used for the ZSK

  % dnssec-keygen -a RSASHA256 -b 2048 -f KSK zoneNN.dnslab.org
  Generating key pair...................................................
  KzoneNN.dnslab.org.+008+04351
  
  % more KzoneNN.dnslab.org.+008+04351.key
  ; This is a key-signing key, keyid 4351, for zoneNN.dnslab.org.
  ; Created: 20160202121714 (Tue Feb  2 13:17:14 2016)
  ; Publish: 20160202121714 (Tue Feb  2 13:17:14 2016)
  ; Activate: 20160202121714 (Tue Feb  2 13:17:14 2016)
  zoneNN.dnslab.org. IN DNSKEY 257 3 8 AwEAAcmn/QkiCne922gBBBuJJOnq9jnG2yYbB10zBS2SgUCUxlZfM2ja
  PAyubB2V+QhFsKf0VKUsVGl28JWAMcG1NGitj+nna4sGwvmeumj70DbG
  ZzynwcFknEZG1Swn2bM/OFmlMS2WV3luzDYKnLeZgvN5geB6ZetONlpP
  H9am3MRmExNIxoFb5NEcUlCzxSUI5GzjPZtGmCtDoNKrGE5nsssCgrjw
  ec6hbeXLOjP9JiQ3egF3+PJHLUOjuqXKwSofHw4jV4Rqc3eP+uAHk5Wp
  iH/BNW7c7lJ9IP+jZYZ3dp3SkO2qU8BOVV4fcm1L+IVcA9jwuuPaOV53
  j9L8fCTL/Uk=
  

• This is the content of the unsigned DNS zone

  $TTL 3600
  $ORIGIN zoneNN.dnslab.org.
  @             IN SOA serverNN.dnslab.org.  hostmaster.zoneNN.dnslab.org. (
                       1001 ; serial
                       1d   ; refreh
                       2h   ; retry
                       4w   ; expire
                       30m  ; negTTL
                    )
                IN NS serverNN.dnslab.org.
                IN MX 10 mail.zoneNN.dnslab.org.
  www           IN A  192.168.53.199
  mail          IN A  192.168.53.199
  

• Add the public part of the ZSK and KSK to the zone (be careful with the shell redirection!) or use master file $INCLUDE syntax.

  % cat KzoneNN.dnslab.org.+008+*.key >> zoneNN.dnslab.org
  

• Sign the zone file

  % dnssec-signzone -o zoneNN.dnslab.org \
          -k KzoneNN.dnslab.org.+008+04351.private \
          zoneNN.dnslab.org \
  	 KzoneNN.dnslab.org.+008+16239.private
  Verifying the zone using the following algorithms: RSASHA256.
  Zone fully signed:
  Algorithm: RSASHA256: KSKs: 1 active, 0 stand-by, 0 revoked
                        ZSKs: 1 active, 0 stand-by, 0 revoked
  zoneNN.dnslab.org.signed
  

• Zone signing tool syntax:

  dnssec-signzone -o <origin> -k <KSK-private> <zone-file> <ZSK-private>
  

  • If the error message dnssec-signzone: fatal: SOA is not signed (keys offline or inactive?) is displayed, then the KSK and ZSK have been given in the wrong order
  • Additional options for dnssec-signzone
    • -j sec Jitter, variation in seconds of the signature validity
    • -M maxttl - specifies the maximum allowed TTL in the signed zone. Higher TTL values will be capped to this number.
    • -s starttime - start of validity of the signatures
    • -e endtime - expire time of the signatures
    • -N SOA-format - format of the SOA serial number
      • keep do not modify the current SOA serial number
      • increment increment the current SOA by one
      • date set the SOA serial number to today's date in YYYYMMDDnn format
      • unixtime use the Unixtime (seconds since 1.1.1970) as the SOA serial
    • -x only create one signature for the DNSKEY record set (using the KSK)
    • -n numcpus use this amount of CPU cores for signing
    • -t print performance statistics after signing
• The signed zone is stored in a new file with the name of the original zone file and the extension .signed

  zoneNN.dnslab.org.      3600    IN SOA  serverNN.dnslab.org. hostmaster.zoneNN.dnslab.org. (
                                           1001       ; serial
                                           86400      ; refresh (1 day)
                                           7200       ; retry (2 hours)
                                           2419200    ; expire (4 weeks)
                                           1800       ; minimum (30 minutes)
                                           )
                           3600    RRSIG   SOA 8 3 3600 (
                                           20160303114542 20160202114542 16239 zoneNN.dnslab.org.
                                           jT+M9IhjViHkoLnC5/y+MaHYoxpcoyHvif7H
                                           sFESQfk7JBx654zb7OZ2LVxsmMnGtiqxkLlD
                                           l5nJtBJuWXMufPLks+qQb42YjdGgU6vf5WOI
                                           GkTyTMf0ZcNc3ULZZCMG/Vkf3Pak4O6He+cB
                                           xvdbDzOOaLcQqlKH8xY/ylmHawJTm8Mmcbb/
                                           1tL3B/Iv0SI9Lv+F/g+ajaA7fd3bcr0Vueol
                                           gOTJ3OZkIEoQFROrn5UMQ/fvbY7Go2TjDT7I
                                           GYMu )
                           3600    NS      serverNN.dnslab.org.
                           3600    RRSIG   NS 8 3 3600 (
                                           20160303114542 20160202114542 16239 zoneNN.dnslab.org.
                                           ColPExa9EdSA1Nt1DsEtX5qjYzNWA8vUl8ef
                                           oJG409V6BX4JJsK7RJGCGlmqDIA7HO1IQKug
                                           64N+fD/IuVjqHsPxc/YtP1sdnnLjYK0rHgG7
                                           kMMkoqU7Ta/l/laKKd3jcI/kh66dOsTwYrEC
                                           aykvfzYCnVj/rxwEomu2aTbPh/Nt7V2OEXDT
                                           EqlmpS34yQ3LvYfnSTTipLYZNS/2kL7NRuEo
                                           1gId5PD/IMAKSpTqfilW1bQUf+CYJF/CGgF5
                                           KRIx )
  

• Compare the file sizes. The signed zone is much larger

  % wc zoneNN.dnslab.org*
       23     133    1554 zoneNN.dnslab.org
      130     314    5178 zoneNN.dnslab.org.signed
      153     447    6732 total
  

• Adjust the zone definition in named.conf to load the new signed zone

  zone "zoneNN.dnslab.org" IN {
      type primary;
      file "zoneNN.dnslab.org.signed";
  };
  

• Check the new BIND 9 configuration. The output should indicate that the zone is now DNSSEC signed

  % named-checkconf -z
  zone zoneNN.dnslab.org/IN: loaded serial 1001 (DNSSEC signed)
  

• Load the DNSSEC signed zone into the BIND 9 DNS server

  % rndc reload zoneNN.dnslab.org
  

• Send the DS record for the KSK to the operator of the parent zone

  % cat dsset-zoneNN.dnslab.org.
  zoneNN.dnslab.org.  IN DS 16239 8 1 C0213 ... 0373A
  zoneNN.dnslab.org.  IN DS 16239 8 2 EAC59 ... E694690565680E3F6D201 E2650EAE
  

• Single CSK ("common signing key", also called a SSK - "single signing key")
  • ECDSAP256SHA256 (algo 13) with unlimited lifetime
  • RRSIG validity 14 days, refreshed 5 days before expiration
• NSEC
• Key timings:
  • DNSKEY TTL: 3600, max zone TTL: 86400 (1 day)
  • Key publish and retire safety times: 3600 (1 hour)
  • Propagation delay: 300 (5 minutes)
• Parent timings
  • DS TTL: 86400 (1 day)
  • Propagation delay: 3600 (1 hour)

• Prevent policy changes on upgrade by using an explicitly defined dnssec-policy, rather than default

• Below is an example of an custom DNSSEC policy in BIND 9.16+ named "one"

  dnssec-policy "one" {
      keys {
          ksk lifetime 365d algorithm rsasha256 4096;
          zsk lifetime 60d  algorithm rsasha256 1024;
      };
  };
  
  zone "example.net" {
      file "example.net.zone";
      dnssec-policy "one";
  };
  

  • keys specify roles instead of specific keys
  • lifetime specifies duration or unlimited
  • algorithm uses mnemonic or number

dnssec-policy "one" {
    keys {
        ksk lifetime 365d algorithm rsasha256 4096;
        zsk lifetime 60d  algorithm rsasha256 1024;
    };
    dnskey-ttl 600;
    publish-safety PT2H;
    signatures-refresh 7d;

    nsec3param iterations 0 optout no salt-length 0;
};

dnssec-policy <string> {
    dnskey-ttl <duration>;
    keys { ( csk | ksk | zsk ) [ ( key-directory ) ]
        lifetime <duration_or_unlimited> algorithm <string> [ <integer> ]; ...
    };
    max-zone-ttl <duration>;
    nsec3param [ iterations <integer> ] [ optout <boolean> ] [ salt-length <integer> ];
    parent-ds-ttl <duration>;
    parent-propagation-delay <duration>;
    publish-safety <duration>;
    purge-keys <duration>;
    retire-safety <duration>;
    signatures-refresh <duration>;
    signatures-validity <duration>;
    signatures-validity-dnskey <duration>;
    zone-propagation-delay <duration>;
};

• A KSK with ECDSAP256SHA256 and no automatic key rollover
• A ZSK with ECDSAP256SHA256 and a rollover interval of 30 days
• Signatures are valid for 10 days
• Signatures will be refreshed after 8 days (2 day buffer)
• TTL of DNSKEY records is 2 hours
• Max TTL of other records in the zone is 1 day
• DNSSEC keys will be published one hour after they have become valid (publish-savety)
• DNSSEC keys will be kept in the zone one our after deactivation (retire-safety)
• Expired DNSSEC keys will be removed after 90 days
• Zone transfer between primary and secondary servers is 5 minutes (zone-propagation-delay)

dnssec-policy "example.eu" {
       dnskey-ttl 2h;
       keys { ksk lifetime unlimited algorithm ECDSAP256SHA256;
              zsk lifetime P30D      algorithm ECDSAP256SHA256;
            };
       max-zone-ttl 1d;
       publish-safety 1h;
       purge-keys P90D;
       retire-safety 1h;
       signatures-refresh 8d;
       signatures-validity 10d;
       signatures-validity-dnskey 10d;
       zone-propagation-delay 300;
 };

• The RRSIG records in DNSSEC secure the positive answers from DNS (answers to queries where DNS records exist)
• But also negative answers need to be protected
  • without protection for negative answers, attackers could spoof negative answers to launch denial-of-service attacks. The attacker can make DNS clients believe that a DNS resource does not exist
• DNS knows two kinds of negative answers: NXDOMAIN and NODATA/NXRRSET
  • NXDOMAIN = the requested domain name does not exist
  • NODATA/NXRRSET = the requested domain name does exist, but the requested record type does not exist for this name
• DNS error messages such as SERVFAIL, FORMERR or REFUSED cannot be secured with DNSSEC (these errors indicate errors in the protocol or in the DNS server infrastructure). If the protocol is broken, DNSSEC can't work either.
• The solution: the NSEC records in the DNSSEC signed zone create a list of all existing data in the zone (domain names and records types for the domain names)
  • When returning a negative answer, the DNS server returns the negative answer containing the NSEC record (plus a signature for the NSEC record) which proves that the requested data does not exist in DNS

• The NSEC record is an elegant solution to the problem, but it has drawbacks:
  • It is now possible for outsiders to list the complete zone contents by following the NSEC record chain. This is called zone walking.
  • For most zones this is not a big problem, as the DNS zone content is public anyway (SOA, NS records, WWW-A, MX records)
  • But it can be an issue for zones with sensitive content (email addresses, hostnames of critical infrastructure, new product names that should no go public yet)
  • Operators of TLDs zones stay away from DNSSEC with NSEC, as it allows outsiders to record all changes to the TLD zone (new delegations and delegation removals)
  • The new compact authenticated denial of existance standard (RFC 9824) solves this issue.
• Example of zone walking:

$ ldns-walk paypal.com

• RFC 5155 (2008) defines the NSEC3 record as an alternative to NSEC
  • All modern DNS servers support NSEC3
  • The NSEC3 record works similarly to NSEC, with the difference that the link between the owner names is done using SHA1 hashes of the domain names instead of the clear text names
  • NSEC3 makes it harder, but not impossible, to list the zone content

• The NSEC3 record contains a number of parameters used for the NSEC3 operation
  • The hashing algorithm used (currently only SHA1 is defined)
  • Flags: allows for DNSSEC opt-out in delegations.
  • Number of hash iterations
  • A salt value

• Every zone with NSEC3 records contains an NSEC3PARAM record. This record holds information needed by authoritative DNS servers to generate NSEC3 records for negative answers
• Example: (SHA1, no flags, 20 iterations, salt value "ABBACAFE")

nsec3.dnslab.org.    0   IN  NSEC3PARAM 1 0 20 ABBACAFE

• The idea of the hash iterations in the NSEC3PARAM record was to allow the operator of the zone to fine tune the work required for calculating the hash
  • A higher number of iterations should make it harder for attackers to brute force break an NSEC3 domain name
  • A higher number also creates more CPU load for DNS resolvers that validate the NSEC3 records, making DNS name resolution slower
  • The iteration parameter of an NSEC3 signed zone should be re-evaluated from time to time (yearly) to adjust the value for the technical progress
• The salt should make it impossible for an attacker to pre-calculate rainbow tables for the zone
• The salt is a hexadecimal number, every hex digit contains 4 bit of information

• The iterations had more an negative impact on the CPU load of the authoritative DNS servers than preventing attacks
• The salt is not really needed, as each zone naturally has unique hashes so that a rainbow table for multiple zones is not possible

• RFC 9276 - Guidance for NSEC3 Parameter Settings (Best current practice) gives updates guidelines for NSEC and NSEC3 deployments
• The iterations value should be set to 0 (meaning 1 iteration of SHA1 hashing)
  • DNSSEC responses containing NSEC3 records with iteration counts greater than 150 are now treated as insecure by major DNS resolvers!
• As NSEC3 zones are inherently salted, the salt parameter should be set to -
• The current recommendation for NSEC3PARAM is 1 0 0 - (SHA1 Hash, no flags, 1 iteration, no salt)

• Most DNS zone can work with NSEC instead of NSEC3
  • Don't store names in DNS that should be secret (internal names, product names etc)
  • DNS is a service to publish data, not for hiding data

• RFC 7129 (also RFC 4470 and RFC 4471) describe a special variant of NSEC3 usage, called the narrow mode
  • With narrow mode, the NSEC3 records are not pre-calculated when the zone is signed, but they are created on the fly whenever a negative response is needed
    • To be able to calculate the signatures for such answers, every authoritative DNS server for the zone must have access to the private DNSSEC keys (at least the ZSK)!
• With NSEC3 narrow mode, the DNS server does not return an NSEC3 chain of existing records, but a synthetic NSEC3 record that proves that the requested name does not exist
  • This prevents zone walking, as the number of possible NSEC3 records returned is near infinite. It will exhaust the memory of the attacker that will try to store this NSEC3 chain
• Known implementations:
  • Phreebird (Dan Kaminski, Proof-of-Concept, not maintained)
  • PowerDNS Authoritative
  • Cloudflare CDN NSEC3 Narrow-Mode (closed source implementation)

• The DNSSEC key material is public
  • Including the signatures and the matching clear text (DNS record sets)
• The private key can get in un-authorized hands
  • By accident
  • By attacks on the signing server/HSM
  • When using online-signing, the private keys must be live on the signing DNS server
• The key length or the algorithm used is not considered secure for a longer amount of time (for example 1024bit RSA keys)

• The DNS is not consistent
  • DNS zone data can differ for some time between authoritative server of the same zone (delay in zone transfer)
  • DNS data is cached in DNS resolvers, operating systems, and applications
• During a DNSSEC key-rollover, the chain of trust must be un-broken at all times from all vantage points of the Internet

• RFC 6781 - DNSSEC Operational Practices, Version 2
• RFC 7583 - DNSSEC Key Rollover Timing Considerations

• DNSSEC keys do not have a technical lifetime, they don't expire
• The operational life time of DNSSEC keys is decided by the responsible administrator(s) and can be changed at any time
• The DNSSEC community has different views on KSK key rollovers
  • Often and regularly
  • Often but irregular (to not give attackers information when the system might be more vulnerable due to the key rollover)
  • Only if there is evidence that the key has been compromised or stolen

• The Zone-Signing-Key has no dependencies to external resources (such as the parent zone)
• A ZSK Rollover can be started at any time
• The 'pre-publication' rollover scheme is used for ZSK rollover

• The KSK has a dependency on its DS record in the parent zone
• For the KSK rollover we will use the 'double-signing' rollover scheme (the DNSKEY record set will get two signatures, from both, old and new, KSK)

• An algorithm rollover is used when the DNSSEC key algorithm of the zone needs to be changed
  • e.g. when switching from RSASHA256 to ECDSASHA256
• During an algorithm rollover, the KSK and ZSK will be changed at the same time using a double-signing rollover scheme
  • During such a roll over, the zone should be monitored for failures and over sized DNS answer messages
  • Link: DNSSEC algorithm rollover HOWTO by Tony Finch (Univ. of Cambridge)

• In an emergency, time is at an essence
• To save time on an KSK rollover, step one (publication) can be done ahead of time
  • This published extra key is called the standby key
  • It saves time, but makes the DNSKEY RRSet larger
• The standby key is not used during normal DNSSEC signing operation
• If there is a need to change the KSK in an emergency, the zone can almost immediately switch over to the standby key
• The standby key should be replaced (rolled) whenever the production KSK is rolled

• a DNS infrastructure with DNSSEC signed zones is more fragile
  • more complex configuration
  • most errors are fatal, the zone cannot be resolved anymore (this is a security feature of DNSSEC!)
• DNSSEC monitoring helps to detect issues before the DNS service is affected

• We have compiled 15 essential monitoring test scripts
  • These scripts are simple (Bourne-) shell scripts that should work on any Unix/Linux system (and on Windows 10 with Linux-Sub-System or Windows with Cygwin)
  • The scripts are available in our Github repo at https://github.com/cstrotm/dns-monitoring-scripts
  • Please send pull-requests for fixes and additions
• The scripts are deliberately simple
• Each script takes one input parameter
  • The domain-name of a delegated zone
• The scripts can be used from a cron-job
  • or embedded into an monitoring system.

• Test 1 - UDPv4 reachability - test for each authoritative server of the DNS infrastructure

  $ dig -4 test-domain +nssearch
  

• Test 2 - UDPv6 reachability - test for each authoritative server of the DNS infrastructure that it is reachable over UDP IPv6

  $ dig -6 test-domain +nssearch
  

• Test 3 - TCPv4 reachability - test for each authoritative server of the DNS infrastructure that it is reachable over TCP IPv4

  $ dig -4 test-domain +tcp +nssearch
  

• Test 4 - TCPv6 reachability - test for each authoritative server of the DNS infrastructure that it is reachable over TCP IPv6

  $ dig -6 test-domain +tcp +nssearch
  

• Test 5 - EDNS0 response size: tests that the server signals the correct EDNS0 response size. Size needs to be checked against the local policy. Usually 1220-1232 bytes.

  echo " == #5 - EDNS0 response size == "
  
  err=0
  ednspolicy=1232
  
  dig NS ${1} +short | while read server; do
    echo "Server: ${server} "
    ednsbuf=$(dig @${server} ${1} | grep "; EDNS:" | cut -d " " -f 7)
  
    if [ "${ednsbuf}" -eq "${ednspolicy}" ]; then
      echo " EDNS0-Bufsize is ${ednsbuf}, good "
    else
      err=1
      echo " EDNS0-Bufsize is ${ednsbuf}, out of policy range "
    fi
  done
  
  exit ${err}
  

• Test 6 - test that all authoritative server for a zone respond. Count is tested against the number of delegation authoritative servers for the zone.

  echo " == #6 - IPv4 zone response tests == "
  
  err=0
  # get TLD for the zone
  tld=$(echo ${1} | rev | cut -d'.' -f 1 | rev)
  # pick one TLD auth server
  tldns=$(dig NS ${1}. +short | tail -1)
  # query and count the delegation NS records for the zone
  parentnsnum=$(dig @${tldns} NS ${1} +short | wc -l)
  # query the authoritative DNS servers for the zone
  childnsnum=$(dig -4 ${1} +nssearch | wc -l)
  
  if [ "${parentnsnum}" -eq "${childnsnum}" ]; then
    echo "all authoritative DNS-Server answer"
  else
    err=1
    echo "Error: Mismatch"
    echo "Auth DNS-Servers in Delegation: ${parentnsnum}"
    echo "Auth DNS-Servers in answering: ${childnsnum}"
  fi
  
  exit ${err}
  

• Test 7 - test that all authoritative server for a zone respond via TCP. Count should be tested against the known good number of authoritative servers for the zone. The return code of the dig command should be checked for errors.

  echo " == #7 - IPv4 (TCP) zone response tests == "
  
  err=0
  # get TLD for the zone
  tld=$(echo ${1} | rev | cut -d'.' -f 1 | rev)
  # pick one TLD auth server
  tldns=$(dig NS ${1}. +short | tail -1)
  # query and count the delegation NS records for the zone
  parentnsnum=$(dig @${tldns} NS ${1} +short | wc -l)
  # query the authoritative DNS servers for the zone
  childnsnum=$(dig -4 ${1} +nssearch +tcp | wc -l)
  
  if [ "${parentnsnum}" -eq "${childnsnum}" ]; then
    echo "all authoritative DNS-Server answer"
  else
    err=1
    echo "Error: Mismatch"
    echo "Auth DNS-Servers in Delegation: ${parentnsnum}"
    echo "Auth DNS-Servers in answering: ${childnsnum}"
  fi
  
  exit ${err}
  

• Test 8 - test that all authoritative server for a zone have the same SOA serial. The return code of the dig command should be checked for errors.

  dig zone +nssearch
  

  • the SOA serial can be different for short amount of times after an update on the primary (propagation delay during zone transfer)
  • on a test interval of 5 minutes the test should issue a warning if the same SOA difference is seen in two successive tests
  • is the same SOA difference seen after three or more tests, an event of severity ERROR should be generated
• Test 8 - test that all authoritative server for a zone have the same SOA serial. The return code of the dig command should be checked for errors.

  echo " == #8 - SOA serial == "
  
  err=0
  oldsoaserial="0"
  
  dig ${1} +nssearch | while read serverres; do
    soaserial=$(echo ${serverres} | cut -d ' ' -f 4)
  
    if [ "${oldsoaserial}" -eq "0" ]; then
      oldsoaserial=${soaserial}
    else
      if [ "${oldsoaserial}" -eq "${soaserial}" ]; then
       echo "Match for ${soaserial}"
      else
       err=1
       echo "Mismatch for ${soaserial} != ${oldsoaserial}"
      fi
    fi
  done
  
  exit ${err}
  

• Test 9 - test for AA-Flag. Repeat this test for each authoritative server for the zone. Each server must respond with an AA-Flag.

  echo " == #9 - AA-Flag == "
  
  err=0
  
  dig NS ${1} +short | while read server; do
    echo "Server: ${server} "
    aaflag=$(dig @${server} ${1} SOA +norec | grep ";; flags" | cut -d " " -f 4 | cut -b 1-2)
  
    if [ "${aaflag}" = "aa" ]; then
      echo " AA-Flag found, good "
    else
      err=1
      echo " no AA-Flag, Server not authoritative "
    fi
  done
  
  exit ${err}
  

• Test 10 - test for Parent-Child NS-RRset. Tests that the NS-RRset in the parent zone (delegation) matches the NS-RRset in the zone data.

  echo " == #10 - test for Parent-Child NS-RRset == "
  
  if [ "$1" = "" ]; then
    echo "This test fails without param. Exiting..."
    exit 1
  fi
  
  err=0
  # get one authoritative server for the zone
  child_dns=$(dig NS ${1} +short | tail -1)
  # get TLD of Domain
  tld=$(echo ${1} | rev | cut -d'.' -f 1 | rev)
  # get one authoritative server for the TLD
  tldns=$(dig NS ${tld}. +short | tail -1)
  # query the delegation records
  parns=$(dig @${tldns} NS ${1} +norec +noall +authority | grep "IN.*NS" | sort)
  
  while read nsrec; do
    ns=$(echo ${nsrec} | cut -d ' ' -f 5)
    parentns="${parentns} ${ns}"
  done <<EOF
  ${parns}
  EOF
  
  # query the zone records
  childns=$(dig @${child_dns} NS ${1} +short +norec | sort)
  parentns=$(echo ${parentns} | tr ' ' '\n' | sort)
  
  echo "Parent delegation:"
  echo ${parentns}
  echo "Child zonedata:"
  echo ${childns}
  
  if [ "${childns}" = "${parentns}" ]; then
    echo "Parent/Child NS-RRSet matches"
  else
    err=1
    echo "Parent/Child NS-RRSet mismatch"
  fi
  
  exit ${err}
  

• Test 11 - test for DNSKEY RRset answer size. The full answer packet of the DNSKEY rrset should be below the IPv6 fragmentation payload limit (1232 byte)

  echo " == #11 - test for DNSKEY RRset answer size == "
  
  err=0
  maxsize=1232
  replysize=$(dig ${1} DNSKEY +dnssec +cd | grep ";; MSG SIZE" | cut -d " " -f 6)
  
  if [ "${replysize}" -le "${maxsize}" ]; then
    echo "Good, DNSKEY RRSet size is ${replysize} which is below or equal to ${maxsize}"
  else
    err=1
    echo "Bad, DNSKEY RRSet size is ${replysize} which is above ${maxsize}"
  fi
  
  exit ${err}
  

• Test 12 - RRSIG validity: check for the lifetime timestamps of RRSIGs in the zone. This test should be done for every important RRset in the zone (SOA, DNSKEY, MX, A/AAAA)

  dig zone soa +dnssec | egrep "RRSIG.*SOA" | cut -d " " -f 6
  dig zone soa +dnssec | egrep "RRSIG.*SOA" | cut -d " " -f 5
  

• compare the output with the current system time date "+%Y%m%d%H%M%S"
  1. issue an ERROR event, if the inception time is in the future
  2. issue an ERROR event, if the expiry time is in the past
  3. issue an WARNING event, if the expiry time will be reached in less than 5 days
  4. issue an ERROR event, if the expiry time will be reached in less than 2 days

  echo " == #12 - RRSIG validity == "
  
  if [ "$1" = "" ]; then
    echo "This test fails without param. Exiting..."
    exit 1
  fi
  
  err=0
  today=$(date "+%Y%m%d%H%M%S")
  inception=$(dig ${1} SOA +cd +dnssec | egrep "RRSIG.*SOA" | cut -d " " -f 6)
  expiry=$(dig ${1} SOA +cd +dnssec | egrep "RRSIG.*SOA" | cut -d " " -f 5)
  
  echo "Today    : ${today}"
  echo "Inception: ${inception}"
  echo "Expiry   : ${expiry}"
  
  if [ "${inception}" -gt "${today}" ]; then
    err=1
    echo "ERROR: RRSIG validity (${inception}) is in the future"
  fi
  
  if [ "${expiry}" -lt "${today}" ]; then
    err=1
    echo "ERROR: RRSIG validity (${expiry}) is in the past, DNSSEC signature has expired"
  fi
  
  twodaysahead=$(date +%s)
  twodaysahead=$((${twodaysahead}+172800))
  twodaysahead=$(date -u --date="@${twodaysahead}" "+%Y%m%d%H%M%S")
  
  if [ "${expiry}" -lt "${twodaysahead}" ]; then
    err=1
    echo "ERROR: RRSIG validity (${expiry}) will end in less than two days"
  fi
  
  fivedaysahead=$(date +%s)
  fivedaysahead=$((${fivedaysahead}+432000))
  fivedaysahead=$(date -u --date="@${fivedaysahead}" "+%Y%m%d%H%M%S")
  
  if [ "${expiry}" -lt "${fivedaysahead}" ]; then
    err=1
    echo "WARNING: RRSIG validity (${expiry}) will end in less than five days"
  fi
  
  exit ${err}
  

• Test 13 - DS Records - test the number and the content of the DS records in the parent zone. Issue a warning when the count or the content changes

  echo " == #13 - DS Records == "
  
  err=0
  
  if [ -f "$0.$1.saved.dscontent" ]; then
    oldds=$(cat $0.$1.saved.dscontent)
    olddscount=$(cat $0.$1.saved.dscount)
  else
    echo "First run. This result won't be meaningful until the next run.";
  fi
  
  ds=$(dig ${1} DS +short)
  echo "${ds}" > $0.$1.saved.dscontent
  
  dscount=$(dig ${1} DS +short | wc -l)
  echo "${dscount}" > $0.$1.saved.dscount
  
  if [ "${ds}" != "${oldds}" ]; then
    err=1
    echo "Warning: DS-Record has changed!"
  else
    echo "OK: DS-Record is the same as last time tested!"
  fi
  
  if [ "${dscount}" != "${olddscount}" ]; then
    err=1
    echo "Warning: number of DS-Record has changed!"
  else
    echo "OK: number of DS-Record is the same as last time tested!"
  fi
  
  exit ${err}
  

• Test 14 - DS Records and KSK - test that the DS-Record matches the KSK in the zone. The two values (Key-ID) must match.

  echo " == #14 - DS Records and KSK == "
  
  if [ "$1" = "" ]; then
    echo "This test fails without param. Exiting..."
    exit 1
  fi
  
  err=0
  
  dskeyid=$(dig ${1} DS +short +cd | cut -d " " -f 1 | tail -1)
  rrsigkeyid=$(dig ${1} DNSKEY +dnssec +short +cd | egrep "^DNSKEY" | grep "${dskeyid}" | cut -d ' ' -f 7)
  
  if [ "${dskeyid}" != "${rrsigkeyid}" ]; then
    err=1
    echo "Error: Key-Tag of DS-Records does not match the Key-Tag of RRSIG on DNSKEY"
  else
    echo "OK: Key-Tag of DS-Records does match the Key-Tag of RRSIG on DNSKEY"
  fi
  
  exit ${err}
  

• Test 15 - Count of DNSKEY records in the zone. The number can change during a key-rollover. Any change should create an WARNING event

  echo " == #15 - DNSKEY record count == "
  
  if [ -f "$0.$1.saved.dnskeycount" ]; then
    olddnskeycount=$(cat $0.$1.saved.dnskeycount)
  else
    echo "First run. This result won't be meaningful until the next run.";
  fi
  
  err=0
  dnskeycount=$(dig ${1} DNSKEY +cd +dnssec | egrep "DNSKEY.*2" | grep -v "RRSIG" | wc -l)
  
  echo "${dnskeycount}" > $0.$1.saved.dnskeycount
  
  if [ "${dnskeycount}" != "${olddnskeycount}" ]; then
    err=1
    echo "Warning: Number of DNSKEY-Record has changed!"
  else
    echo "OK: Number of DNSKEY-Record is the same as with last test!"
  fi
  
  exit ${err}
  

• the DNSSEC monitoring system should write an audit trail of DNSSEC zone changes:
  • changes to the DNSKEY records (KEY-ID and SOA Serial of the change)
  • changes to the DS-Record (KEY-ID and SOA serial of the parent zone)

• DNSSEC tools from .SE TLD: https://github.com/dotse/dnssec-monitor
• Verisign jdnssec-tools http://www.verisignlabs.com/dnssec-tools/
• YAZVS — Yet Another Zone Validation Script http://yazvs.verisignlabs.com/
• ldns-verify from the LDNS package http://www.nlnetlabs.nl/projects/ldns/
• Nagval - Nagios Plugin by JP Mens https://github.com/jpmens/nagval
• Key-Checker - Monitors Key-Rollover https://github.com/bortzmeyer/key-checker
• Mess with DNS https://messwithdns.net/

Prometheus is a very popular application for event monitoring and alerting. It records metrics in real-time in a time-series database using an HTTP pull model. It's written in Go and is Open Source.

For monitoring BIND, Prometheus requires a so-called exporter which provides metrics which Prometheus consumes, and we'll use bind_exporter for doing so. bind_exporter has to access BIND's metrics which named will make available via its statistics-channels{} feature.

dig @localhost dnslab.org a       # NOERROR
dig @localhost xyz.dnslab.org a   # NXDOMAIN
dig @localhost fail02.dnssec.works a  # SERVFAIL

• For reverse DNS lookup, the IP address must be converted into a domain name (because in DNS the query index is always a domain name)
  • In IP addresses, the hierarchy is from left to right (network part on the left of an IP address, host part on the right)
  • In domain names, the hierarchy is from right to left (Root-Zone and Top-Level domain on the right, host part on the left)
  • To map an IP address to a domain name, we reverse the order of all digits of the address, and add the second level domains in-addr.arpa. (IPv4) or ip6.arpa. (IPv6)
    • For IPv4 = 192.0.2.100 -> 100.2.0.192.in-addr.arpa.
    • For IPv6 = 2001:db8:ff::1 -> 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.F.F.0.0.8.B.D.0.1.0.0.2.IP6.ARPA
  • The domain name for an IP address is stored as a PTR (Pointer) record with the domain name in the record data

132.0.2.192.IN-ADDR.ARPA.    3600 IN PTR www.example.com.

• arpaname is a BIND tool for generating a PTR RR from an IP address.

  $ arpaname 10.4.13.132
  132.13.4.10.IN-ADDR.ARPA
  $ arpaname 2001:db8:a1:CAFE::fab
  B.A.F.0.0.0.0.0.0.0.0.0.0.0.0.0.E.F.A.C.1.A.0.0.8.B.D.0.1.0.0.2.IP6.ARPA
  

• dig supports a shortcut notation for reverse lookup queries with the -x switch:

  $ dig -x 192.168.1.36
  

• is the same as

  $ dig 36.1.168.192.in-addr.arpa PTR
  

• Reverse domains are delegated from the Regional Internet Registries (RIRs) and Local Internet Registries (LIRs)

• You need to request your reverse space be delegated to your authoritative servers by the entity that gave you IP address space (ISP, RIR, LIR etc)

• Zonefile for reverse zone for network block 192.168.1.0/24:

  $TTL 60
  $ORIGIN 1.168.192.in-addr.arpa.
  
  ;; the domain names of the authoritative name servers are always in
  ;; a "normal/forward" zone, not in the namespace of a reverse zone!
  
  @               SOA   nsNNa.dnslab.org.    .    1001 1h 30m 40d 60
                  NS    nsNNa.dnslab.org.
  
  ;; the Pointer Records point to the names of the machines
  1               PTR   beta.zoneNN.dnslab.org.
  20              PTR   alpha.zoneNN.dnslab.org.
  25              PTR   delta.zoneNN.dnslab.org.
  100             PTR   gateway.zoneNN.dnslab.org.
  

• Zone-Block in named.conf

  zone "1.168.192.in-addr.arpa" {
       type primary;
       file "1.168.192.in-addr.arpa";
  };
  

• $ORIGIN is a variable that is appended to non full qualified domain names (FQDN).
  • It applies to owner names and RDATA name fields.
  • The default $ORIGIN is the name of the zone.
  • @ allows the explicit use of the $ORIGIN value.

• A RR that begins with a white space inherits the most recently specified owner name.
  • This means that order matters in a zone file.
  • Be careful not to add a new RR with a new name before a RR inheriting its name.
  • A new owner name is always left justified.

  zoneNN.dnslab.org.      30 IN NS   dns2.someotherzone.com.
                           30 IN NS   ns1
                           30 IN A    192.0.2.67
  

• A RR without a TTL is assigned the zone's default TTL.
  • The default is defined with the $TTL control statement.
  • $TTL can be used multiple times in a zone file.
  • The most recent $TTL is the default.

  $TTL 30
  zoneNN.dnslab.org.      IN NS   dns2.someotherzone.com.
                           IN NS   ns1
                           IN A    192.0.2.67
  

• Note the different inheritance rules for the Owner Name and the TTL.
  • A RR without an owner name inherits the previous RR's name.
  • A RR without a TTL inherits the zone's default TTL.

• Without being explicitly assigned, a RR inherits its class from the zone’s definition.
  • For BIND, the definition is in named.conf.
  • IN is the default class. You are unlikely to create zones with any other class.

  @         30 SOA ns1 hostmaster 5 7200 3600 2592000 600
            30 NS  ns1
  ns1       30 A   192.0.2.100
  

• $INCLUDE reads another file into the zone file.
• If you have zones with identical data, $INCLUDE lets you avoid recreating and maintaining the same information.

   @         30 SOA ns1 hostmaster 5 7200 3600 2592000 600
             30 NS  ns1
             30 NS  dns2.someotherzone.com.
  ns1       30 A   192.0.2.100
  $INCLUDE "common.rr.txt"
  

$TTL 30
zoneNN.dnslab.org. 30 IN NS   dns2.xy.com.
zoneNN.dnslab.org. 45 IN A    192.0.2.44
zoneNN.dnslab.org. 45 IN A    192.0.2.67

$TTL 300
abc                90 IN AAAA 2001:db8::cafe
abc                90 IN AAAA 2001:db8::dead
def               300 IN AAAA 2001:db8::dab

• stub zones are configured in the named.conf of a DNS resolver
• the backup file is optional and will be created

  zone "private.dns.zone.example" {
    type stub;
    primaries { <ip-addresses-of-primaries>; };
    file "private.dns.zone.example.stubzone";
  };
  

• If multiple resolvers use a forwarder, they benefit from the forwarder's cache.
• If the forwarder has better Internet connectivity than a resolver using it, the resolver benefits again.
  • This is particularly applicable for resolvers blocked by firewalls from reaching authoritative servers.

• Forwarding architectures that depend on a small number of forwards are brittle.
• Forwarding intercepts the standard resolution process and is harder to troubleshoot.
• Forwarding configuration is not replicated by zone transfer, where normal delegation is (named.conf vs. zone data information).
  • Forwarding must be configured on each resolver using it.

• If all forwarders fail, a DNS resolver server resolves normally (forward first mode).
• forward only; completely disables normal resolution. The default is forward first;
• If no forwarder returns an answer, the resolver returns: SERVFAIL

options {
  forwarders { <IP address>; <...> };
  forward only;   # The server can no longer be accurately called recursive!
};

• A forward zone configures the resolver to forward for one domain.
  • The domain "." is the equivalent of global forwarding.
  • Remaining queries are through the normal recursive process.
  • They are used primarily in intranets, were introduced in BIND 9.1, and are known as conditional forward in Microsoft DNS (introduced in Windows Server 2003).

• A forward zone should be directed to a recursive server that can resolve for the specified domain.
• However, a forward zone is often directly to the authoritative server of the zone.
  • Before stub zones were introduced, this was reasonable.
  • Today, stub zones are the best way to redirect to an authoritative server.

zone "example.com." {
     type forward;
     forwarders { 192.0.2.110; 192.0.2.120; };
};

• Forward zones apply to queries ending in the specified domain name.
• Queries for sub.example.com and ab.de.fg.example.com would be forwarded.