DNS Troubleshooting

Posted by on



Some basic DNS troubleshooting Steps.

1.       First collect the information which has created the problem by doing some investigation(such as logs, debug messages etc) .
2.    Troubleshooting is nothing but considering so many issues that may cause the problem and working one by one to pin point the actual culprit which created the issue.

BIND is the Berkeley Internet Name Domain, DNS server. It is wildly used on UNIX and Linux like oses. You can use following tools to troubleshoot bind related problems under UNIX or Linux oses.

Task: Port 53 open and listing requests

By default BIND listen DNS queries on port 53. So make sure port 53 is open and listing user requests. by running any one of the following tests. See if you can telnet to port 53 from remote computer:
$ telnet remote-server-ip 53
OR
telnet ns1.linux-test.org domain
Output:
Trying 192.168.0.5...
Connected to ns1.linux-test.org.
Escape character is '^]'.
If you cannot connect make sure firewall is not blocking your requests. Next use netstat command to list open and listing port 53 on server itself:
$ netstat -tulpn | grep :53
OR
# netstat -atve
Output:
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode
tcp        0      0 ns1.linux-test.org:domain *:*                     LISTEN      named      10386
tcp        0      0 rhx.test.com:domain     *:*                     LISTEN      named      10384
tcp        0      0 *:ssh                   *:*                     LISTEN      root       1785
tcp        0      0 rhx.test.com:rndc       *:*                     LISTEN      named      10388
tcp        0      0 rhx.test.com:smtp       *:*                     LISTEN      root       1873
tcp        0      0 ns1.linux-test.org:ssh    w2k.linux-test.org:1057   ESTABLISHED root       10501
tcp        0      0 rhx.test.com:32773      rhx.test.com:domain     TIME_WAIT   root       0
tcp        0      0 ns1.linux-test.org:32775  ns1.linux-test.org:domain TIME_WAIT   root       0
tcp        0      0 rhx.test.com:32774      rhx.test.com:domain     TIME_WAIT   root       0
Make sure iptables firewall is not blocking request on server:

# iptables -L -n
OR
# iptables -L -n | less
Make sure named is running:
# /etc/init.d/named status
If not start named:
# chkconfig named on
# service named start

Task: Use log files

You can use log files after starting/restarting bind to see error messages:

# tail –f /var/log/message
Output:
Nov 17 16:50:25 rhx named[3539]: listening on IPv4 interface lo, 127.0.0.1#53
Nov 17 16:50:25 rhx named[3539]: listening on IPv4 interface eth0, 192.168.0.5#53
Nov 17 16:50:25 rhx named[3539]: command channel listening on 127.0.0.1#953
Nov 17 16:50:25 rhx named[3539]: zone 0.0.127.in-addr.arpa/IN: loaded serial 1997022700
Nov 17 16:50:25 rhx named[3539]: linux-test.org.rev:1: no TTL specified; using SOA MINTTL instead
Nov 17 16:50:25 rhx named[3539]: zone 0.168.192.in-addr.arpa/IN: loaded serial 12
Nov 17 16:50:25 rhx named[3539]: zone localhost/IN: loaded serial 42
Nov 17 16:50:25 rhx named[3539]: zone linux-test.org/IN: loaded serial 12
Nov 17 16:50:25 rhx named[3539]: running

Task: Check zone file for errors

You can check zone file syntax and /etc/named.conf file using following utilities. named-checkconf command is named (BIND) configuration file syntax checking tool.
# named-checkconf /etc/named.conf
Output:
/etc/named.conf:32: missing ';' before 'zone'
Plesse note that if named-checkconf did not find any errors it will not display in output on screen.
Check zone file syntax for errors. named-checkzone is zone file validity checking tool. named-checkzone checks the syntax and integrity of a zone file. It performs the same checks as named does when loading a zone. This makes named checkzone useful for checking zone files before configuring them into a name server.
# named-checkzone localhost /var/named/localhost.zone
OR
#named-checkzone linux-test.org /var/named/linux-test.org.zone
Output:
zone linux-test.org/IN: loaded serial 12
OK

Task: Testing BIND/DNS with utilities

You can use host and dig utilties to test your bind configuration.
  • host: host is a simple utility for performing DNS lookups. It is normally used to convert names to IP addresses and vice versa.
  • dig: dig (domain information groper) is a flexible tool for interrogating DNS name servers. It performs DNS lookups and displays the answers that are returned from the name server(s) that were queried. Most DNS administrators use dig to troubleshoot DNS problems because of its flexibility, ease of use and clarity of output. Other lookup tools tend to have less functionality than dig.
List IP address associated with host names:

# host linux-test.org
OR
# host www
Output:
www.linux-test.org has address 192.168.0.6
 
Perform a zone transfer for zone name using -l option:
# host -l linux-test.org
linux-test.org SOA ns1.linux-test.org. admin.linux-test.org. 12 10800 900 604800 86400
linux-test.org name server ns1.linux-test.org.
linux-test.org mail is handled by 10 mail.linux-test.org.
linux-test.org has address 192.168.0.5
gw.linux-test.org has address 192.168.0.254
mail.linux-test.org has address 192.168.0.7
ns1.linux-test.org has address 192.168.0.5
w2k.linux-test.org has address 192.168.0.1
www.linux-test.org has address 192.168.0.6
linux-test.org SOA ns1.linux-test.org. admin.linux-test.org. 12 10800 900 604800 86400

Other examples
# dig mail.linux-test.org
# dig 192.168.0.5

Generic troubleshooting tips

1)      Use tail –f to watch log file in real time, advantage is simple you can spot error or warning message in real time.
tail –f /path/to/log/file
Example(s):
# tail –f /var/log/maillog
2)      Use telnet command to see if you get response or not. Sometime you will also see some informative message:

telnet ip port
Example(s):

# telnet localhost 53
# telnet localhost 25

3) Make sure you can see PID of your service.

pidof service-name
cat /var/run/service.pid
Example(s):
# pidof sshd
# cat /var/run/sshd.pid

4) You need to make sure that your DNS server or third party DNS server (ISP) is accessible. This is an important step, as many network services depend upon DNS; especially sendmail/postfix or Squid etc for example. Run dig or nslookup. No timeout should occurred.

# dig your-domain.com
# nslookup gw.isp.com
# more /etc/resolv.conf

5) For networking troubleshooting, make sure your ip address configuration is right, gateway, routine, hostname etc all configured. Here is list of tools on RedHat Linux to verify or modify information:

Hostname verification or setup tools

  • hostname : To get hostname of server.
  • hostname –s : To get FQDN hostname of server
  • more /etc/sysconfig/network : To setup hostname and networking can enabled or disabled.
  • dnsdomainname : List or setup domainname.
  • more /etc/hosts :Make sure at least localhost entry do exist.

Ethernet configuration tools

  • ifconfig : To see running network card information.
  • ifconfig eth0 up|down : To enable|disable network interface
  • service network reload|restart|stop|start : To reload (after changed made in ip config file)|restart|stop|start network interface with all properties.
  • route|netstat –rn : To print routing table
  • ping ip-address : To see if host is alive or dead
  • more /etc/modules.conf : To see your network card configuration alias for eth0 exists or not.
  • lsmod : To list loaded modules (read as drivers), here you need to see that eth0 module is loaded or not, if not loaded then use insmod to insert (load) driver.
  • dhclient : Dynamic Host Configuration Protocol Client, run this if your Ethernet card is not getting ip from DHCP box on startup; this command does by default shows useful information.



To see if service blocked because of access control

  • iptables –n –L : To list all iptable rules; useful to see if firewall blocks service or not.
  • service iptables stop|start : To start|stop iptables
  • more /etc/xinetd.conf
OR
  • more /etc/xinetd.conf/SERVICENAME = To list configuration of xinetd server. Again useful to see if firewall xinetd based security blocks service or not (xinetd includes host-based and time-based access control)
  • more /etc/hosts.allow : To see list of hosts allowed to access service.
  • more /etc/hosts.deny : To see list of hosts NOT allowed to access service. NOTE first TCP wrappers (hosts.allow|hosts.deny) checked and then xinetd-based access control checked.
  • more /etc/path/to/application.conf : See your application configuration file for access control. For example smb.conf and many other applications/services got own access control list in application. You need to check that as well.
Read man pages of all above command for more details on syntax and usage.