Local Managed DNS (Java)

Introduction. As you probably know it is quite often when Web developer goes to change default DNS behavior on his workstation. I mean sometimes it is important to change host name resolution locally and point specific host name to localhost ip address. Suppose you work under 'blahblah.com' web site on your workstation and you have to test this site locally, for example by starting you browser and point it to http://blahblah.com. While this site is already on production you browser resolves 'blahblah.com' to real IP address instead of local one. Of course the simplest way here (and I am sure it is quite wide spread) is to change you local hosts file (/etc/hosts or %SystemRoot%\system32\drivers\etc\hosts) by adding the following entry: 127.0.0.1 blahblah.com It is ok and works quite well not only for browser but also for all IP applications running on you workstation. Today I have a deal with system which has at least three stages: development, staging and production. Thus, sometimes I have to change my hosts file and switch host name-IP mapping for all those configurations. My host now looks like: #this is for dev 12.12.12.10 blahblah.com #this is for staging #12.12.12.11 blahblah.com #this is for production #12.12.12.12 blahblah.com

Problem area. My real hosts file looks much more complicated, because we have more than one server on each stage. Every time when you want to switch environment you go to hosts and comment/uncomment certain lines. Everything works fine until you have to write automated stand alone tool in Java which has to manage this hostname-ip mapping itself. There are a lot of possible targets for such tools. It can be some kind of testing tool, which has to execute several tests on each environment at ones (dev, stage, prod). Or it can be quite complicated tool which has to monitor cluster of web sites (i.e. by execute functional tests against it) where each host has its unique IP address but they all belongs to one host name. It is not very good idea to install this monitoring tool on each node in the cluster. It is better to have dedicated monitoring node which will run its tests in sequence or in parallel, and perform its operations under all nodes in the cluster. There are no any problems in implementing such a tool along if you know all IP addresses of all nodes in the cluster and you tests relay only on this IP addresses. However, in web development host name it is very important. It helps web server which listens on specific IP_address:port to determine certain application requested by particular URL request. And sometimes this web application performs HTTP redirects in order to perform several business operations, which also relay on host name. So at this point we have to implement something like:

Implementation. Do you have any ideas how to implement it? My first idea was to manage hosts file content from my monitoring java application? He-he. Yes, it was not very good idea. There are a lot of other applications can be run on monitoring node and I should not affect them at all. Thus I have to do something inside my JVM to perform host name resolution management. This is a good point let's dig inside it. I have my monitor tool ready, and it uses a lot of network libraries (apache http client, http Unit, CORBA nameservers and so forth). All those libraries use java.net package to perform all network operations. By default this package internally relies on Sun implementation of the IP stack. It means that I have to do something to alter default host name resolution behavior inside java.net package. Let's start. Definitely the entry point is InetAddress class and its

byte[][] lookupAllHostAddr(String name)

method which performs lookup by given hostname, and returns array of IP addresses belong to this hostname. You probably cannot find this method using your javadoc, he-he, and you are right this method is a part of java.net.InetAddressImpl interface which is package level and InetAddress itself doesn't implement it. But internally it uses Inet4AddressImpl class to perform several operation, and this Inet4AddressImpl implements InetAddressImpl interface. lookupAllHostAddr is the native method inside Inet4AddressImpl. Good investigation, but it still won't help us because we still have no idea how to alter name resolution behavior. It is good practice and common way to override default implementation using standard discovery mechanism. You have to place in your META-INF/services/facroty_name_here text file with one line as content with full qualified class name of the factory. This file is used by resource factory configuration.which instantiates this factory class mentioned in file and uses it to produce concrete objects. Common example is XML related libs which uses the following descriptors in META-INF/services:

• com.sun.org.apache.xerces.internal.xni.parser.XMLParserConfiguration
• javax.xml.parsers.DocumentBuilderFactory
• javax.xml.parsers.SAXParserFactory
• javax.xml.validation.SchemaFactory
• org.w3c.dom.DOMImplementationSourceList
• org.xml.sax.driver
• com.sun.org.apache.xml.internal.dtm.DTMManager
• javax.xml.transform.TransformerFactory

Preliminary the same situation we have here in Sun's InetAddress. InetAddress looks at the sun.net.spi.nameservice.provider.X system property in order to decide which name service implementation to use. By default it uses Inet4AddressImpl to create anonymous sun.net.spi.nameservice.NameService object. You have two options here:

1. specify sun.net.spi.nameservice.provider.1=default|dns,sun system property. This will use Sun's DNS name service provider through JNDI.
2. create your own name service and specify sun.net.spi.nameservice.provider.1 system property with you custom value.

Hey! It seems we are about to bring an issue to a close very fast! Let's create our own sun.net.spi.nameservice.NameService, god bless Sun, there are only two methods to override. And then we will be ready to plug it into sun.net.spi.nameservice.provider.X infrastructure. This falls into several steps.

1. create descriptor in META-INF/services/sun.net.spi.nameservice.NameServiceDescriptor and mention our new descriptor class there: org.ots.dns.LocalManagedDnsDescriptor
2. create LocalManagedDnsDescriptor itselfs, it should implements sun.net.spi.nameservice.NameServiceDescriptor and return NameService in createNameService() method
3. create our custom NameService itself: public class LocalManagedDns implements NameService

Now we are going to implement only lookupAllHostAddr which can look as follows:

        if ("blahblah.com".equalsIgnoreCase(hostname)) {
            byte[] ip = Util.textToNumericFormat("12.12.12.10");
            return new byte[][] { ip };
        } else {
            throw new UnknownHostException();
        }

This is quite enough for the first test. If you run this code you will find that it works! However it works only for 'blahblah.com', every other host name lookup will throw UnknownHostException. It is good that we substitute our host name with desired IP address, but we also have to do something with other hostname lookups which are not involved into manipulations. The first idea here is to create sun.net.spi.nameservice.providerS tree, in the way if one provider cannot lookup host then another try to do the lookup itself: sun.net.spi.nameservice.provider.1=dns, LocalManagedDns sun.net.spi.nameservice.provider.2=default sun.net.spi.nameservice.provider.3=dns, sun However, these or any other combination of the properties won't help us, because InetAddress creates only one NameService for its own needs. As I understand this is the first one which is successfully created. For us it means that we have to deal with all host name lookup in our LocalManagedDns. Eh-h-h, it is not very good news. So, let's try to use Sun name service implementation inside our LocalManagedDns. Go to java.net and let's grab something from it. But all interesting classes there are final and package level. Thus, we cannot use them at all. Of course we can try to write our own class in java.net package and extends it from Inet4AddressImpl. But we will get "java.lang.SecurityException: Prohibited package name" at runtime in this case. And as I know there is no way around it, neither java.policy will help us. I could not find any way to use native JVM name service functions from user code. Thanks god, there is java libraries that can take care about DNS functions. It is DnsJava project. It has full DNS server/client functionality, but in our case all we need is to lookup host by name. With DnsJava it can be done in one line of code. Let's create DNSJavaNameService instance inside our LocalManagedDns and delegate all unmatched call to it. Also let's introduce NameStore, it is singleton which will store custom hostname/IP mapping and provide API to manage such a mappings.

package org.ots.dns;

import java.net.UnknownHostException;

import org.apache.commons.lang.StringUtils;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import sun.net.spi.nameservice.NameService;

/**
 *
 * @version $Id$
 * @author Roman Kuzmik
 */
public class LocalManagedDns implements NameService {
    private static final Log log = LogFactory.getLog(LocalManagedDns.class);

    NameService defaultDnsImpl = new DNSJavaNameService();

    /**
     * @see sun.net.spi.nameservice.NameService#getHostByAddr(byte[])
     */
    public String getHostByAddr(byte[] ip) throws UnknownHostException {
        log.debug("");

        return defaultDnsImpl.getHostByAddr(ip);
    }

    /**
     * @see sun.net.spi.nameservice.NameService#lookupAllHostAddr(java.lang.String)
     */
    public byte[][] lookupAllHostAddr(String name) throws UnknownHostException {
        log.debug("");

        String ipAddress = NameStore.getInstance().get(name);
        if (!StringUtils.isEmpty(ipAddress)){
            log.debug("\tmatch");
            byte[] ip = Util.textToNumericFormat(ipAddress);
            return new byte[][]{ip};
        } else {
            log.debug("\tmiss");
            return defaultDnsImpl.lookupAllHostAddr(name);
        }
    }

}

Let's design NameStore in way it can handle singleton scope mapping as well as local thread level mapping. In my monitoring tool I have thread pool which executes task in parallel and every task thread has to have its own hostname/IP mapping.

package org.ots.dns;

import java.util.Collections;
import java.util.HashMap;
import java.util.Map;

import org.apache.commons.lang.StringUtils;

/**
 *
 * @version $Id$
 * @author Roman Kuzmik
 */
public class NameStore {

    protected static NameStore singleton;

    protected Map globalNames;
    protected ThreadLocal localThread;

    protected NameStore(){
        globalNames = Collections.synchronizedMap(new HashMap());
        localThread = new ThreadLocal();
    }

    public static NameStore getInstance(){
        if (singleton == null) {
            synchronized (NameStore.class) {
                if (singleton == null) {
                    singleton = new NameStore();
                }
            }
        }
        return singleton;
    }

    public void put(String hostName, String ipAddress){
        globalNames.put(hostName, ipAddress);
    }
    public void remove(String hostName){
        globalNames.remove(hostName);
    }

    public synchronized void putLocal(String hostName, String ipAddress){
        Map localThreadNames = (Map) localThread.get();
        if (localThreadNames == null){
            localThreadNames = Collections.synchronizedMap(new HashMap());
            localThread.set(localThreadNames);
        }
        localThreadNames.put(hostName, ipAddress);
    }
    public void removeLocal(String hostName){
        Map localThreadNames = (Map) localThread.get();
        if (localThreadNames != null){
            localThreadNames.remove(hostName); 
        }
    }

    public String get(String hostName){
        String ipAddress = null;
        Map localThreadNames = (Map) localThread.get();
        if (localThreadNames != null){
            ipAddress = (String)localThreadNames.get(hostName); 
        }
        if (StringUtils.isEmpty(ipAddress)) {
            return (String)globalNames.get(hostName);
        }
        return ipAddress;
    }

}

At this point we are ready to write simple test:

        hostName = "google.com";
        ipAddress = "127.0.0.1";
        NameStore.getInstance().put(hostName, ipAddress);
        performLookup(hostName);

        hostName = "google.com";
        NameStore.getInstance().remove(hostName);
        performLookup(hostName);

This code should lookup google.com to 127.0.0.1 at a first stage and then lookup the same host to its real IP addresses. First stage works fine, but second fails. It is not because we done something wrong in stage two, it is because InetAddress maintains addressCache. And during second lookup request it simply return cached value. Ups-s-s. We've created our own NameServiceProvider, plugged it into JVM, incorporated JavaDns and all these things do not work because of InetAddress.addressCache ?! Nice:. Let's go back to Java sources again:. searching: found: there is property which can help us! //disable DNS cashe Security.setProperty("networkaddress.cache.ttl", "0"); Now our test code works fine:

        hostName = "google.com";
        ipAddress = "127.0.0.2";
        NameStore.getInstance().put(hostName, ipAddress);
        performLookup(hostName);
      
        hostName = "google.com";
        ipAddress = "127.0.0.3";
        NameStore.getInstance().putLocal(hostName, ipAddress);
        performLookup(hostName);

        new Thread(){

            public void run() {
                String hostName = "google.com";
                try {
                    performLookup(hostName);
                } catch (UnknownHostException e) {
                    e.printStackTrace();
                }
            }
 
        }.start();

Output:

[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDnsDescriptor.getType():33]
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDnsDescriptor.getProviderName():41]
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDnsDescriptor.createNameService():53]
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDns.lookupAllHostAddr():34]
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDns.lookupAllHostAddr():38]     match
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDnsTest.performLookup():74] google.com/127.0.0.1
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDns.lookupAllHostAddr():34]
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDns.lookupAllHostAddr():42]     miss
[java] 19:52:05(DEBUG)[vzb.dns.DNSJavaNameService.lookupAllHostAddr():32] google.com
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDnsTest.performLookup():74] google.com/64.233.187.99
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDnsTest.performLookup():74] google.com/72.14.207.99
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDnsTest.performLookup():74] google.com/64.233.167.99
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDns.lookupAllHostAddr():34]
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDns.lookupAllHostAddr():38]     match
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDnsTest.performLookup():74] google.com/127.0.0.2
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDns.lookupAllHostAddr():34]
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDns.lookupAllHostAddr():38]     match
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDnsTest.performLookup():74] google.com/127.0.0.3
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDns.lookupAllHostAddr():34]
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDns.lookupAllHostAddr():38]     match
[java] 19:52:05(DEBUG)[vzb.dns.LocalManagedDnsTest.performLookup():74] google.com/127.0.0.2

Conclusion. In this article we wrote our custom NameServiceProvider which able to manage its name resolution behavior through it's simple API. It looks like we have our own hosts file inside JVM. See attached LocalManagedDns.zip for the full source code provided along with this article. This is an example code so there is no license required. Please see corresponding license for libraries included into this package.

References. http://java.sun.com/j2se/1.5.0/docs/guide/net/properties.html http://www.xbill.org/dnsjava/