Multihomed Computers and RMI

A multihomed computer is one that has more than one
network interface. Problems arise when you export an RMI object
from such a computer. Here's why, and some ways you can work
around the problem.

A typical example of a multihomed computer is a laptop with
both Ethernet and WiFi interfaces. If you look closely at such
a computer, you'll see that each interface has its own IP
address. Use ifconfig -a on Unix or
ipconfig/a on MS Windows to see these addresses.
The picture here shows my laptop, which is connected through an
Ethernet cable to the company intranet and through WiFi to a
wireless access point.

width="482" height="386" alt="My laptop connected to two networks"/>

There's an important point here, which is that an IP address is the
address of a network adaptor, not the address of a computer.

Now let's look at some details of how RMI works. To interact with a
remote object, you need to get a stub for it. Typically you connect
to an
RMI registry to get a remote object to start from. Methods on
that object may return references to other remote objects, which are also
stubs.

For example, one way to use the JMX Remote API is to pick up a stub
for a remote
RMIServer object. Then you call
newClient() on that object, which returns you a stub for a new

RMIConnection object. The picture here shows the first step, where you
pick up the stub from the RMI registry.

width="581" height="414" alt="Looking up a stub in the RMI registry">

What do these stubs look like? The stub object implements the
appropriate
Remote interface, for example RMIServer or RMIConnection or Spume. The
implementation of these methods in the stub forwards the method to the
remote object. So when you call stub.newClient(), for example, the call
is forwarded over the network to the real RMIServer object on the remote
machine. This is just basic RMI and should be no surprise.

Stubs are
serializable. This is how you can get them from a remote RMI
registry, of course. But it also means that you can send the stub to
someone else. A stub doesn't care where you get it from - it just
connects to its remote object and does its stuff.

So what's inside a stub?

width="390" height="405" alt="What's inside a stub?">

• The host where the remote object lives. This is a string,
  and we will have much more to say about it later.
• The port on that host where RMI is listening for requests
  for that object and possibly other objects.
• The object id that allows RMI to know which object you
  are talking to.
• The socket factory, an instance of
  
  RMIClientSocketFactory that controls how the connection to the
  given host and port is made.

The first item we're interested in here is the host. This
is a string, and by default it is the result of

InetAddress.getLocalHost().
getHostAddress(). In other words it is a numeric IP address like
10.0.10.58. We can immediately see why this default behaviour is going
to be a problem for my multihomed laptop. This address is the WiFi
address. If a client from within the other network (the company intranet)
wants to connect using this stub, it can't.

The simple solution that RMI provides for this situation is a

system property, "java.rmi.server.hostname". If I only ever want
connections from clients within the intranet, I just set this to
the intranet address of my laptop, 129.157.209.250, and I'm done.

But what if I want connections both from the intranet and from the
wireless network? That's where things get a bit more complicated.

Domain Name Service

First, if my laptop has a
DNS
name such as eamonnslaptop.france.sun.com
then I might be able to arrange for that name to resolve to more than one
IP address. (Look at the

DNS lookup for google.com for example.)
So I could set java.rmi.server.hostname to the DNS name and
wait for
RFE 5052134 to be implemented.

Apart from the fact that I might not be
able to wait, this solution is unsatisfactory because it's highly unlikely
that such a DNS name exists in my case. The two IP addresses come from
two different
DHCP servers, one for the local intranet and one for the local
wireless network. They will probably be different if I come back tomorrow
and connect up my laptop again. There are plenty of other scenarios,
for example involving "floating IP addresses", that will also fail.

Client socket factory solutions

All is not lost, though. Remember that, in addition to a host and
port, the stub also contains a
client socket factory which controls exactly how a connection is made
to the host and port. By supplying our own socket factory, we can try
to do the right thing for all clients, regardless of where they are
connecting from.

Before we see what that might look like, a note about what it implies.
The client socket factory is part of the RMI stub, which means, paradoxically,
that it is defined by the server. So if I want my clients to be
able to do some magic to choose the right address to connect to, I'll
need to export my remote object appropriately. Clients will not be able
to choose to use the magic factory on their own, or indeed not to use it
if I have defined one. (Well, using reflection or serialization games,
they might, but it will almost certainly not be portable.)

Another implication is that my clients will need to have the
client socket factory class in their classpath. (Or they could
set up
code downloading if they are very brave.)

Given that, what might the client socket factory look like?

ThreadLocal client socket factory

One idea (due to Laurent Farcy) is to have a

ThreadLocal variable that the client sets explicitly around code
that might use a stub. The outline might be something like this...

            String oldHostName = ThreadLocalRMIClientSocketFactory.getHostName();
            try {
                ThreadLocalRMIClientSocketFactory.setHostName("129.157.209.250");
                ...operations using the stub...
            } finally {
                ThreadLocalRMIClientSocketFactory.setHostName(oldHostName);
            }
        

...with a socket factory that looks something like this (untested
code)...

            public class ThreadLocalRMIClientSocketFactory
                    implements RMIClientSocketFactory {
                private static final ThreadLocal<String> hostName =
                    new ThreadLocal<String>();
                
                public Socket createSocket(String host, int port)
                        throws IOException {
                    String hostOverride = getHostName();
                    if (hostOverride != null)
                        host = hostOverride;
                    return new Socket(host, port);
                }

                public static String getHostName() {
                    return hostName.get();
                }

                public static void setHostName(String name) {
                    hostName.set(name);
                }
            }
        

Basically, the socket factory ignores the address contained in the
stub and forces the address to be the one you set with the
ThreadLocal instead.
This is workable if (a) you know the address that your stub is
supposed to connect to, and (b) you can be sure that the host name
is set appropriately around every use of the stub. (RMI can close
idle connections at any time and will create a new one the next
time you use the stub.) These conditions can be satisfied for
certain uses of the

JMX Remote API and you could encapsulate them in a custom

JMXConnectorProvider. For example, you could arrange for connections
to the
JMXServiceURL
service:jmx:mrmi://129.157.209.250:8888/jmxrmi to pick
up an RMIServer from the RMI registry at that address and to set the
ThreadLocal to 129.157.209.250 around the call to

RMIServer.newClient() and around all calls to the resultant RMIConnection.
It's heavy going, but you can do it.

Choice of IP addresses

Another possibility is to set the java.rmi.server.hostname property
to a list of all local IP addresses. Then you can define a
client socket factory that picks the right IP address for the client.

The tricky part is in that last sentence. How do we know which IP
address is appropriate? If I have the list [129.157.209.250, 10.0.10.58],
how do I know which address is the one for me?

One answer is that I could simply try to connect to all of them and
pick whichever one works. I can make all the connection attempts in
parallel using a NIO

Selector. Or, if I'm on at least version 5.0 of the Java platform,
I could use

isReachable() on each address, perhaps in parallel in different threads.
(The remarks in the isReachable() documentation might discourage me from
doing that, however.)

Either way, though, I have to face up to an uncomfortable truth,
which is that not all IP addresses are unique. The 10.0.10.58
address of my WiFi interface is an example. The 10.* IP addresses are
specifically
reserved for private networks. A client on the same wireless
network as my laptop can contact it using that address. A client on
the company intranet can't, but it's possible that the same
address is being used for something else on that network. Such
a client might end up connecting to some totally different machine
thinking it was my laptop.

For any given network configuration, you can probably find appropriate
logic to pick the right IP address out of the list. But you probably can't
write logic that will work everywhere.

With that caveat, let's look at the details. First,
we need to set the java.rmi.server.hostname property appropriately.
In my example,
it will be set to "129.157.209.250!10.0.10.58". Notice that if there is only
one network interface, the property will be set to the same value
that RMI would have used anyway. But
if there is more than one, it will be set to a string that only our
magic client socket factory can understand. Because system properties
are global to the Java Virtual Machine, this means that all RMI
objects in the JVM had better be exported with the socket factory.
Given that default RMI doesn't work splendidly with multihomed machines
this is not necessarily a big drawback. You could also try to set the
property just at the point where you export your objects, though it will
be hard to coordinate with other possible RMI exporters in the JVM.

So here's the code to set the property.

        System.setProperty("java.rmi.server.hostname",
            addressString(localAddresses()));
        ...
            
    private static Set<InetAddress> localAddresses() throws SocketException {
        Set<InetAddress> localAddrs = new HashSet<InetAddress>();
        Enumeration<NetworkInterface> ifaces =
                NetworkInterface.getNetworkInterfaces();
        while (ifaces.hasMoreElements()) {
            NetworkInterface iface = ifaces.nextElement();
            Enumeration<InetAddress> addrs = iface.getInetAddresses();
            while (addrs.hasMoreElements())
                localAddrs.add(addrs.nextElement());
        }
        return localAddrs;
    }

    private static String addressString(Collection<InetAddress> addrs) {
        String s = "";
        for (InetAddress addr : addrs) {
            if (addr.isLoopbackAddress())
                continue;
            if (s.length() > 0)
                s += "!";
            s += addr.getHostAddress();
        }
        return s;
    }
        

(This code, and the remainder of the code, should work unchanged on
versions 5.0 or 6 of the Java platform, and should work on 1.4 if you
remove the generics.)

Now here's the magic client socket factory. As I described, it
uses NIO to connect to all the addresses in parallel, and picks whichever
address works first. Again, be aware that
this may not work if there are network-private IP addresses in the picture.
You may want to modify the logic to work appropriately for your network
environment.

import java.io.IOException;
import java.io.Serializable;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.net.SocketAddress;
import java.nio.channels.Channel;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.rmi.server.RMIClientSocketFactory;
import java.rmi.server.RMISocketFactory;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;

public class MultihomeRMIClientSocketFactory
        implements RMIClientSocketFactory, Serializable {
    private static final long serialVersionUID = 7033753601964541325L;
    
    private final RMIClientSocketFactory factory;
    
    public MultihomeRMIClientSocketFactory(RMIClientSocketFactory wrapped) {
        this.factory = wrapped;
    }
    
    public Socket createSocket(String hostString, int port) throws IOException {
        final String[] hosts = hostString.split("!");
        final int nhosts = hosts.length;
        if (hosts.length < 2)
            return factory().createSocket(hostString, port);

        List<IOException> exceptions = new ArrayList<IOException>();
        Selector selector = Selector.open();
        for (String host : hosts) {
            SocketChannel channel = SocketChannel.open();
            channel.configureBlocking(false);
            channel.register(selector, SelectionKey.OP_CONNECT);
            SocketAddress addr = new InetSocketAddress(host, port);
            channel.connect(addr);
        }
        SocketChannel connectedChannel = null;
        
        connect:
        while (true) {
            if (selector.keys().isEmpty()) {
                throw new IOException("Connection failed for " + hostString +
                        ": " + exceptions);
            }
            selector.select();  // you can add a timeout parameter in millseconds
            Set<SelectionKey> keys = selector.selectedKeys();
            if (keys.isEmpty()) {
                throw new IOException("Selection keys unexpectedly empty for " +
                        hostString + "[exceptions: " + exceptions + "]");
            }
            for (SelectionKey key : keys) {
                SocketChannel channel = (SocketChannel) key.channel();
                key.cancel();
                try {
                    channel.configureBlocking(true);
                    channel.finishConnect();
                    connectedChannel = channel;
                    break connect;
                } catch (IOException e) {
                    exceptions.add(e);
                }
            }
        }
        
        assert connectedChannel != null;
        
        // Close the channels that didn't connect
        for (SelectionKey key : selector.keys()) {
            Channel channel = key.channel();
            if (channel != connectedChannel)
                channel.close();
        }
        
        final Socket socket = connectedChannel.socket();
        if (factory == null && RMISocketFactory.getSocketFactory() == null)
            return socket;
        
        // We've determined that we can connect to this host but we didn't use
        // the right factory so we have to reconnect with the factory.
        String host = socket.getInetAddress().getHostAddress();
        socket.close();
        return factory().createSocket(host, port);
    }
    
    private RMIClientSocketFactory factory() {
        if (factory != null)
            return factory;
        RMIClientSocketFactory f = RMISocketFactory.getSocketFactory();
        if (f != null)
            return f;
        return RMISocketFactory.getDefaultSocketFactory();
    }

    // Thanks to "km" for the reminder that I need these:
    public boolean equals(Object x) {
        if (x.getClass() != this.getClass())
            return false;
        MultihomeRMIClientSocketFactory f = (MultihomeRMIClientSocketFactory) x;
        return ((factory == null) ?
                (f.factory == null) :
                (factory.equals(f.factory)));
    }

    public int hashCode() {
        int h = getClass().hashCode();
        if (factory != null)
            h += factory.hashCode();
        return h;
    }
}
    

The MultihomeRMIClientSocketFactory constructor takes an
RMIClientSocketFactory parameter which can be null or another factory
to be used once the address to connect to has been determined.
For example, it could be an

SslRMIClientSocketFactory.

It would be nice if RMI came with something like this by default,
but I'm not sure given the design of RMI stubs that there is a good
general solution. At a minimum, it would be good if RMI allowed you
to specify the java.rmi.server.hostname locally for a given export,
and if it allowed you to inspect and change the socket factory inside
a stub.