Logical network structure in the DiRT lab

The DiRT group maintains two classes of end-workstations:

• production -- connected directly to the departmental network
• experimental -- connected to the department through one of our routers, if at all.

Production machines

Production machines are the ones that need a steady connection to the department network. These are primarily our development machines, mounting NFS partitions, etc. through the department. The production machines all have their network cards configured to use their 137A subnet address. These machines either use a netmask of 0xffff0000 and a default router of 152.2.254.254 (the cisco router in oit) or a netmask of 0xfffff000 and set their default route to the interface that connects to the department ethernet. This is an odd set-up (or at least seems so to me) which I really don't understand but it's the way most of the departmental unix machines seem to be set up. Note that in either of these cases the 137A address isn't really part of a 137A "subnet" but part of the larger 152.2.128.0 or 152.2.0.0 subnets, respectively. We've just been given the 137A portion of that address space for naming our machines.

Experimental machines

Experimental machines are anything that we don't want connected to the department ethernet. Most of these machines will usually have connectivity to the department via a router but these machines may also be disconnected from time to time. We have a configuration that offers 11 subnetworks for our experimental machines. The subnets are in the range 152.2.134.* to 152.2.139.* with the netmask 0xffffff80. We refer to these subnets by the number of their third octet and A or B for a 0 or 1, respectively, in the 25th bit (e.g. goober134A). Using 25 bits for the network mask leaves the last 7 bits for machine identifiers within a given subnet (i.e. 64 machines per subnet). Note that this netmask offers 12 subnets within that address range above. The 137A namespace is used for the production machines as noted above, leaving 11 experimental subnets. It is doubtful that we will acutally use all 11 subnets in the forseeable future but chances are good that we may exceed the 6 subnets offered with a 24 bit netmask. In addition to the ip address and netmask, these machines also have to specify a default router. In most cases this is the machine connecting the subnet to the departmental network. For example, speedy specifies mtpilot138a (152.2.138.107) as its default router,

Routers and Proxy-Arp

Routing

A router is simply a machine with multiple network interfaces (ethernet or token-ring cards). Each interface is configure based on the network it's attached to. It if is the departmental interface it is configured like a production machine above. If it is an experimental subnetwork interface it is configured like an experimental machine above. We use 6611s, RS6000s, and FreeBSD machines as routers. That is the sole purpose of the 6611s. The FreeBSD machines must have a flag set at boot-up in order for the machine to act as a gateway. I'm not sure if this is necessary for RS600s or not (the ones we use have been configured as routers since before I got here). When a router receives a packet intended for the IP address of another machine, it attempts to send it out over the interface with the appropriate netmask or to the default router otherwise (if one is specified). Packets get directed to these routers off of the department network via an evil hack called proxy-arp.

Proxy-Arp

In order for an experimental machine to have proper connectivity to the department it must be configured with the appropriate ip address and netmask as described above, and be plugged into the correct network which is connected to a properly configured router. In addition to these "obvious" steps our departmental configuration offers an additional requirement, proxy-arp. Because our departmental machines all communicate directly (without a router) they all just find local machines (those in the departmental subnet) via ARP requests. Since our experimental machines are hidden behind a router they cannot see these ARP messages and thus, cannot answer them. Instead, some other machine must answer on their behalf. Further the proxy machine doesn't respond with the machine's true hardware address, but with the hardware address of the router that handles that machine's connection to the departmental subnet. For example, consider mt-pilot, which handles routing between the department and the 138A subnet which speedy138A is connected to. Topsecret (also directly connected to the departmental network) handles proxy-arp for the 138A&B subnets. When a deparmental or production machine (e.g. taz) broadcasts an arp request for speedy138A, topsecret responds with the hardware address of the card connecting sarah to the department. Taz then sends the packets destined for speedy138A to sarah, who routes the packets onto the 138 subnet.

Token rings and ethernets

It may be worth noting that traditionally we have used the 134 and 135 subnets for token ring and 136-139 for ethernet. Until now we have only had 6 subnets so all of the addresses were effectively on the "A" subnets as we had no machine addresses with the final octet greater than 127 (0x7f). Some older demos still expect the networks to be set up this way.

Name to address-mapping:

Each of our machines has (or should have) 19 different entries in the domain name service. Most of them are of the form . That is, goober on the 134B subnet (152.2.134.129) is goober134B. For historical reasons, the "A" subnet addresses also have entries without the A/B identifier (i.e. goober134 == goober134A = 152.2.134.1 ). Finally, the 137A addresses are also in the DNS as the straight hostname (e.g. goober). The logic for this is that since this is subnet we use for machines that are connected directly to the department network, those should be the unqualified names.

Here an example of all of the entries for goober.

152.2.134.1   = goober134,goober134A
152.2.134.129 = goober134B
152.2.135.1   = goober135,goober135A
152.2.135.129 = goober135B
152.2.136.1   = goober136,goober136A
152.2.136.129 = goober136B
152.2.137.1   = goober,goober137,goober137A
152.2.137.129 = goober137B
152.2.138.1   = goober138,goober138A
152.2.138.129 = goober138B
152.2.139.1   = goober139,goober139A
152.2.139.129 = goober139B