How usernet works in tmix

How usernet w= orks in tmix

(All here taken directly from Felix’s dissertati= on)

Figure 5.3 illustrates the relationship between tmix a= nd the network layers in the traffic generation endhost in which a tmix instance r= uns. Tmix instances rely on the standard socket interface to create connections,= send and receive ADUs, and to close connections. The socket interface is an application programming interface (API) that enables user-level programs, s= uch as tmix, to communicate with other endhost using a programming abstraction similar to a file. Calls to the socket interface are translated by the kern= el into requests to use the process-to-process communication service provided = by the transport layer (TCP). The transport layer itself uses the host-to-host= communication service provided by the network layer (IP), and the network layer uses the = link layer to handle the network interface and create physical packets.

Our experiments also require a special simulation serv= ice, usernet, that can apply per-flow round-trip times and loss rates. This requ= ires a communication between tmix and usernet that is not directly supported by = the network stack. In order to overcome this difficulty, we use a covert channe= l to communicate between them. By having tmix assigning specific source port num= bers to each connection, we can use ioctl calls to create a table that maps sour= ce port numbers to round-trip times and loss rates in the usernet layer. Source port numbers are unique for each connection in the same endhost, so usernet= can appropriately use the source port number to decide which network parameters should be applied to each segment, which always includes the source port nu= mber[1].

Two instances of tmix can replay an arbitrary subset o= f Tc by establishing one TCP connection for each connection vector in the trace, with one instance of the program playing the role of the connection initiat= or and the other program the connection acceptor. To begin, the connection initiator performs a number of socket writes in order to send the number of bytes specified in the first data unit a1. The other end point will read as many bytes as specified in the data unit a1. In addition, this first data u= nit is used to synchronize the two instances of tmix, by including a 32-bit connection vector id in the first four bytes of the first data unit. Since = this id is part of the content of the first data unit, the acceptor

can uniquely identify the connection vector that is to= be replayed in this new connection. If a1 is less than 4 bytes in length, the connection initiator will open the connection using a special port number designated for connections for which the id is provided by the connection a= cceptor. This approach guarantees that the two tmix instances always remain properly= synchronized (i.e., they agree on the Ci they replay within each TCP connection) even if= connection establishment segments are lost or reordered. It also makes it possible to generate traffic without any control traffic.

In addition, we make use of a modified version of dumm= ynet to add arbitrary delays and

drop rates to the segments in a connection. Our modifi= ed version, that we will call usernet in the rest of this text, implements a user-level interface that can be used by tmix instances to assign per-connection delays and loss rates read from the input set of connection vectors. Traditional dummynet and earlier modification could not match individual connections with specific delays and loss rates. Furthermore, th= ey only implemented uniform distributions of delays, or required recompilation= for making use of empirical distributions. In our modification, the user-level program (tmix in our case) makes use of an ioctl system call to assign dela= ys and loss rates to specific connections, which are uniquely identified by th= eir source port number. The user-level program has therefore to keep track of t= he (dynamic) source port number that is used for each new TCP connection it op= ens.

Using this technique, usernet can determine the delay = and loss rate that should be applied to each segment simply by reading an entry= in a table indexed by source port number, so the lookup time is O(1). The numb= er of source port numbers is small (216), so this table does not require too m= uch kernel memory (524 KB).