Don Smith created a nice dummynet modification for FreeBSD 4.5 during Summer 2002. The idea is to create per-flow delays sampled from a cdf, rather than using the same delay value for all flows associated with a dummynet pipe.
There are three sysctl variables involved:
Long Le added an additional sysctl variable several months after Don's work that allows you to set the seed used by the random number generator. This variable is:
Note that the assignment of delay values using the given distribution is done when two pipe configuration conditions are true:
If you statically configure either of these to non-zero values, then this per-flow assignment scheme no longer functions.
If you need to disable Don's feature (e.g., you need to use dummynet to generate loss or queue size limitations only), then your only option is to set either delay or bandwidth to some non-zero value. Setting net.inet.ip.dummynet.delay_cdf_p1 and net.inet.ip.dummynet.delay_cdf_p2 both to zero results in random values that are not range-constrained. (I tried it just now.)
Don's source code for FreeBSD 4.5 is at ~smithfd/src/delaybox/dummynet/src/4.5. Long's revision, also for FreeBSD 4.5, is on puck at /usr/src/sys-altq/netinet. Here's a source listing for Long's code:
Below is Don's original email message announcing the new feature, and Long's email mentioning his additional modification.
-------- Original Message -------- Subject: FYI Date: Thu, 20 Jun 2002 10:36:57 -0400 From: Don Smith I have been able to get some new tools working that some of you may find useful. First, I modified dummynet to add a new mode in which it delays packets on a per-flow basis. In this mode, each time dummynet sees a new flow (address 5-tuple) it picks a random delay for that flow and creates a new queue for packets belonging to that flow. Each arriving packet of the flow is delayed by the delay value chosen for that flow. The delay for a flow is chosen from a uniform distribution, but that can be changed easily if we want. Jay and Long are now using per-flow delays (on the clients) in the gigabit experimental networks and it seems to work fine on the highest aggregate throughput we tried (>250 Mbps). The source for FreeBSD-4.5 is at ~smithfd/src/delaybox/dummynet/src/4.5 (search for "FDS" to find the changes). This turned out to be really trivial since most of the mechanisms are already built into dummynet to do per-flow bandwidth allocation. In order to test the per-flow delays, I also put together some programs to extract RTT values for TCP connections from tcpdump traces. For this you need to trace both directions of a link (as we do in the gigabit experimental network). The process is fairly complicated and consists of a series of steps to first prepare the tcpdump files using tcpdump and sort, extracting data and ACK segments from the traces, a merge-sort to order each connection's data segments from one trace with its ACK segments from the other trace, and finally using the merged data to compute RTT values. The final step produces two files: (a) one with the RTT for each data segment for which there is a corresponding ACK for the highest sequence # in the segment (+1), and (b) one with the min, max, and mean of these RTT values for each connection in the traces. The programs are in ~smithfd/src/delaybox/dummynet/{2way_prep.c, get_data_seq.c, get_ack_seq.c, data_rtt.c}. The whole process is done by the scripts, prepRTT and doRTT, run in that order (also found in the same directory). -------- Original Message -------- Subject: Re: Modified dummynet Date: Mon, 26 Aug 2002 00:09:04 -0400 (EDT) From: Long Le To: Don Smith Hi Don, I made the change that you suggested. The new file is in /usr/src/sys-altq/netinet/ip_dummynet.c on puck. The seed for the random number generator can now be read or set via the sysctl variable net.inet.ip.dummynet.dn_randseed. -- long