The previous chapters presented a complete methodology for reproducing the traffic observed on a network link in a closed-loop manner, and proposed a number of metrics for studying the realism of the generated traffic. In this chapter, we study ways to introduce statistical variability in synthetic traffic in a meaningful and controlled manner. In addition, we address the need for changing offered load in network experiments. The methods that we introduce in this chapter add significant flexibility to our traffic generation approach, enabling researchers to perform a wider range of experiments.
In the approach presented so far, traffic is generated according to a trace
. Each augmented connection vector 
is replayed starting at time 
. This implies that two different replays
of 
using the same hardware and the same physical network
result in very similar synthetic traffic. In both cases, the synthetic traffic
has the same number of TCP connections, replaying the same source-level
behaviors under the same network-level parameters, and starting exactly at
the same times.
Only tiny variations would be introduced on the end-systems by changes in clock
synchronization, operating system scheduling and interrupt handling,
and at switches and routers by the stochastic nature of packet multiplexing.
This reproducibility was exactly what was needed to evaluate how well synthetic traffic
approximated the real traffic from which it derived.
However, in the practice of experimental networking, experimenters often want to introduce more variability in their experiments. One way of accomplishing this is to use more than one trace in a replay, exposing the studied network protocol or mechanism to different types of workloads. This is highly desirable, but it has its drawbacks. First, the experimenter may want to perform a number of experiments that is larger than the number of available traces. Second, traces from different sites, and even traces from the same site but collected at different times of the day, may be so different that it becomes difficult to extrapolate from the results of the experiments.
A different, and complementary, approach is to conduct several experiments using traffic that
``looks like'' some specific trace , without exactly replaying over and over.
The first challenge in devising a method for
accomplishing this task is to define what ``looks like'' mean. This involves creating
a model (either formal or informal) of the traffic which is general enough to contain
but specific enough to always resemble the original trace.
Running different experiments then requires to instantiate this model several times to
create new derived traces
,
and to generate traffic with these new traces using their source-level trace replay. In this chapter, this instantiation
consists of resampling the set of connection vectors in and
assigning them new start times. Statistical variability in the derived traces comes from the resampling
of the original connection vectors, and from the process of connection start times.
We preserve the statistical properties of the original set of
connection vectors by resampling entire connection vectors, i.e., we do not manipulate the
sizes and order of the ADUs and inter-ADU quiet times inside connection vectors.
Our belief is that a trace created by modifying the source-level behavior of the connection
vectors or their network-level parameters ``does not look like'' the original trace.
For example, doubling the size of the ADUs in is an easy way of
creating a new trace and increasing the offered load.
However, the resulting connection vectors
have little to do with the connections observed in the link from which
was collected.
Our choice to maintain connection vectors intact is reasonable, and consistent with the
spirit of our overall methodology, which goes to great lengths to accurately characterize
the source-level characteristics of each connection.
Other researchers may have a different mental model of the legitimate level of statistical
variability which could be introduced in
We propose
a specific solution and demonstrate that it is reasonable using quantitative data. A
discussion of the different philosophies is outside the scope of this work.
The two sections in this chapter describe two techniques for introducing variability
in the source-level replay of a trace. Section 7.1 describes
Poisson Resampling.
This technique assumes that connections are independent of each other, which
is a reasonable choice for highly aggregated traffic.
Poisson Resampling involves randomly resampling the connection vectors in
in an independent manner to create a new
. New start times are given
to each resampled connection vector in a such a way that connection inter-arrivals
are exponentially distributed.
As we will show, empirical data support the choice of exponential inter-arrivals.
Section 7.2 describes Block Resampling. This technique involves resampling blocks of connection vectors, preserving arrival dependencies among the connections inside the same block. Each block is the set of connections observed in an interval of fixed duration (e.g., 1 minute) in the original trace. We will demonstrate that this technique, unlike Poisson Resampling, preserves the long-range dependence in the connection arrival process found in real traces. This cannot be achieved by sampling independently from an exponential (or any other) distribution. Note that we will reserve the term resampling for randomly selecting connection vectors, and the term sampling for randomly drawing values from a parametric distribution, such as the exponential distribution.
The second topic of this chapter is how to manipulate a trace to
modify the traffic load (i.e., average byte throughput) that this trace offers
during its source-level replay.
This is a common need in experimental networking research, where the performance
of a network mechanism or protocol is often affected by the amount of traffic to which
it is exposed.
For example, active queue management mechanisms have very different performance depending
on the level of utilization of the input link, so researchers generally perform
experiments with offered loads that range from 50% to 120% of the link bandwidth.
Rather than trying to find or collect traces with the exact range of loads needed
(which is generally difficult), we propose to produce a collection of resampled
traces with the intended range of offered loads.
Average load 
is defined as the total number of bytes injected in the network 
divided by the total duration of the experiment 
.
Changing the average load in an experiment of constant duration therefore implies
creating a scaled trace
with a higher or a lower total number of bytes.
Once again, the assumption is that it is possible to create a scaled trace
which ``looks like'' the original
but with a larger or smaller number of bytes. This requires a model of traffic that is general enough
to encompass and traces derived from with different offered loads.
As should be obvious, the problems of introducing
statistical variability and changing average load are related, and can naturally be treated
together, as we will do in this chapter. The two techniques mentioned above,
Poisson Resampling and Block Resampling,
provide the foundation for deriving scaled traces. In both cases,
the resampling of to create a scaled
can be
modified to achieve a target average load. This means that our scaling method
is predictable, which is an advance over earlier traffic generation methods,
e.g., [CJOS00,LAJS03,SB04]. These earlier methods required
a separate experimental study, a calibration,
to construct a function coupling
the parameters of the traffic generator and the achieved load.
For example, web traffic generators usually require a calibration to discover the
relationship between average load and the number of user equivalents.
The scaling methods presented in this chapter eliminate the need for this extra study.
Their starting point is the observation that the average load
offered by the source-level replay of
is a deterministic function of the total number of bytes in the ADUs of .
We will show that these observation holds true using numerical simulations
and testbed experiments. In contrast, the same analysis will demonstrate
that the average load offered by the replay of is not strongly correlated
with its number of connections.
In the case of Poisson Resampling, our method to construct a new trace
with a specific target offered load involves resampling
until the desired total number of bytes (coming from ADUs) is reached. In the case
of Block Resampling, constructing a new trace
with a specific target
offered load involves subsampling blocks (``thinning'') to decrease
load, or combining two or more blocks (``thickening'') to increase load.
Doctoral Dissertation: Generation and Validation of Empirically-Derived TCP Application Workloads
© 2006 Félix Hernández-Campos