1. Throughput with avg_size=8 or alpha0.875:

2. abest when using exponential avg 0.875 and pstream length 360:

CDF of abest using the same data in the above graph but try different smoothing:

3. Sample pstreams:

pstream 235 with exponential smoothing:

pstream 235 with arithmatic smoothing:

pstream 350 with exponential smoothing:

pstream 350 with arithmatic smoothing:

Some Insights:

We notice that when probing rate is close to NIC capacity, the receive gaps in probe stream 350 seem to present a bimodal pattern. After collecting the sendgaps and recvgaps for all pstreams, the recvgaps actually show a bimodal pattern: big spikes followed by extreme low valley:

We also notice that if exponential smoothing is applied to pstream 350, the smoothed prune stream still keeps information about actual abest because the sawtooth patterns stablize at the rate 1Gbps. So we tried to apply a heuristic "spike to spile" averaging for pstream 350, and the recvgaps in the new smoothed stream are always larger than sendgap when probing rate exceeds available bandwidth:

Try this "SPIKE TO SPIKE" averaging for pstream 351 and 400:
For pstream 351:

For pstream 400:

Apply "SPIKE TO SPIKE" averaging for all pstreams and recalculate the AB_est:
****(Heuristics to use: if recvgap[i] is smaller than recvgap[i+1] by 50us, then spike begins from i+1. This spike ends when we find recvgap[j] is smaller than recvgap[j+1] by 50us. Then calcualte arithmatic averaging sendgaps and recvgaps between prune segments [i+1] to [j].)

Get CDF of abests for pstream 350-900:

Modify the spike-searching algorithm: spike_threshold=30us and do exponential or arithmatic averaging after spike-to-spike-averaging. And then re-calculate abests and the results are followd:

Some sample pstreams:

The first plot is original send gaps and receive gaps; in the second plot are gaps after spike to spike averaging; in the third plot are gaps processed with spike to spike averaging and then arithmatic averaging with avg_size=8; in the fourth plot are gaps processed with spike to spike averaging and then exponential averaging alpha=0.875
Pstream 350:

Pstream 351:

Pstream 400:

When ptream length= 80

1. CDF of sendgaps and recvgaps

2. CDF of AB_est with different smoothing algorithm:

3. Some sample pstreams:

1st plot: original sendgaps and recvgaps
2nd plot: pruned gaps with arithmatic smoothing avg_size=8
3rd plot: pruned gaps with spike to spike averaging only
4rd plot: pruned gaps with spike to spike averaging and arithmatic smoothing (avg_size= 8)
5rd plot: pruned gaps with spike to spike averaging and exponential smoothing (alpha=0.875)

Pstream 1286:

Pstream 3461:

Pstream 7930: