Lab 5. Result Analysis

To analyse the results of a simulation we need to

• run the simulation multiple times (see previous lab)
• for each simulation run obtain the metrics which are important in our study of the network's performance
  • such as throughput, delay, packet loss
• perform a statistical analysis of the obtained results by calculating the
  • average (mean or median)
  • confidence intervals.

We will be working with sixth.cc, which is a minor modification of fifth.cc (which we worked with previously). If you're interested in automatically creating plots from the simulation results, look at seventh.cc which shows how to use the GnuplotHelper object.

Throughout this instruction the PDF version of the ns-3 tutorial is referenced because it contains numbered sections. However, you may want to use the HTML version if you wish.

Task 1. Code analysis [1 point]

Analyze the difference between fifth.cc and sixth.cc. A detailed explanation can be found in Section 7.3.6 "Using Mid-Level Helpers" of the tutorial.

What code is responsible for having the congestion window values and dropped packets stored in separate output files? How can you open these files to analyse the results?

Task 2. Basic statistical analysis [4 points]

Perform five independent simulation runs of sixth.cc and calculate the average TCP congestion window size in each run.

Calculate the mean, median, and the confidence interval for the obtained average window values. Are there any outliers?

Task 3. Calculating performance metrics [5 points]

Perform three simulation runs of sixth.cc for three different values of the error rate of the channel. Choose unique error rate values based on your index number. Generate plots of the three basic metrics: throughput, delay, packet loss (e.g., average throughput vs error rate). Note: do not provide plots which are a function of time (e.g., instantaneous throughput vs time) -- we are interested in a metric's average value calculated over the whole simulation time.

To calculate the performance metrics I recommend one of the following methods:

• Wireshark - has a GUI but requires clicking,
  • Tip: calculate average RTT instead of delay by creating a TCP Delta column, filtering the 10.1.1.1-->10.1.1.2 flow, exporting to csv and calculating the average of the delta column.
• FlowMonitor - an ns-3 module designed for extracting results from a simulation. If you have trouble integrating the code look at examples/routing/simple-global-routing.cc or examples/wireless/wifi-hidden-terminal.cc.

Report card