Traffic Analysis
Overview
This module introduces some key tools for network traffic analysis. The ping and tcpdump tools are demonstrated and key options are highlighted as students capture traffic on a real link.
Setup Time: Varies
Tutorial Time: 30 minutes
Objectives
Upon completing this module you will:
- Be able to use ping to determine round trip time and network connectivity
- Be able to use tcpdump to capture and analyze network traffic
- Be familiar with other network analysis and traffic generation tools
Tutorial
A. Slice Creation and Instrumentation
This module assumes you have an active slice with two connected nodes and SSH terminals to both nodes. If you don't, follow the steps in the GENI Setup module and the Instrumentation module, then continue here. It is also assumed that the IP addresses and hostnames are assigned as described in the GENI Setup module (client=10.1.1.1, server=10.1.1.2). If this is not the case, you will need to substitute the correct IP addresses and hostnames below.
B. Video
If you haven't already, watch the video above. It will walk you through the steps of the module.
C. Ifconfig
1. On the client SSH terminal, type:
ifconfig
to list the node's network interfaces and the attributes and status of each. Make note of the name of the interface that is assigned the 10.1.1.1 address (i.e. “eth1”).
D. Adjust The MTU
If both your nodes are within the same aggregate, skip this section and continue to section E.
1. On the server SSH terminal, type:
ifconfig
to list the node's network interfaces and the attributes and status of each. Make note of the name of the interface that is assigned the 10.1.1.2 address (i.e. “eth1”).
2. A GRE tunnel is used to connect nodes from different aggregates. When a GRE tunnel is used, it is necessary to adjust the Maximum Transmission Unit (MTU) size for the interfaces. This can be done by issuing the following command:
sudo ifconfig <interface name> mtu 1400
where <interface name> is replaced with the name of the interface noted in the previous step (i.e. "eth1"). The MTU size for the interface is now compatible with the GRE link between the two nodes.
3. Repeat steps 1 and 2 on the client SSH terminal(for interface assigned address 10.1.1.1); however, be careful, as the interface name may be different.
E. Ping
One of the most used networking tools is Ping. Ping inherits its name from active sonar, where pulses of sound, called "pings", are transmitted and information is gleaned from their reflections. Similarly, the ping tool sends packets into the network and gleans information from their "echoes". Ping can be used to determine if another host on an IP network can be reached and what the round-trip time (RTT) to that host is.
1. On the client SSH terminal, type:
ping 10.1.1.2
This will begin sending ICMP echo request packets to the target host, which we have set as the host in our network named "server" with IP address 10.1.1.2. If/when the request packet arrives at the remote host, it generates an ICMP response and sends it back. Upon receipt of the ICMP response, the originating node calculates the time from transmission to reception (RTT). Lost packets are also detected and reported. Ping can do hostname resolution for you, so "ping server" would also have worked above.
2. Type Ctrl-c to stop pinging.
3. To verify ping isn't lying to us, let's take down the IP connection between the nodes. To do this, type:
sudo ip link set dev <interface name> down
where <interface name> is replaced with the interface name noted above (i.e. "eth1"). This command says "do" the following action as the superuser (root), call the "ip" Linux command for the "link" object, and "set" the given device or network interface "down".
4. Ping the server again from the client SSH terminal (can press up arrow twice to reuse previous command input):
ping 10.1.1.2
and note that there are no ping responses.
5. Type Ctrl-c to stop pinging.
6. Bring the IP connection back up using a similar command as before, but replacing down with up:
sudo ip link set dev <interface name> up
7. Use Ping again to verify that the connection is re-established.
F. Introduction to Tcpdump
Tcpdump is an extremely valuable tool for network analysis. It is able to capture and display information on packets entering and/or leaving a network interface. The following steps will introduce you to some of the features of tcpdump and give you hands-on experience with it.
1. On the client SSH terminal, issue the command:
sudo tcpdump -i <interface name> -nn
where "-i <interface name>" is the name of the interface on which to listen. This will be the same as the interface name used in the previous section (i.e. "eth1"). The "-nn" instructs tcpdump to use the numerical IP addresses rather than using DNS to resolve the symbolic addresses. It also instructs tcpdump to use the numerical ports rather than substituting them with commonly used protocol strings. You should see a few lines of text appear. Tcpdump is now ready to print information on network packets which enter or leave through the given interface.
2. Now, switch to the server SSH terminal and ping the client by typing:
ping 10.1.1.1
You should see something like the following printed in the client terminal for each ping:
10:58:37.662843 IP 10.1.1.2 > 10.1.1.1: ICMP echo request, id 7748, seq 3, length 64
10:58:37.762902 IP 10.1.1.1 > 10.1.1.2: ICMP echo reply, id 7748, seq 3, length 64
Each line of the output corresponds to a packet that has been sent or received on the specified interface. The first column contains the timestamp for the event, with microsecond precision. Next, the protocol is listed (i.e. ARP or IP). For IP packets, the source and destination IP addresses are listed, followed by the layer 3 protocol name and associated information. In this case, the attributes of the ICMP packets used by the ping tool are shown. Notice that an echo request is sent from 10.1.1.2 to 10.1.1.1 and shortly thereafter an echo reply is sent in the opposite direction.
3. Type Ctrl-c in the server SSH terminal to stop pinging.
4. Type Ctrl-c in the client SSH terminal to top tcpdump.
5. Restart tcpdump, but add "-S" to the options and "tcp" at the end as in:
sudo tcpdump -i <interface name> -nn -S tcp
The "-S" tells tcpdump to print absolute TCP sequence numbers rather than converting them to small sequence numbers that are easier for humans to read. This will allow us to see the raw sequence numbers in our initial analysis. "tcp" adds a filter to tcpdump such that only TCP traffic will be printed to the screen.
6. In the server SSH terminal, type:
nc -l 44444
This command uses the netcat tool to start listening on the given port for a connection.
7. In GENI Desktop, open another SSH terminal to the client. (Make sure client is selected and click SSH from the side menu).
8. In this new client SSH terminal, type:
nc server 44444
This command establishes a TCP connection with the listener on the server. Notice that tcpdump has now printed something like the following:
10:26:33.508856 IP 10.1.1.1.46520 > 10.1.1.2.44444: Flags [S], seq 1812506012, win 14600, options [mss 1460,sackOK,TS val 1375289021 ecr 0,nop,wscale 7], length 0
10:26:33.508939 IP 10.1.1.2.44444 > 10.1.1.1.46520: Flags [S.], seq 3211754307, ack 1812506013, win 14480, options [mss 1460,sackOK,TS val 1375289021 ecr 1375289021,nop,wscale 7], length 0
10:26:33.508958 IP 10.1.1.1.46520 > 10.1.1.2.44444: Flags [.], ack 3211754308, win 115, options [nop,nop,TS val 1375289021 ecr 1375289021], length 0
As before, the timestamp, protocol, and IP addresses are present, but an extra number after the IP addresses indicates the port number. In this case, a packet was sent from port 46520 on 10.1.1.1 to the port 44444 of 10.1.1.2. The remainder of the each line (after the ":") gives the details of the TCP packet. The TCP flags set, if any, are indicated within the square brackets ([]) after the word Flags. The S indicates a SYN packet and a period (".") indicates an ack. The rest of the potential attributes include "seq" for the packet's sequence number, "ack" for the acknowledgement number, "win" for the source's TCP window size, "length" for payload/data length, and several optional fields after "options".
What was just printed by tcpdump is the 3-way TCP handshake that initiated the connection. Notice that the payload "length" of each of the three messages is 0. No application data has been transferred, but a connection has been established. First, the client (10.1.1.1) sent a SYN (indicated by [S]) to the server (10.1.1.2), communicating its initial client-side sequence number of 1812506012. Then, the server (10.1.1.2) replied with a SYN/ACK (indicated by [S.]), giving its initial server-side sequence number of 3211754307, and acknowledging that all data before 1812506013 (one more than the server-side sequence number) has been received. Finally, an ACK is sent from the client (10.1.1.1) to the server (10.1.1.2) to complete the 3-way handshake, acknowledging that all data before 3211754308 (one more than the client-side sequence number) has been received. The TCP connection is now ready to be used.
9. In the client SSH terminal (the one not running tcpdump), type:
Hello World
and press "Enter". The netcat tool packages this input into a TCP message and sends it to the server, where it is unpackaged and displayed. Check tcpdump again, and you should see something like the following:
10:28:13.538448 IP 10.1.1.1.46520 > 10.1.1.2.44444: Flags [P.], seq 1812506013:1812506025, ack 3211754308, win 115, options [nop,nop,TS val 1375389051 ecr 1375289021], length 12
10:28:13.538530 IP 10.1.1.2.44444 > 10.1.1.1.46520: Flags [.], ack 1812506025, win 114, options [nop,nop,TS val 1375389051 ecr 1375389051], length 0
Notice that a TCP message with 12 bytes of data (length 12) was sent from the client (10.1.1.1) to the server (10.1.1.2). This is one byte for each of the 11 characters in your message plus the carriage return. Also, notice that the sequence number is now 1812506013:1812506025, indicating the range of sequence numbers for the data being sent. The "P" flag may be present. It stands for "push" and tells TCP on the receiving side of the connection to immediately push all buffered data to the application. Finally, the server responds with an ACK, acknowledging the receipt of the data.
10. Now, send a message in the opposite direction by typing it in the server SSH terminal and pressing "Enter". Verify that the sequence number and ack fields of the TCP messages printed by tcpdump increment as expected.
11. Type "Ctrl-c" to quit out of both net cat sessions and tcpdump.
G. More Tcpdump Options
This section will describe some more useful features of tcpdump; however, for a full treatment, you are encouraged to find a good online tutorial.
1. tcpdump is able to write what it captures to disk rather than just displaying it on the screen. However, care should be taken to honor privacy when using tcpdump. It is customary to only capture a certain number of bytes of each packet's header. This is done by setting a snap length of N. Only the data in the first N bytes is recorded by tcpdump. Restart tcpdump in the client SSH terminal, but use the "-w" option to write to a file and add a snaplength of 96 bytes with the "-s" option:
sudo tcpdump -i <interface name> -nn -S -s 96 -w <filename.pcap> tcp
The format of the resulting file will be the pcap (for packet capture) format. It is customary to give these files the extension ".pcap".
2. In the server SSH terminal, start the iperf server by typing:
iperf –s
The node can now receive iperf traffic.
3. In the client SSH terminal, start the iperf client by typing:
iperf –c server ‐t 1
4. After 1 second, the client should have completed sending its traffic. Type "Ctrl-c" in the tcpdump window and the server window to stop tcpdump and the iperf server.
5. tcpdump can display and filter previously captured pcap files. To display the first 10 packets of the file just captured, use the count ("-c") and the read ("-r") options, typing:
tcpdump -nn -r <filename.pcap> -c 10
Notice we no longer need sudo because we are reading the packets from a file. 6. Now, display only packets whose source is the server (with IP address 10.1.1.2). To do so, we add an "and" and a "src" attribute to the filter string, like this:
tcpdump -nn -r <filename.pcap> -c 10 tcp and src 10.1.1.2
This will display only the first 10 tcp packets whose source IP address is 10.1.1.2. In this way you can build up complex filters for your traffic. Many other filtering commands are available, but their descriptions are outside the scope of this module.
Going Further - Assignment
1. Use ping to find short and long RTTs. Ping various web servers (i.e. ping www.?.com, or ping www.?.co.uk, etc), and find the one with the longest consistent RTT. Why do you think the RTT is so long? Next, ping various web servers and try to find the shortest RTT to a server outside of the aggregate. (hint, try popular websites) Note, ping will not work for some web servers because they have been configured not to respond to ICMP messages, or ICMP messages are blocked. For those completing this module for credit, submit the longest and shortest times and associated URLs or IP addresses.
2. Repeat steps 5 through 11 of section F above, except use the "-u" option of both nc commands to use UDP instead of TCP and change the tcpdump filter to output only udp instead of tcp messages, as in:
sudo tcpdump -i <interface name> -nn udp nc -u -l 44444 nc -u server 44444
What is different about the resulting tcpdump output when UDP is used, compared with the output when TCP was used? Why are they different? Those completing the module for credit should submit a screenshot of their tcpdump output and the answers to the above questions.
3. ifconfig will show you that there is another network interface on your node. This is the connection to the outside world (i.e. eth999). Use tcpdump with snaplength of 96 to record to a file only the tcp traffic on this interface for about 30 seconds. Then, use tcpdump to read this file and output the first 5 packets from a source IP of your choosing. Those completing this module for credit should submit a screenshot of their final tcpdump command and the 5 packets output.
Shutdown
Upon completion of the module please delete your slice's resources as described in the Shutdown module.