Why Packet Headers
One effective framework for traffic measurement is TCP/IP packet header collection, and organization of the headers into TCP connection flows. The framework has been in place throughout much of the short history of the Internet, and important fundamental work has arisen from it. TCP connection flows provide a large amount of information both across the Internet and on the collection wire.

Across the Internet
Each flow is an end-to-end connection traversing the Internet. The TCP/IP headers contain the IP addresses of the two computers, so we know their location in the vast Internet topology. Thus flows can be used to study network-wide characteristics.

On the Wire
A TCP connection flow database also provides information about the traffic on the wire. The TCP/IP headers have the size of each packet in bytes, so together with the timestamps, we have the aggregated packet process: the arrival times and sizes of all packets. Studying aggregates is important because the devices at each end of a wire must handle packets, in time order, and the performance of the devices depends on the packet inter-arrival times and the packet sizes. Forming the aggregate of all packets from the flows takes us back to the packet information in its original state: packets in time order. But storage by connection flow is still important because we often study sub-aggregate traffic: time-ordered packets from a subset of the flows. For example, each flow results from an application such as HTTP, FTP, SMTP, or Telnet requesting a connection and transfer of information; it is important to study aggregate traffic by application because the packet processes for different applications are different. We can also study derived processes formed from any sub-aggregate. A common one is byte counts; time is divided up into intervals of equal length, and the number of bytes of packets arriving in each interval is computed.

Collection on MHWire1
We capture all packet headers on the wire that connects a Bell Labs Research network of about 3000 machines to the rest of the Internet. The wire is located in Murray Hill, N.J. Collection began on November 18, 1998 and has continued through today, January 1, 2001, on a continuous basis except for monitor down time. The collection, the database organizing, and the analysis is carried out in S-Net, the system for packet header collection and analysis. Our current database consists of packet 12 billion packet headers for 600 million TCP connection flows.

Collection on Helios Wire1
As part of the Helios Next Generation Internet project, a major packet header collection effort has been carried out on the 1 gb/s Ethernet link connecting the Chapel Hill campus of the University of North Carolina to an OC48 fiber ring that carries UNC traffic to other local campuses and to the rest of the Internet. The ring is part of the NCNI gigapop. Our current database consists of 42 hours of collection on this link, 7 six-hour collection intervals during a single week, chosen to reflect traffic during high and low loads.