Map-tools allows quick and easy visualization of very high-dimensional data. In particular, it has been applied to detect spatial anomalies in network backbone traffic.
This page shows example visualizations of network traffic data, and descriptions of the tools used to generate them. Most visualization of networks is based on connectivity, rather than similarity of the traffic that travels between them. Further, traffic tools for detecting (and displaying) anomalies typically watch for temporal anomalies. This tool can be used also to detect and display spatial anomalies.
N. Patwari, A. O. Hero, and A. Pacholski, Manifold Learning Visualization of Network Traffic Data, in Proceedings of the 2005 Workshop on Mining Network Data (MineNet'05), Philadelphia, PA, August 26, 2005, pp. 191-196.
Slides from the lecture, "Network Visualization," made by Neal Patwari to CS 5480, Computer Networks, also provide some motivation and background into the network data visualization problem, along with an introduction to manifold learning.
Map-tools is a set of C-code and bash-script utilities for command-line processing of NetFlow data. There are several tools used to process NetFlow data into sensor map visualizations. The flow of the several tools is shown below.
The code is freely available for download. Download and install proceeds as follows.
$ cd < directory where nplib is to reside > $ mkdir nplib $ cd nplib
$ gunzip nplib.tar.gz
$ tar xvf nplib.tar
$ make all
Program Flow Chart:
The code was developed in part using Will Naylor and Bill Chapman's WNLIB subroutine library, which is a free, unrestricted ANSI C subroutine library.
Example Usage of Map-tools
$ cat Thursday, January 6, 2005.1755.f9.S1.sdat | spl2dist > temp.dst
$ cat temp.dst | wmds -n 11 -K 5 -p fourWeekJanAvg.f8.S1.K5.r10-3.crds -r 0.001 -w loess -ND > temp.crds
$ cat temp.crds | coords2eps -n 11 -m fourWeekJanAvg.f8.S1.K5.r10-3.crds -z -c abilenePrior.conn > temp.eps
$ cat Thursday, January 6, 2005.1755.f9.S1.sdat | spl2dist | wmds -n 11 -K 5 -p fourWeekJanAvg.f8.S1.K5.r10-3.crds -r 0.001 -w loess -ND | coords2eps -n 11 -m fourWeekJanAvg.f8.S1.K5.r10-3.crds -z -c abilenePrior.conn > temp.eps
See the .man files associated with each command, included in the map-tools code download, for a detailed description of all command-line options.
NetFlow data was collected from January 2 to January 29, 2005 from the 11 routers in the Abilene backbone network. Sample visualizations are given at http://www.ece.utah.edu/~npatwari/mnd05/.