IDSanalyzer

Overview

IDSanalyzer is a tool designed to ease the process of evaluating an Intrusion detection system. It incorporates traditional evaluation metrics such as the ROC with expected cost parameters, and incorporates the new evaluation method, B-ROC. It is written in Java, and should be capable of running on just about any computer with an installation of Sun's Java SE 1.4 or higher.

IDSanalyzer reads in ROC data plots and allows the user to specify additional parameters such as the probability of an attack or the cost of a missed detection. This data is processed, and can be viewed in the form of a variety of metrics.

Table of Contents

• Screen Shots
• Downloads
• Tutorial
• To Do
• Outside Links
• Contact Information

Screen Shots

• The Emerald Dataset displayed as B-ROC curves
• Same as above, but zoomed in
• The Emerald Dataset displayed in Robust ROC mode with custom parameters set

Downloads

Current Release:

• IDSanalyzer 1.0 (May 17, 2006)

Sample Data:

• Bayesian IDS ROC data, from:
  C. Kruegel, D. Mutz,W. Robertson, and F. Valeur. Bayesian event classification for intrusion detection. In Proceedings of the 19th Annual Computer Security Applications Conference (ACSAC), pages 14–24, December 2003.
• Columbia ROC and Emerald ROC data, from:
  J. E. Gaffney and J. W. Ulvila. Evaluation of intrusion detectors: A decision theory approach. In Proceedings of the 2001 IEEE Symposium on Security and Privacy, pages 50– 61, Oakland, CA, USA, 2001.

Tutorial

To use IDSanalyzer, first load a ROC data file (You might want to have a look at the sample data files listed above-any data file you create must be formatted the same way).

Data points imported from the file may be edited, added, and removed using the controls in the lower left corner of the window. The chart should update to reflect the changes immediately.

The tab bar along the top of the screen allows you to change the displayed evaluation method and its associated parameters. For more information on the available parameters, consult A Framework for the Evaluation of Intrusion Detection Systems.

The chart view may be adjusted by a variety of means. There are three buttons along the lower right side of the chart, zoom in, zoom out, and zoom to full, respectively. Using the mouse, you can move the view around by clicking and dragging (when not zoomed all the way out). Double-clicking on a point in the chart will zoom in on that point. Additionally, holding shift, then clicking and dragging will allow you to zoom in on a rectanglular area in the chart.

You can print the currently visible chart or save it to an image by choosing the appropriate item from the "File" menu.

To Do

Major Features:

• Support for display of multiple datasets to aid in comparing IDSes
• An XML-based input file format
• In-program help and documentation (Basic documentation is currently available in the tutorial section of this page)

Minor Features:

• An about screen
• An icon for the program

Known Bugs:

Currently, there are no known bugs in IDSanalyzer

Other:

• Prepare a Sourceforge page
• Make source available on Sorceforge page under GPL

Outside Links

IDS Analysis

• A Framework for the Evaluation of Intrusion Detection Systems
• B-ROC Curves for the Assessment of Classifiers over Imbalenced Data Sets

Related Organizations

• HyNet: The University of Maryland Hybrid Networks Center
• The Institute for Systems Research
• The University of Maryland

Contact Information

Email addresses are obsfucated to deter spam bots. For example, "bob at company dot com" translates to bob@company.com.

• Karl Seamon, primary IDSanalyzer and web page developer: kks at isr dot umd dot edu
• Alvaro Cardenas, graduate student: acardena at umd dot edu
• Dr. John Baras, Director, HyNet: baras at isr dot umd dot edu