Automatic Evaluation of Intrusion Detection Systems

Frédéric Massicotte
Communications Research Center

François Gagnon
Carleton University

Yvan Labiche
Carleton University

Lionel Briand
Carleton University

Mathieu Couture
Carleton University

An Intrusion Detection System (IDS) is a critical
aspect of a network security posture. Although today
there are many IDS products available, it is rather
difficult to find information about their accuracy. Only a
few organizations evaluate these products. Furthermore,
the data used to test and evaluate these IDS is mainly
proprietary. Thus, the research community cannot benefit
from those data sets to test and evaluate the next
generation of IDS. Toward this end, DARPA provided in
1998, 1999 and 2000 an Intrusion Detection Evaluation
Data Set. However, no new data set has been released by
DARPA since 2000, in part because of the
cumbersomeness of the task. In this paper, we propose a
strategy to address certain aspects of generating a
publicly available documented data set for testing and
evaluating intrusion detection systems. We also present a
tool that automatically analyzes and evaluates IDS using
our proposed data set.

Keywords: Intrusion Detection Systems, Testing Data Set

Read Paper Read Paper (in PDF)