|
Department of Computer Science University of California, Davis One Shields Avenue Davis, CA 95616-8562 |
Dahlgren Division, Code CD2S Naval Surface Weapons Center 17320 Dahlgren Road Dahlgren, VA 22448-5100 |
The Intrusion Detection and Response Data Sharing Workshop, held at the University of California at Davis on July 15, 1998, had three goals. First, identify the needs of practitioners and researchers and identify opportunities for co-operation. Secondly, broker exchanges of research prototypes and better products to practitioners, and real incident data to researchers. Thirdly, provide funding agencies with a set of hard research problems for possible future funding.
Attending the workshop were:
After each set of three speakers, the workshop attendees reviewed the problems and offers each speaker presented. In some cases, the offers matched problems. Otherwise, the attendees noted the problem was potentially a hard one, worthy of future research. At the end of the workshop, attendees reviewed the unmatched problems to find any missed matches.
Stephen Northcutt reviewed his work at the Naval Surface Weapons Center. As a practitioner, he has developed a toolkit for intrusion detection because none of the available IDS meet his needs. Among his problems are that the tools are too expensive, miss many attacks, and have very complex, unwieldy interfaces (especially important since Stephen uses beginning security analysts to monitor them). Stephen can provide real data (with some sanitizing), as well as logs of new attacks and is willing to run others' IDSes in his DMZ and compare the results to his current tool suite.
Karl Levitt followed with a description of the intrusion detection and response work at the Computer Security Laboratory at UC Davis. Researchers are developing a large-area, scaleable intrusion detection system called GrIDS that uses graphs to visualize attacks. Also, Boeing is funding research into response based upon an adaptive automated response protocol called IDIP. This work raises issues such as when a response becomes a denial of service attack, and where a response should take place to minimize the effects on legitimate users. How can IDIP engines aggregate data from multiple sources? How one can determine the policy that policy-based IDSes enforce? The last is particularly important to systems using IDIP, as policy defines appropriate responses. Karl offered to make GrIDS available (but it is beta test, and not easy to install), and said that Boeing expected to release IDIP-based software when ready.
Angelo Bencivenga continued with his experiences at the U. S. Army Research Laboratory. His site had over 40 root compromises in one year, but the number of such compromises is decreasing. Given the nature of the work at his site, distinguishing the ankle biters from the very dangerous attackers is critical. Among his problems is the aggregation of data from multiple sources, a lack of tools for visualizing attack patterns, and an inability of tools to auto-report attacks. Angelo offered to try to provide access to his data, although (as with Stephen) legal requirements may hinder this.
Todd Heberlein, the developer of much of the original intrusion detection software used by the Department of Defense, reviewed the problems caused by advances in network technology and protocols. In particular, encryption (such as provided by IPSEC) foils content-based network monitoring systems, and high-speed networks cause monitoring tools to miss packets (there was considerable discussion about how fast "high-speed" was). Finally, Todd said that researchers need access to a complete set of known exploits that run on reference systems (although not all on the same reference system). Todd's current software, NetRadar, is available to the government.
Mary Walker, a member of Motorola's Space and Systems Technology Group, raised problems related to commercial and industrial firms: industrial espionage in particular, and cracking in general, concern her group. Her problems include funding, a need to help management understand how attackers work, and that no single product does all that needs to be done. Balancing privacy issues and security issues is critical in her environment. She is willing to help test new IDSes, and may be able to share her group's comparison of commercial IDSes.
Don Tobin, a student of Deborah Frincke at the University of Idaho, discussed co-operative IDSes across networked systems. Problems include the development of a formal model, lack of access to current attacks, the development of an interface suitable for naïve analysts, and the need to distinguish among the importance of various user requirements. His group is willing to make their experimental system, Hmmr (pronounced "hummer"), available.
Ulf Lindqvist of the Computer Science Laboratory at SRI, presented IDLE, a library of attacks designed for IDS tool builders. It supports frequent updates. Problems include obtaining data on attacks to populate the database; the data on attacker web sites is incomplete and/or of limited quality, and others rarely share this data. Further, IDLE stores data in XML format, so no query mechanism exists yet. Other problems are determining who should have access to the database, and whether the IDS community would trust one center to hold all data, or will expect several centers to share the burden? Ulf offered to share IDLE with any interested party once the database format was finalized.
Joe Thompson discussed intrusion detection at the Los Alamos National Laboratory, home of the NADIR system. He would like a set of example intrusion detection systems to test and evaluate. He is concerned that systems should provide secure communication between components and integrate network-based and system-based intrusion detection. Central to distributed intrusion detection systems is the issue of centralized or decentralized control; which is safer, which is easier to implement and maintain, and which is better according to relevant site metrics? Joe offered to run IDSes on his network and evaluate them with respect to their Java-based system. He provided pointers to an example of this Java-based system and to papers describing the system.
Stuart Staniford-Chen of UC Davis is leading the Common Intrusion Detection Framework (CIDF) effort for DARPA. CIDF is to facilitate interoperation of IDSes, and if needed may provide a data interchange format. SIDS are fields used to communicate information; CIDF provides some, but needs a larger set. Problems of interest to the working group developing CIDF are a need for a standard list of all known attacks, as well as an algorithm for calculating the severity of an attack, or indeed for what "severity" means with respect to a given policy. CIDF is on version 0.7, and still evolving; the working group is open, and Stuart invited any interested parties to join.
Matt Bishop discussed the vulnerabilities database work, and vulnerability research, at the Computer Science Laboratory at UC Davis. The database, known as DOVE, is to provide high-quality vulnerability and exploit information to the other projects at the lab, and to provide data for vulnerability research. It requires attack tools and information on attacks and vulnerabilities. However, the isolated network on which the raw data resides is cumbersome to work with, so attack tool testing is cumbersome. The policy for access is not final, but he expects to allow researchers access to the full database, and commercial firms and developers access provided they contribute something to the work. The first scheduled release of this data is at the end of summer 1998.
Rob Cunningham discussed the testbed and evaluation work at MIT’s Lincoln Laboratories, where DARPA is funding a program to test intrusion detection systems. Among the metrics are the rates of false negatives and false positives, both serious problems for practitioners. This also allows tracking a single IDS to see how it improves throughout the evaluation process, or between evaluations. The data used for the simulation is synthesized from network traffic collected at an Air Force base, analyzed, and similar data generated. This eliminated the possibility of leaking sensitive data from the Air Force traffic. Two weeks of data is available now; six weeks will be available soon. He asked whether researchers would be ready to test intrusion detection systems for Windows NT next year. Rob also presented a technique he called bottleneck verification, in which the number of ways to change state are determined and monitored. For this, he needed transcripts of what all types of attackers do (both the
Catherine Francis spoke about Security Dynamics’ work in intrusion detection. Their product, the Kane security monitor, is a host-based system designed to counter insider threats. It is driven by patterns, and training the pattern creator to detect previously unknown intrusions is an important problem, as is the rapid dissemination of patterns used to detect intrusions. Her group is also looking at appropriate responses and damage assessment. She also asked how important secure communications between IDS components is. She brought a set of CDs for the workshop attendees to evaluate, and offered to consider them for beta testers. She is willing to exchange information on intrusion detection products.
Phyllis Lee of the National Security Agency works in a group that tests intrusion detection tools (if the tool uses cryptography, they will also evaluate the quality of the cryptography) and performs penetration testing (both as attackers and defenders). Her group does not see intrusion detection as the most serious problem; they focus on securing systems. She is willing to share information within the Department of Defense, and possibly with others not in that Department, and is willing to test security-related tools on their testbed. They will give highest priority to tools that they think the Department of Defense would like to use.
Delores Quade, a developer, spoke about Network Flight Recorder’s products. NFR is a general-purpose network monitor. NFR selects the traffic based on a programmable filter language, and can analyze both packet headers and packet content. They need more intrusion detection filters and feedback from users; what do users most desperately need? Delores, and NFR, offer a free version of NFR (version 1.6.2, source code, and documentation) as well as white papers. The release of NFR version 2.0 is expected to be in late August or September 1998.
Scott Chapman, of Centrax Corporation, concluded the presentations with a talk on Centrax's product, eNTrax, an audit trail analysis tool for Windows NT. Among the problems they face are the data aggregation problem and presenting evidence obtained from the tool in court. How must the tool record and present information so courts will accept the information as evidence? Scott is currently funding research at UC Davis, and the Windows NT vulnerabilities that research develops will be added to DOVE, and through that medium made available to the community.
Some of these problems are very difficult to solve, and attendees did not attempt to solve any of them at the workshop. Attendees felt that some of their interests and work overlapped enough to lead to fruitful collaborations:
The attendees agreed that the following were problems worthy of future research:
The participants judged the workshop a success. Each participant left with a better understanding of what others in the intrusion detection community are doing. The participants identified several opportunities for collaboration; for example, the DOVE and IDLE efforts may help one another, and developers needing secure IDS communications learned of several efforts under way (including CIDF and other research). Indeed, some participants paired up at the workshop to help one another solve problems. Finally, the workshop attendees created a list of problems that practitioners feel need to be solved to move the field of intrusion detection beyond its current state.