Infrastructure and Communications Architecture

Introduction

The rapid pace of technological change makes it likely that any static IDS (Intrusion Detection System) will rapidly become obsolete. As vulnerabilities are exploited and then patched, and as fundamentally new systems replace some (but not all) older "legacy systems", new holes and attack scenarios will undoubtedly emerge.

In this context, Intrusion Detection is best viewed not as a static set of tools and techniques, but rather as a dynamic, evolving, emergent process . Moreover, any successful IDS that is widely deployed will, itself, drive the evolution of more sophisticated attacks.

Thus, to have any reasonable longevity, an IDS must be based on an architecture that is extremely open and readily adaptable not only to new attack modes, but also to new tools and components for defense and detection.

Architectural Design Considerations

Infrastructure must be scalable .
Arch. must be flexible for experimentation and tuning.
It must encourage synergistic cooperation between relatively independent IDS components. This implies fully dynamic reconfiguration capabilities.
Authenticate dynamic reconfiguration commands.
Store evidence of possible intrusion off-host for safety.
Tradeoff: minimize both network traffic and on-host disc/CPU usage.
Data reduction and deletion must occur somewhere.
Want efficient distributed "database" operations:
- Insert, Delete (tag data or datatypes for persistence - Time To Live)
- Search (queries from GUI or from intermediate analyzers)
- ReConfigure (move/add data archives and new data source/analysis components)
IDS configuration within an enterprise must be constrained by both implicit and explicit security/privacy policies within that domain. In particular, balkanized organizational dynamics make it unlikely that the historic IDS paradigm of a single omniscient observer will be acceptable.
Thus, intra- and inter-enterprise IDS cooperation must tolerate limited degrees of trust between heterogeneous security domains (e.g., when policies are largely incompatible, there should be graceful degradation in cooperation across organizational boundaries).

Inter-Operability of Data Source & Analysis Components

SCL = Simple Communications Layer

IDS components should not need to know anything about other cooperating components. Instead, all routing of data between IDS components should be handled by routing Agents on each monitored host (or shared filesystem).
All data handled by IDS Routing Agents has syntactic form:
( fieldname[1] value[1] fieldname[2] value[2] ... )
Routing controlled by dynamically configurable rules , e.g.:
FOREACH incoming_record
IF ( defined(fieldname[x]) AND value[x] == 'y' OR
(value[x] >= 'A' AND value[x] <= 'Z') )
THEN route incoming_record to destination D;
Advantages:
- Supports flexible and extensible IDS topology
- Allows arbitrary log records
- Sufficient structure that records can be processed and routed easily
Tradeoff: Not imposing a universal semantic model on IDS data makes it easy to plug in new data source components; however, that shifts the problem of semantic interpretation upstream to data analysis components.