Subukan Handbook

I. Getting Started

1. Introduction

1.1 Objective

"A simple, easy to administrate, yet powerful network traffic sensor platform."

Subukan is built by Security Engineers, for Security Engineers with the goal of providing a product that will function not only as a state-of-the-art Network Intrusion Detection sensor platform, but also provide the additional functionality Information Assurance professionals and alike demand in order to excel in there respective capacities. Like many NIDS solutions, Subukan is good at detecting suspicious network traffic. However, in addition, Subukan provides the the very important cohesive research and correlation capabilities needed by Security Engineers when confirming or validating possible malicious network traffic. This helps in keeping within the scope of the Security Analyst's life cycle (detect, confirm, correlate, and report). Simply put, the Subukan sensor platform is designed to better equip Information Assurance professionals with the right tools needed to make sound judgments, accurate recommendations, and to adequately facilitate their network security and monitoring mandates.

In addition to the application capabilities of Subukan, we also aimed at developing a platform that functions more like an appliance, that is easy to use, easy to administrate, easy to upgrade, and can be scaled on an enterprise level. The sensor software is base on "Firmware" image technology which means the Operating System and supporting applications exist on a single file allowing for painless, even remotely executed upgrades. This and other capabilities are accomplished by providing Security Engineers with an easy-to-use information and research console or Web-Graphical User Interface (WebGUI). The WebGUI offers both administrative and traffic analysis functionality for the operator, seamlessly allowing the instant cross referencing of various informational network gathering resources to help the operator better understand the nature of the network traffic in question. If you need more than one sensor, Subukan's client/server topology also allows for sensor clustering on an enterprise level.

Product Highlights

Using a unique compilation of Open Source applications, coupled with custom software solutions; below you'll find a few of the highlights in the Subukan Sensor's capabilities:

• Packet-level data capture (TCPdump)

• Signature-based intrusion detection (Snort)

• Network traffic patterns (IPAudit)

• Packet Anomalous Statistical Tracking (PAST)*

• Appliance OS - Remote firmware image upgradable

• Unified sensor policy configurations updates

• Dual-purpose roles [Sensor / Collector]

• Web-based data retrieval research

• IDS Signature rule repository

* Note: PAST is not yet available and is still in the development stages.

1.2 The Subukan Model

Subukan is built by analysts, for analysts. The Subukan network attempts to address four fundamental areas:

• Detection

• Acquisition

• Presentation

• Administration

1.2.1 Detection

The cornerstone of any NIDS network is malicious traffic detection. Subukan implements the Snort intrusion detection engine. Snort is an open source network intrusion prevention and detection system utilizing a rule-driven language, which combines the benefits of signature, protocol and anomaly based inspection methods. Analyzing all the data transmitted on the network Snort recognizes known exploits and malicious traffic patterns. Furthermore Snort's signature-based detection is supported widely among the professional and Open Source communities.

1.2.2 Acquisition

Once malicious or anomalous traffic has been detected, analysts need a way to acquire the report data and corroborating information such as packet-level data and system logs. Subukan solves this need by allowing quick and easy access to the data collection resources on the sensors. Upon detecting an alert, Subukan allows the analyst to use a drill-down strategy to acquire amplifying data such as host to host traffic patterns and even dig down to packet-level data for immediate confirmation.

1.2.3 Presentation

Subukan uses a web-GUI front end to present alert data to the analyst. This simple and interactive tool allows analysts to quickly identify, acquire, and verify suspect traffic. Strategic hyper-linking adds convenience and significantly reduces the effort in tracing down questionable traffic anomalies. Instant data and snapshots can be generated for email correspondence and reporting purposes.

1.2.3 Administration

Subukan attempts to fill the gap where many Open Source IDS solution seem to fall short. For example, Snort administration has been historically difficult in that much of the configuration data is stored in several textual files. Furthermore, if managing more than one Snort system, updating each configuration can be daunting as one would typically need to access or upload these files to each individual system. Subukan implements a graphical interface and configuration libraries to automatically generate sensor policies which can be push out to multiple sensors. An entire cluster of Subukan sensors configuration can be virtually changed at the same time.

System upgrades are extremely easy in that Subukan implements an appliance-like architecture. The entire operating system and applications are distributed as a single binary image or 'Firmware' which can even be upgraded remotely. Updating one file updates the entire system.

1.3 Subukan Network Overview

A Subukan IDS network typically consists of two types of servers known as either the Sensor or the Collector. Sensor nodes are one or more systems which monitor the traffic data along strategic points along the network, while the Collector act as a central administration and data collection hub to manage a cluster of sensors.

The sensor's purpose is to monitor and capture network traffic and traffic patterns. Some of the data captured is cached locally on the sensor while some data is propagated to the Collector depending on configuration options. Sensors are remotely controlled by a central Subukan Cluster Collector receiving both configuration policy updates and data collection jobs.

The Collector houses all the configuration policies for the sensors. The Collector serves as a central administration console to remotely manage a cluster of one or more sensors. The Collector offers a web-based user interface used to synchronize multiple sensor configuration policies within cluster of sensors. It also provides a central analytical interface for researching and analyzing the traffic captured by all the sensors. Additionally the Collector can be configured with greater storage capacity to the purpose of archiving data feeds from the sensor cluster.

1.4 Roles

The Subukan distribution image software is dual-role oriented. Therefore the system can be configured as either a Sensor or a Collector, or as both (standalone). Furthermore the roles are dynamic in that they can be switched at any time for added redundancy in that any sensor can take on the role as Collector if needed. The three supported roles are as follows:

• Subukan Cluster Sensor

• Subukan Cluster Collector

• Subukan Standalone Sensor

1.4.1 Subukan Cluster Sensor

Configures the system to function as an IDS sensor within a cluster of two or more sensors nodes which will be remotely managed by the Subukan Collector.

1.4.2 Subukan Cluster Collector

Configures the system to function as a centralized administration and data console for a cluster of sensors. Choose this option if you have one or more Cluster Sensors and would like to remotely manage and/or replicate data to a centralized system known as the Collector.

1.4.3 Subukan Standalone Sensor

Ideal for small networks for which you only have one physical system available and you only plan to access and manage a single IDS sensor on your network. This option simply combines the functionality of both the Sensor and Collector on a single system.

1.5 Configuration Strategies

Determining which Subukan image roles to choose depends largely on your network's total design layout. Whether you need a cluster of sensors or can simply have a single standalone sensor is based on your monitoring objectives and capability. It's a complex challenge. For example let's examine several types of network scenarios:

1.5.1 Small

Let's say you have a relatively small home/office network. These usually contain a single subnet for all networked nodes and have only one DMZ (network router/hub to your Internet provider). A single Subukan Standalone Sensor might work best by allowing you to provision a single machine to function as both the sensor and collector. However, from a security standpoint it might make more sense to provision another machine to act solely as the collector and place this machine inside your network.

1.5.2 Medium-to-Large

This is for networks with multiple subnets, routers, and/or DMZ's. Typically this scenario fits better with the Subukan cluster model. Obviously you will want to implement multiple cluster sensors, placing them at strategic points in your network. Additionally you will need to establish a cluster collector. If for some reason some sensors cannot communicate with the same collector, you might consider multiple collectors or running standalone sensors at those locations. Please see 'Collector Placement' section for more information.

When it comes to choosing the specific roles and total number of Subukan machines, it boils down to the culmination of network size, type, equipment availability and finally personal preference. The flexibility of the three Subukan roles can provide you with plenty of latitude to achieve your information assurance goals.

1.6 Subukan Hardware Requirements

1.6.1 Sensor Hardware Requirements

When choosing a system to function as a sensor you will want to consider the volume of network activity to monitor. You will want enough processing power to handle your network bandwidth. You will also want a system that has at least two network interface adapters (possibly three if using a A B taps), one adapter dedicated to monitoring the subject network while the other is used for data and configuration access to the sensor. Finally, and hard drive storage space should be considered large enough satisfy data retention needs for the packet-level data spool and alert history. Actual length of this history is dependent on hard drive space, network volume, and packet capture mode (partial or full).

1.6.2 Collector Hardware Requirements

The collector's hardware demands may be slightly less from that of a sensor. The collector only needs one network interface port for data communications to the sensors. Ironically the collector doesn't require the high resource demand as of the sensors. If you plan to replicate and archive sensor data you may consider a large enough storage array, but for sensor administration and policy deployments a basic web server setup is sufficient. However, if you plan to duel-role a machine such as a standalone sensor, you should model towards the Sensor Hardware Requirements.

1.7 Sensor Placement

It is highly recommend that Subukan sensors and the collector be placed on a separate (private) network or VLAN which is not connected to the network you are monitoring. Insure the systems you choose to use as your IDS sensors have at least two network interface ports, one for monitoring traffic and one for sensor data communications to the collector. The sensor's 'monitor' port will be connected to the network you are to monitor while the 'data' port is connected to an isolated LAN. Note: it is possible to connect the sensors data and monitor ports to the same network, but for security reasons we advise against this.

Topology speaking, the IDS sensors should be placed in strategic areas where the most 'visible' network traffic can be seen. This usually means placing sensors in the DMZ where all network traffic is funneled to and from your Internet service provide. Ideally you will want to place the sensor inside the DMZ, just after the firewall. This will allow you to monitor what is actually getting into your network from the outside.

Physically speaking, you will want to connect the sensor's 'monitor' port to a network tap device which sits inline on the network and allow bi-directional packet frames to be echoed to the tap port. Additionally most routers support a span port which is a special port configured 'span' all traffic for both ingress and egress traffic. The sensor's monitor port will be configured in read-only mode and most network taps and switches make the span port read-only as well, therefore the need for a one-way or passive network cable is not necessary. Recommend using network tap devices or consult your switch/routers instruction manual for details on how to setup your router's span or mirror ports.

1.7 Collector Placement

Since the collector only has one network interface port it can reside totally off the monitored network but should be located within your network, preferably on a private LAN or VLAN. All sensors within the cluster should be able to connect to the collector as remote admin data will be sent to and from the individual sensors.

1.8 Upgrading

Maintaining an underlying operating system can add a great deal of administration requirements. Simply tracking and maintaining OS patches and updates alone can add an enormous amount of time on top of the application updates. Subukan addresses this by combining the applications and OS into a single binary image or firmware transforming the actual sensor hardware into an appliance. This strategy eliminates the need to maintain application or OS level patches and updates. Administration now becomes as simple as applying a single file or firmware update to the system. Best of all this update can be performed remotely in most cases saving valuable hands-on time.

II. Administration

2. WebGUI

TODO : explain the WebGUI purpose

2.1 Firmware Images ' Upgrading Subukan

The Subukan System is actually a complete platform. In short, both the Operating System and the supporting third-party applications are distributed on a single binary file referred to as the Firmware System Image. Replacement Image files can be easily uploaded to the system, made active thus effectively upgrading the system. Because the Subukan master configuration is stored separately, upgrading the system can be virtually transparent and in many cases only requires a single reboot and you're done. This process mimics the very way most hardware routers are upgraded today allowing eliminating the need for remote hands administration.

Under 'System -> Firmware/Config' section, the WebGUI allows you to facilitate these tasks by providing the ability to remotely upload, remove, and select which Firmware Image to actively used upon next boot-up sequence. The total number of Images that can be stored is only limited by storage space available on the system's primary partition, however only one Image file can be made active. Also note that a system reboot is mandatory when switching Image files.

2.2 Master Configuration File

Subukan uses a single file (subukan.conf) to store all of the system's internal configuration settings. Each time you make a change to the system a backup copy of the previous configuration is automatically created. This is useful in the event that you would need to roll the system back to a previously known configuration state.

The WebGUI provides the ability to manage the archive of past configuration files. In the 'System -> Firmware/Config' section, lower half, you will see the active configuration file along with a list of past configuration states if the were any. You may view, re-activate, or permanently delete any of these past configurations. There is also the ability to view and edit the active configuration. However, it is highly advised to never edit the raw configuration directly as this may irreversibly render the system inoperatable thus requiring a complete reinstall. The edit feature is there merely as a capability.

2.3 Physical Volumes

Subukan's WebGUI section 'Storage -> Physical Volumes' provides the ability to report hard drives and other storage volumes. The devices are presented in a tree style display which quickly allows you to identify the number and capacities of physical storage devices detected on the system. It also enables you to partition and initialize volumes as necessary. This is rarely required aside from the likelihood of removing or adding storage devices to the system. Mostly this section functions as an informational area.

2.4 Virtual Volumes

Virtual volumes are comprised of multiple physical volumes or actual hard drive partitions. The purpose is to usually combine the capacity of several drives into a single device node. If you have multiple drives on your system that are independent (not hardware RAID capable), then it might be feasible to combine these drives to take advantage of total capacity, redundancy, or perhaps a combination of both.

Virtual volumes can be configured using the system's WebGUI, although the option is also available during installation. From the 'Storage -> Virtual Volume' section you simply select from a list of detected available partitions followed by which type of Virtual mode (combine, mirror, or both), each mode having distinct advantages and disadvantages. The new volume is given a name serves as its device node identifier. If configuring the Spool to use a Virtual device, then this name will then appear in the Spool setup option box.

2.5 Spool Volume

Subukan implements a first in first out (FIFO queue) logging system known as the Spool storage volume. The spool strategy has been implemented due to the enormous storage capacity and availability requirements necessary to continuously monitor and capture network activity. The Spool provides temporary storage for the network monitoring logs generated by the network monitoring services such as ipAudit and tcpDump trace files. Therefore, the Spool is sub-divided into multiple 'Reels' dedicated to each of these network monitoring services. Each Reel is then allocated a portion of the total Spool volume. In the event that a Reel reaches its designated storage limit, older logs are automatically purged to make room for newer logs. The longevity of the individual logs depends on many factors such as the overall capacity of the Spool volume, the allocation between Reels, and amount of network bandwidth being monitored to name a few.

The Spool volume is usually setup and configured during the initial Subukan installation process. The WebGUI then provides the ability to configure or change the Spool device, as well as monitoring the health and usage of the Spool. Adjusting the allocation of Spool space between Reels is available as well. However, it is advisable to leave a small percentage of space left unallocated to serve as 'slack' space in the event that any given Reel momentarily overlaps its size limit during log rotation.

The Spool device itself can be any valid storage medium such as a physical volume as in a hard drive partition, hardware RAID device, or a local SANS cluster. The Spool can also be a designated Virtual storage device comprising of multiple hard drives in which can be merged into a large volume or mirrored for added redundancy (aka software RAID). You will need to have a Virtual volume properly setup and established before it will be available as a Spool device.

Whether physical or virtual, insure it is the largest available local storage device possible in order to maximize the amount of total historical network data available on the system. Also be aware that several critical system processes are dependent on the Spool volume including the network monitoring and capturing facilities. It is important that the Spool be setup and functioning properly or else these processes will automatically be not available. Most notably are several CRON processes which govern the traffic and packet capturing log services.

2.6 Network Interfaces

The WebGUI 'Network -> Interfaces' sections allow you to define and configure the network interface adapters available on the system. To configure an interface you simply add a new interface, define its network properties and then finally set the 'role'. The role setting helps Subukan identify which interfaces require special setup configuration throughout the system.

2.7 Firewall

Subukan has the ability to firewall itself. This acts as an enhanced security measure for those who desire added protection. If enabled the firewall can deny or restrict access to various services such as WebGUI, FTP, and ssh remote console access. This can be based on individual IP or subnets. Furthermore, the option for user-defined rules is available if the built-in rules are not sufficient. Beware when enabling or configuring the firewall options via a remote connection to not inadvertently deny access including your own remote connection. If this occurs, depending on what services are locked out, you may require local console access to reset the firewall via the subukan CLI.

2.8 ____

3. Subukan CLI

todo...

III. Common Tasks

todo...