THE SHADOW
KNOWS

For years, the private sector has tapped into the government sector for new technologies that can be reapplied for business use. Network-based intrusion detection Systems (NIDS) like SHADOW are good examples. The government-bred SHADOW, which stands for Secondary Heuristic Analysis for Defensive Online Warfare, is freely available and highly regarded as a weapon in the war against network attacks. http://www.nswc.navy.mil/ISSEC/CID/

A typical NIDS sits in front of the network, usually between the firewall and the external network such as the Internet, and sniffs all packets, searching for a pattern that resembles an attack. SHADOW takes a different approach and typically uses only the headers for traffic analysis. SHADOW is unaffected by whether or not the data is encrypted. SHADOW was developed at the Naval Surface Warfare Center in Dahlgren, Virginia. The SHADOW system detects suspicious activities such as network scans and probes, denial of service attacks and unauthorized connection attempts. First publicly introduced in 1998, SHADOW is especially noteworthy at a time when businesses are more wary than ever of network security risks. SHADOW also represents a strong argument for the merits of Open Source development process. Here to explain what SHADOW has accomplished, what’s ahead, and the Open Source role, is the  Information Systems Security Manager at the Naval Surface Warfare Center, Dahlgren Division, Fred Kerby, who chronicles the SHADOW group’s journey from its earliest beginnings.

As Bill Ralph, who programmed the software, tells its history, at that time little money was available for buying or building custom devices to examine traffic patterns. Stephen Northcutt, then at the NSWC (now director of the GIAC Training Program at the SANS Institute) began building sniffers from old discarded workstations and public domain software. By 1998, a number of SHADOW success stories occurred: First, SHADOW proved instrumental in the detection and identification of coordinated attacks; before long other divisions within the Naval Surface Warfare Center began implementing and using SHADOW; then other DoD programs began using SHADOW and providing helpful feedback. In 2000, we were one of seven recipients of the SANS Institute’s Security Technology Leadership Award.

Where does SHADOW stand today at the NSWCDD? As opportunities for improvement are identified internally or by external users, we prioritize them and work to get them implemented. Typical changes might improve security of the product or provide additional features or both. Version 1.7 was released last September, carrying improvements that provide a view of the site traffic from 50,000 feet. It identifies the top 25 computers inside the firewall that engaged in communication with computers outside the site, and it also identifies the top 25 computers outside the firewall with which the site exchanges information, which is very enlightening. If the analyst wants to see with whom the computer on the inside exchanged information, he can do a search and get the results returned in tcpdump format.

As for the next version, expect to see the top talkers represented as a link that the user can click to see with whom the information exchanges took place. When the user clicks on the link for one of the top 25 computers on the inside, he will get another page that produces a summary of the connections (computers and amounts of data) to computers outside the firewall. Security Officers are responsible for reviewing the statistics page to identify anomalous activity (why would a desktop client PC show up as one of the top 25 'talkers'? Is there probable cause for a given computer to be one of the top 25 'talkers' for a given day?) and advising line managers as appropriate. Because we provide the source code for the scripts that comprise SHADOW, users can customize them to meet their needs. If the analyst wants to see the top 20 or top 50 instead of the top 25, it's a fairly simple change.

At the NSWCDD, however, SHADOW is hardly seen as the magic bullet, and we use other additional tools to protect the site. NSWCDD is to partner with some non- DoD activities to either develop tools that can be used in addition to and in conjunction with SHADOW, as well as collaborating on improvements to SHADOW.

This continued work on SHADOW marks its key trait as software under continuous product improvement. That is why use of Open Source software has been a key element in SHADOW's success. SHADOW is based entirely on Open Source code such as Linux, tcpdump, perl, OpenSSH, and Apache. Because the code is freely available and runs on a low-cost platform, just about anyone with the desire and knowledge can install and operate a SHADOW system. SHADOW was originally built here on Red Hat Linux v5.0 and has been migrated to every subsequent version of Red Hat Linux.

No organization needs to remain in the dark about who or what is trying to communicate with their systems. All the same, while SHADOW’s strengths are impressive, the reality is that THIS IS no POINT-AND-CLICK experience. The software, while freely available, can be a challenge for the uninitiated to install and configure. Beyond downloading the program, businesses implementing any form of intrusion detection program will need to know how to interpret and respond to the information or alerts provided by the IDS.

The analyst must understand the underlying networking concepts to be able to interpret and understand the output from the intrusion detection system. The company that decided they need an intrusion detection system must train employees already working for them or can use contract help as well. We often point business people to the SANS site (SANS stands for System Administration, Networking, and Security) Institute, which is a cooperative research and education organization.

In today’s world, what every business person must understand is that security is an integral part of their doing business. Good security is good business.