In particular, Campbell noted that "Open source technology is today in embedded printers, our storage NAS product, HP-UX with Gnome, Linux affinity, Apache in multiple products." Adding fuel to the fire, Campbell warned that Microsoft was "specifically upset about Samba, Apache and Sendmail," noting "Samba [the key to NAS] is first."

As a sign of how seriously HP took his warnings, well-founded or not, you have to look for signs of the action plan. Implementation of the plan would render the original memo, in the words of Ms. Phillips, "not relevant today."

Curiously, just one product cycle following that call for an action plan, HP is rolling out a new line of NAS appliances. Anchoring that new line is the behemoth StorageWorks NAS 9000, which sports dual Xeon processors, 4GB of RAM, dual load-balanced gigabit Ethernet NICs, a hardware-mirrored system drive array, a hot-plug PCI-X backplane, remote hardware control over IP, not to mention fully redundant hot-swap mechanical components that exhaust any space that might have been available for storage components?

If that doesn't sound like your typical NAS appliance, welcome to the latest storage-market brouhaha. Along with the buzz from backup sans server to smart SAN servers, get ready to move from NAS and SAN confusion to SAN NAS fusion. The concept is simple: Create a LAN gateway to a SAN. The gateway benefits from the scalability and performance associated with block-level access on a SAN, while it distributes easy-to-use file-level data over a LAN.

On close inspection, the NAS 9000 turns out to be a densely packed version of the ProLiant DL580 G2,  a 4-way SMP server designed for environments requiring maximal compute power. That makes the NAS 9000 notably dense in everything except internal storage. So why turn a robust server into a NAS appliance?

The question is not as strange as it sounds.  Sites looking for a server that can handle all of the tasks surrounding very compute-intensive applications require a lot more than just raw, seething CPU power. More often than not, these sites are looking for guaranteed maximum uptime even more than a plethora of CPU cycles. As a result, the ProLiant DL580 is packed with high-availability features that make it an ideal choice for an enterprise-class storage server for which downtime can be a career-ending cost.

Nonetheless, a storage server is not exactly a storage appliance. The leap from compute server to storage appliance requires two very important software dimensions: acquisition cost and ease of use. By definition, an appliance is supposed to be cheap and easy. In the massively deflationary world of high-tech, Linux and BSD provide systems integrators and designers with a high degree of margin insurance. What’s more, the addition of a little Web magic from Apache provides the essential ingredient to mask all the command-line nastiness from many unsuspecting consumers for whom even pointing and clicking is a technical hurdle.

To regain lost ground in the IT-appliance space, Microsoft is offering vendors a deal that the software giant hopes no OEM systems vendor will be able to let pass. At the heart of this deal is a very low-cost OEM 'appliance edition' of Windows Server 2003. Storage OEMs get the first appliance edition, which is dubbed Windows Storage Server 2003 and targets headless devices—systems sans monitor, keyboard, and mouse. Here Gary Campbell's action plan takes shape as Microsoft provides the software bride for the HP's new line of NAS hardware in an intriguing marriage of convenience.

The broader goals for storage appliances using this OS include a quick—less than 15 minutes—setup, a minimum number of maintenance chores, and access to enterprise-level features via the OS and third-party software. To that end, services inherited from Windows Server 2003 include the creation of point-in-time copies of data via the Volume Shadow Copy Service (VSS). Further buttressing this assumption is the fact that there are are a lot more features bundled into Windows Storage Server 2003 than there are features missing from Windows Server 2003. The need to run third-party backup and storage management packages insures that this is indeed the case.

Gone are all of the event-logging policies of Windows Server 2003 that are designed to support the role of an application server. Policies such as the requirement to log the reason for a server shutdown are contrary to the role of an appliance. In their place is a new and somewhat troubling policy that forbids logging into the Windows Update service. The logic for such a policy makes sense in the context of a headless SOHO device. Nonetheless, in the context of an enterprise SAN gateway where both security and performance are paramount, such a policy raises troubling issues.

To test the HP NAS 9000, we inserted a dual-port Emulex LightPulse 10000 HBA into its backplane. Not only does this NAS appliance come with no drives, the system also ships sans SAN HBA. The choice of a Fibre Channel HBA is left to the site. The Emulex LP10000 is a dual-channel HBA device that combines ARM9 microprocessors with an advanced Fibre Channel protocol engine with full 133MHz PCI-X support. This enterprise-class HBA features a large data buffer, which supports cable runs of over 50km.

Drivers for the LP10000 are part of the standard Windows Server 2003 distribution. As a result, the LP10000 was immediately recognized by the HP NAS 9000 and we were immediately connected to the SAN. Included in our SAN were two QLogic SANbox 5200 stackable switches along with an nStor 4520 Storage System.

Equally powerful was our LAN connection. By having an OS based on Windows Server 2003, HP was able to include its Windows NIC teaming utility, which works with up to four NICs. Using the two Gigabit NICs installed on the server, we set up a team employing switch-assisted load balancing for both transmitted and received network packets. Alternatively, the team can be set up for load balancing of just the transmitted packets without switch interaction or in an active-passive configuration for fault tolerance. The utility creates an HP Network Teaming Virtual Miniport, which appears to the operating system as an additional NIC with a 2Gb-per-second.throughput rate. Once the virtual NIC is created, it is configured as if it were a real NIC and the actual NICs are never configured.

We were now ready to configure our physical storage volumes and network shares. Since we were using an nStor 4520 Storage System, we were not able to utilize one of the key extensions that HP adds to the Web interface: SAN array management. These WebUI modules apply only to HP storage arrays on the SAN. Nonetheless, we were able to install and host all of the SAN management software for the QLogic and nStor devices.

More importantly, there is also a USB-based Virtual Media feature which allows an IT administrator to use a standard 1.44-MB diskette or a CD on the client machine to install applications on the remote server or even perform a disaster recovery of failed system. With the rapid growth in the use of hosting sites, the iLO port and Light-Out Management software make this system, whether in the guise of the HP NAS 9000 or the ProLiant DL580 G2, a winner in terms of TCO.

From the OS perspective, the WebUI interface to the Volume Shadow Copy Service (VSS) is a highly touted feature for creating point-in-time snapshots—dubbed shadow copies—of volumes. VSS, which was introduced with Windows Server 2003, supports up to 64 shadow copies per volume and works at the block level. The process of creating a shadow copy is counter-intuitive when compared to a photo snapshot. If a snapshot is scheduled for 4 am, then the process halts at 4 am: It does not begin at 4 am.

From the point in time that a shadow copy ends, the next shadow copy begins by monitoring files. As a file’s block structure changes, the original blocks are saved in a cache. This means the files in a shadow copy only contain the original (pre-modified) blocks. If a file is unchanged, then it has 0 blocks in the cache. As a result, the scheme saves only enough data to maintain a consistent temporal view of the data. There is, however, an important drawback to this implementation: Previous versions of files and folders are only available over the LAN to clients running an updated version of Windows Explorer.

To test baseline performance, we mapped a 36GB LUN to the NAS 9000 system from the nStor storage array. For failover, this array was exported from both ports on the nStor server to both ports on the Emulux HBA. As a result, Windows reported this single drive as four different disks—one each logical connection. In contrast, when running on a Linux distribution built on the 2.6 kernel only one disk is visible. The alternate paths remain invisible and are transparently used to maintain a connection in the event of SAN link failure. On Windows, activating a new connection if the current path fails requires a reboot.

Once we had configured the volume, we then exported the volume as both a Windows and as an NFS share under two different aliases. Exporting the drive as a Windows Common Internet File System (CIFS) share was nothing if not trivial. Exporting the drive as an NFS share was nothing if not traumatic.

To simplify matters, we used an NIS server to deliver the required NIS domain information. This shortened the task of configuring a working NFS share down to two days. Compounding the lack of documentation, the system also provides no error checking or feedback. It is therefore very easy to configure a mapping structure that will simply fail to authenticate any and all users. This behavior is further complicated by the behavior of some Linux distributions, including SUSE, to behave as if the service simply does not exist. Only by using the tools supplied by Shaffer Solutions with DiskShare NFS client software for Windows were we able to confirm that the share did indeed exist.

After a considerable block of time was spent on experimentation, we diagnosed the problem as the inability of Microsoft Services for NFS to handle anything other than a 1-to-1 mapping scheme from Windows users to UNIX users. A UNIX user can be mapped to multiple Windows users but not the other way around. Unlike the configuration scheme in Microsoft Services for NFS, DiskShare, the NFS server for Windows from Shaffer Solutions, the explicit mapping feature does allow multiple UNIX users to be mapped onto the same Windows user. Why a many-to-one mapping from UNIX to Windows should make any difference is a mystery—or perhaps not if you take client-access licenses into consideration.

The problem we had encountered relates to the option under Microsoft Services for NFS to create an implicit mapping when UNIX and Windows usernames match. Apparently Microsoft Services for NFS fails to check if both implicit and explicit mappings are enabled, which can easily create many-to-one mapping relationships. Once we eliminated any explicit mappings to any Windows users that had an implicit map from an NIS user, our authorization problems were a thing of the past.

Having broken through the NFS mapping morass, we set up Windows XP and SUSE Linux 9.0 clients on HP Compaq Evo laptops to access both the CIFS and NFS shares, run our oblDisk benchmark, and access baseline performance. Our goal was not to stress the HP NAS 9000, but rather to determine the best client access strategy for utilizing it as a SAN gateway.

Each laptop was powered by a 1GHz P-III CPU and configured with 256MB of RAM. In our tests we ran oblDisk using a 128MB and a 512MB file. In the former case we expected to see the benchmark report throughput on reads that was greater than 11MB per second as a result of caching and throughput on writes on the order or 8-to-10MB per second, which would indicate that we were likely bottlenecked by the speed of our LAN connection.

We first ran our oblDisk benchmark locally on the HP NAS 9000 to calibrate the server headroom. Under optimal conditions with true asynchronous I/O requests on small files, throughput was on the order of 180-to-190 MB per second. With back-end access to the SAN at 2Gb per second and front-end access to the LAN at 2Gb per second, via HP NIC teaming software, the only issues remaining were overhead and throughput related to our choice of file-sharing protocols for clients.

We then reconnected our SUSE Linux laptop to the HP NAS 9000 using the NFS share created using Microsoft Services for NFS. Read performance leaped right to wire speeds as our benchmark locked onto a very consistent throughput of 11.2MB per second with no indication of any caching on reads. Write performance was a different matter that bordered on the bizarre. Using synchronous writes between the NAS 9000 and clients, which is the default, write throughput plummeted to a glacial 0.6MB per second!

As expected, performance on reads jumped significantly as it had obviously been bottlenecked by the LAN connection speed. More importantly, along with the 6X improvement read throughput, we measured a 5X improvement in write throughput. For many, if not most, workstation-class applications, an average I/O throughput of 68MB per second on reads will be more than adequate. Nonetheless, even with a 5X boost in performance, 19MB per second throughput on writes is hardly dazzling performance, especially if you consider disk-to-disk (D2D) backup applications.

Fundamentally, with the ROI arguments for moving storage onto a SAN continuously getting stronger, the merit in the case for SAN NAS fusion equally strengthens. What will shortly be of real interest in this arena will be the performance of enterprise-ready distributions of Linux on the 2.6 kernel, which are due to be released next month.