With attributes such as lower infrastructure acquisition costs and the ability to run on an existing IP network, ISCSI storage solutions should be particularly attractive to small and mid-size companies. When faced with the hot issue of an ever expanding need for storage capacity, however, IT decision makers have shown remarkable coolness toward early adoption of iSCSI. General satisfaction with the performance of existing direct attached storage (DAS) and networked attached storage (NAS), along with concerns over the level of expertise required to manage a SAN have combined to slow iSCSI adoption at SMB sites.

openBENCH LABS SCENARIO

UNDER EXAMINATION iSCSI Storage Server   WHAT WE TESTED StorMagic SM Series Storage Server •  One-button iSCSI management via SM Disk Manager •  Automated disk provisioning and migration to iSCSI storage •  Web-based GUI for robust management   HOW WE TESTED Windows 2003 Server SP2 QLogic QLE4052 iSCSI HBA nSTOR 4540 Array   Benchmark •  oblDisk •  IOmeter   KEY FINDINGS •  1,500 IOPS benchmark throughput (8KB requests using SATA drives) •  137MB per second benchmark throughput from one iSCSI target with two active-active connections •  Round-robin iSCSI MPIO load balancing based on StorMagic DSM provides multi-gigabit throughput

While both DAS and NAS technologies have served IT well in the past, the new focus on achieving higher levels of resource utilization introduces a new set of technologies that are dependent upon shared network storage rather than DAS. For any IT department, especially one at a small to medium enterprise (SMB), building a SAN for the first time will raise significant concerns over configuration and management complexity. Intent on demolishing old notions about iSCSI complexity, StorMagic™ created its SM Series of iSCSI arrays to radically simplify for IT all of the necessary SAN configuration and management tasks.

As an iSCSI storage appliance, an SM Series array is client OS agnostic. A Windows-based environment, however, characterizes the typical SMB site. StorMagic leverages this fact with a special software package—dubbed “SM Disk Manager”—for client systems running a Windows-based OS. For a system administrator who has little or no SAN experience, SM Disk Manger on a Windows server turns an SM Series array into an automated, self-managed, storage solution.

Strong growth in server virtualization is also spurning iSCSI adoption. VMware® Virtual Infrastructure (VI) simplifies the requirements for storage virtualization by eliminating the mandate to ensure exclusive ownership of disk volumes in a SAN. The VMware ESX file system (VMFS) eases this constraint by handling logical disks analogously to CD-ROM image files. As a result, advanced VI environments are designed specifically to leverage shared storage, which opens the door to using iSCSI as the transport mechanism of a very cost-effective, easy-to-manage SAN.

In assessing the StorMagic SM Series array, openBench Labs set up a testing scenario that would allow us to focus on the issues of functionality, manageability, and performance within the context of an SMB site. For an iSCSI client system, we chose a Dell PowerEdge 1900 Server with a quad-core CPU and a Gigabit Ethernet TCP Offload Engine (TOE).

System administrators working in a Windows OS environment have two choices for a GUI with which to work with the SM Series array. As with other iSCSI arrays there is an embedded web browser that is independent of the OS of the client system. This robust interface provides the means to precisely configure and manage all of the array components, including physical storage pools, logical disk targets, and iSCSI sessions.

For SAN connectivity, we installed QLogic FC and iSCSI HBAs. We chose to utilize an iSCSI HBA rather than server NICs in order to test the versatility and extraordinary ease-of-use in a Windows OS environment that StorMagic touts as central to the value proposition of their SM Series array for IT.

For the OS we ran Windows® Server 2003 and installed the MS iSCSI initiator, QLogic’s SANSurfer HBA Management GUI, and the SM Disk Manager. The SM Series array that was used by openBench Labs for testing was provisioned with eight 500GB SATA drives from Seagate and two Gigabit Ethernet ports for iSCSI connections. Optionally, the array could have been configured with four Gigabit Ethernet ports and up to twelve SAS drives.

For performance testing, we used our oblDisk benchmark to assess a RAID-0 pool on the SM Series array to test support for applications that use large files, such as digital content creation, editing, and distribution. We then used Iometer and a hardware enforced RAID-5 pool to test for the array’s suitability for use in a database-driven environment with a transaction processing application. While I/O throughput performance was impressive, it was iSCSI functionality and ease-of-management for the SM Series array that would steal the show. 

For administrators with iSCSI clients that run a Windows OS, the SM Disk Manager is a Windows-based client. SM Disk Manager greatly simplifies ISCSI configuration and management with an interface that is very similar to the presentation of DAS drives by Windows.

When configuring or managing the SM Series array, a system administrator can work with either the array’s full-featured, embedded, web GUI or install the Windows-based SM Disk Manager utility. While our test plan called for beginning the evaluation process with the more sophisticated web GUI, we installed the SM Disk Manager before starting out tests. That choice provided a serendipitous and important benefit: As part of the installation process, SM Disk Manager made the unique iSCSI qualified name (iqn) of our test server’s initiator known to the SM array. As a result, we did not have to make the initiator iqn known manually to the software running on the SM array. Mapping a client’s iSCSI iqn to an iSCSI array is a necessary and onerous task needed to make disk targets on an array available to the client system.

For seasoned storage administrators, StorMagic’s automatic registration of a system’s initiator iqn provides a welcomed relief from an annoying task, which many administrators have likely automated with their own cut-and-paste routines. On the other hand, thanks to automatic iqn registration, system managers installing their first SAN will avoid a very arcane task that would have reinforced any misconceptions that networked storage adds more overhead to system administration than it eliminates.

A key element of our test scenario strategy was to use the Microsoft software iSCSI initiator in conjunction with the QLogic hardware iSCSI HBA. The QLogic family of QLA4050C HBAs is designed to unburden a host processor from all of the overhead associated with the processing for both TCP and SCSI command packets. From the perspective of system overhead, the presence of a QLogic iSCSI HBA is little different from that of a QLogic FC HBA. What’s more, this dovetails with Microsoft’s software architecture for extending advanced I/O functionality.

We setup all iSCSI sessions with targets via the MS Initiator. For each target drive, we created two active sessions. Each session associated with a target exposed a unique device defined by a distinct NIC port on the SM Series array and a distinct port on the QLogic iSCSI HBA. As a result, we had two active independent parallel paths connecting each target disk with our host client. This gave us an outstanding fabric topology for round robin I/O request load balancing and failover.

The QLogic QLA4050C incorporates a TOE, which handles all of the TCP packet processing, and a SCSI command engine, which handles both command processing and CRC error checking, to provide a fully functional iSCSI HBA. This allows a QLA4050C to appear to its host operating system not as a network device, but as a disk controller via a QLogic storport driver. In turn, the QLA4050C provides the ability to boot a host system directly over iSCSI without additional software.

Complementing QLogic’s HBA, the Microsoft iSCSI software initiator package includes a kernel mode mini-port driver and an iSCSI initiator service. When installed with the QLogic iSCSI HBA, Microsoft’s initiator service uses the HBA while handling login and logout for all iSCSI sessions with storage arrays. The Microsoft iSCSI service also provides support for advanced SAN I/O functionality, including multipathing of I/O requests and load balancing, when there are multiple active session connections for a target device.

Version 2 of the Microsoft iSCSI Initiator provides a system administrator with two options for configuring automatic aggregation and load balancing of I/O over multiple active TCP connections: First there is the iSCSI protocol option dubbed Multiple Connections per session (MCS or MC/S) and second there is Microsoft’s Multipath I/O (MPIO). While both options will spread I/O request packets for a single application accessing a single logical volume across multiple TCP connections, the means to that end for each of the options is entirely different.

For the iSCSI protocol stack, MC/S is a new option that creates multiple paths in the iSCSI session layer. Both the host initiator and the storage system that provides the logical target must support MC/S within their respective iSCSI stacks in order to configure this option. Moreover, since MC/S is implemented within the iSCSI stack, MC/S is OS agnostic. Nonetheless, this is very disruptive for storage vendors to support and is particularly problematic for HBAs, which put the iSCSI stack in firmware.

While OS specific, the Microsoft MPIO solution is transport agnostic: MPIO works equally well with iSCSI, Fiber Channel, Infiniband, Fiber Channel over Ethernet (FCoE), or any other transport. To accomplish that task, Microsoft’s MPIO architecture requires storage vendors to develop a Device Specific Module (DSM) as part of the MPIO solution architecture. The DSM provides an interface between the Microsoft multipath driver package and the vendor’s
storage hardware.

The SM Disk Manager leverages the DSM to create a number of one-button provisioning utilities, including a migration utility. Using the SM Disk Manager, a system administrator simply selects an existing disk and invokes the Migration utility. The utility provisions a new logical disk from a storage pool on the SM Series array.

StorMagic leverages this multi-tier software architecture by tightly integrating all of the proprietary hardware and software that they have developed for the SM Series storage array with a Windows OS running on an iSCSI client system. That proprietary software includes a 5th-generation iSCSI stack for the DSM and SM Disk Manager, which is installed on the client and acts as StorMagic’s vehicle for integration.

In particular, StorMagic leverages the DSM to set up a default iSCSI configuration that features high-availability. The StorMagic DSM sets up an active-passive pair of MPIO sessions between a logical disk target and a host system. The sessions utilize distinct NICs on the storage array and share the host’s default iSCSI port.

The DSM is only aware of the host’s default iSCSI address, which makes an active-passive configuration the logical choice. A system manager can override the default high-availability MPIO scenario by configuring multiple active connections: The StorMagic DSM will automatically take advantage of those multiple active connections by instantiating a round robin load-balancing policy for I/O packets on all active sessions.

More importantly, StorMagic significantly extends the notion of providing advanced Windows MPIO configurations for iSCSI clients by providing system administrators with storage-provisioning utilities that also leverage the advanced capabilities of their DSM. With the simplicity of one-button control, a system administrator can provision an iSCSI target using the SM Disk Manager GUI more easily than a DAS volume.

The SM Disk Manager automates the entire iSCSI disk provisioning process. Starting with the allocation of space from a storage pool; adding the configuration of active and passive iSCSI sessions for failover; and finally formatting the new disk on the client; the SM Disk Manager eliminates any requirements for specialized SAN or iSCSI knowledge or experience.

Once the new disk is created, the utility creates a pair of iSCSI sessions, one active and one passive, to mount the new disk with a fail over policy. Next the utility copies the contents of the old disk to the new, dismounts both disks, and remounts the new disk using the Windows ID of the original. As a result, openBench Labs needed to provide absolutely no iSCSI information in order to migrate an existing disk onto a new highly available iSCSI disk.

That one-button disk provisioning function is further enhanced by StorMagic with a one-button disk migration feature. By selecting an existing disk, typically a DAS partition, a system administrator can launch a migration process, which will provision a new iSCSI target from an existing storage pool; configure two iSCSI sessions for high availability; format the new drive; copy all of the data from the original drive; and finally mount the new drive with the drive letter of the original volume. As a result, novice system administrator can use the SM Disk Manager to rapidly carry out a task that is typically reserved for a seasoned storage administrator.

The SM Series arrays are populated with Seagate Barracuda ES.2 drives. These drives utilize a perpendicular bit-recording technology to deliver capacities that range from 500GB to 1TB. Moreover, the Barracuda ES.2 drives are available with either a Serial ATA (SATA) or Serial Attached SCSI (SAS): A choice that is entirely transparent to the array’s DSM and SM Disk Manager. As a result, many SMB sites will view the availability of high capacity, low-cost SATA drives in an SM Series array as a potential economic tipping point in the purchase decision.

Complexity of electronics drives the cost differential between SATA and SAS. Drives that implement the SCSI command set, such as SAS and Fibre Channel, separate the processing of I/O commands from the mechanical positioning of the drive’s magnetic heads on distinct processors. Separate processors are needed to handle the tagged command queuing found in the SCSI command set. Using tagged command queuing, controllers re-order I/O commands. As a result, data access that is characteristically random can be made more sequential in order to minimize head movement and make significant gains in performance.

We used the SM Disk Manager to automatically provision a logical drive on our server. Running our oblDisk benchmark, pea I/O throughput reached 110MB per second for both reads and writes. Upon adding a second active connection using the second port on our iSCSI HBA, throughput was aggregated over two connections and exceeded the 1Gb per second limitation of a single connection.

ATA implements a simpler form of command queuing, which allows one processor to handle both commands and positioning. That’s why SATA drives have lower rotational speeds than SAS drives. These issues can be mitigated, however, with large multi-drive arrays. That strategy works well at SMB sites where price and rack density are valued equally with performance. That’s why IT pegs SATA-based storage arrays as an excellent fit for file server and streaming media scenarios and reserves SAS-based arrays for high-end database solutions. That underscores the SM Series family of iSCSI arrays with SM Disk Manager as a powerful alternative to DAS solutions with SCSI drives.

The SM Series array tested by openBench Labs was provisioned with eight 500GB SATA drives. To best leverage the performance of the SATA drives in this array, we tested two configurations. First we grouped all eight drives into a single RAID-0 storage pool, using a Windows-friendly 64KB stripe size. This provided us with a storage foundation featuring peak capacity and optimal throughput performance for clients. Second, for sites that require greater hardware resiliency, openBench labs configured seven drives in a RAID-5 set with the eighth drive acting as a hot spare.

We began performance testing with a RAID-0 storage pool. To examine streaming read and write I/O performance, we configured a single logical drive. We created this drive using the SM Disk Manager to handle the provisioning process automatically. The result was an iSCSI target backed by our 8-drive RAID-0 storage pool. Running our oblDisk benchmark, Read and write I/O throughput converged on a level of about 110MB per second, which represents iSCSI wire speed for a 1Gb per second connection.

Maximum I/O throughput using our StorMagic target drive was completely consistent with the default MPIO configuration that features a Fail Over Only policy setup by the StorMagic DSM. In particular, StorMagic first setup an active MPIO session from port 0 on the SM Series array to port 0 on the QLogic iSCSI HBA. Next, StorMagic setup a passive session from port 1 on the SM Series array to port 0 on the QLogic HBA.

To test the throughput scalability provided by MPIO load balancing, we created a software-based RAID-5 volume, which incorporated five logical drives sporting two active connections with round robin load balancing. Running the oblDisk benchmark on our dynamic disk volume, read performance jumped dramatically for all I/O block sizes compared to our initial RAID-0 volume. This was particularly true for small-block I/O which rivaled the small-block I/O throughput from FC drives spinning at 15000 rpm in our nStor FC array. As a test bed for the scalability of the StorMagic DSM, I/O throughput for reads and writes as well as total I/O operations at both ports of the QLogic iSCSI HBA showed that traffic was perfectly balanced during the IOmeter tests.

Using the Microsoft iSCSI initiator, openBench Labs next reconfigured the passive session with the target. We first switched the connection from port 0 to port 1 on the QLogic HBA and then made the connection active. With two active iSCSI sessions, the StorMagic DSM automatically reset the MPIO policy to load balancing with round robin I/O packet transmissions and throughput immediately exceeded 1Gb per second. In particular, measurable I/O throughput jumped by approximately 20 percent as read and write throughput converged on 126MB per second for 64KB I/O blocks.

The 20 percent boost in throughput that openBench Labs measured with active-active iSCSI connections and round robin load balancing has profound implications for throughput and the scalability of iSCSI sessions. More sessions using more logical volumes will spread I/O more finely across multiple connections. This phenomenon helps scale multiple applications and larger user populations.

We leveraged the scalability of iSCSI sessions to extend sequential I/O throughput for our oblDisk benchmark. In particular, we utilized the MPIO load balancing of the StorMagic DSM in conjunction with dynamic disk support in Windows Server 2003. We began by importing five distinct RAID-0 target volumes, each sporting two active sessions with round robin MPIO load balancing. We then used the dynamic disk capability of Windows Server 2003 to bind the five target volumes as a single software-based RAID-5 logical volume.

I/O performance testing with our RAID-5 volume built with multiple logical disks from our RAID-0 pool proved very enlightening. Running our oblDisk benchmark, peak I/O streaming for reads was about 10 percent greater than what we had measured using a single logical volume. Moreover, that peak streaming I/O performance was about 20 percent greater than that of a 2Gbps FC array, which was using 15,000 rpm FC drives. Equally important, throughput for small block reads, which typify most application I/O, was virtually identical to the FC array.

IOP rates achieved with our RAID-5 volume were more than sufficient to satisfy any SMB transaction processing application. With an 80/20 mix of read and write operations, the iSCSI volume sustained over 700 transactions per second.

That throughput edge is at least in part attributable to the extraordinarily well executed I/O load balancing performed by the StorMagic DSM resident on the SM Series array. All read and write I/O operations on our dynamic RAID-5 volume, which was built with five disk targets that had active-active round robin MPIO connections, were balanced perfectly at the two ports of the QLogic iSCSI HBA. As a result, we had both an iSCSI session with an effective 2Gbps connection and a logical disk that exhibited the rotational latency of a high-speed SCSI array.

Unlike digital-content applications, which stream large-block sequential I/O, transaction-based applications built on Oracle or SQL Server typically generate large numbers of I/O operations that transfer data using small—4KB to 8KB— blocks from random locations across a logical disk. To assess potential performance of the SM Series array in transaction-processing scenarios, we ran the Intel® open source Iometer benchmark.

The Iometer benchmark stresses data access as well as data throughput. In our tests, we fixed both the number of processes making read or write transactions and modified process characteristics by varying the number of outstanding requests that an I/O process was allowed to have open and continue issuing requests.

In many of the transaction-processing applications that are run at SMB sites, the number of processes involved in executing transactions is often limited to a few proxies. Microsoft Exchange, for example, utilizes a JET b-tree database structure as the main mailbox repository. All transactions are passed to an Exchange store and retrieve process, dubbed the Extensible Storage Engine (ESE), which creates indexes, and accesses records within the database.

For both read and write I/O requests, openBench Labs utilized an 8KB I/O block size on each Iometer benchmark test and recorded the number of I/O requests processed per second (IOPS). For a base comparison, we ran the same set of benchmark tests on a logical disk exported from an nStor 4540 FC array with FC drives configured as a RAID-5 storage pool. Finally, to better analyze Iometer results, we plotted the average number of IOPS as a function of the number of outstanding I/O requests, which represents the effective I/O queue length for the underlying array’s controllers and drives.

For SMB-class applications, the transaction processing level reached using our iSCSI StorMagic array with its hardware enforced RAID-5 pool of SATA drives was comparable in IOPS to an FC array with a RAID-5 pool of FC drives. That level of performance would easily satisfy the most demanding SMB application requirements. More importantly, in reaching that level of performance, the StorMagic DSM played a critical role as it balanced traffic quite precisely across multiple iSCSI HBA ports.

The results of the openBench Labs tests make it clear that IT at an SMB site is able to provision an SM Series array with low-cost high-capacity SATA drives without compromising application performance. Not only is the SM Series array ideal for handling data files sequentially, it is equally apt at optimizing targets for use with applications that feature high numbers of short transactions.