The demand for storage capacity continues to surge today, fueled by the growth of the Internet and a rising demand for online access to data. The ability to move and access data has become a mission-critical capability for every enterprise, and there is great pressure both to improve data management and reduce total cost of ownership (TCO). The industry is responding with storage networking technology that will add three significant capabilities to corporate network architectures. First, companies will be able to architect shared storage pools that can scale seamlessly. Second, they will be able to greatly reduce restore and backup times by leveraging online archives and new types of secondary storage media. Third, they will enjoy far greater manageability of data because they will have control of their databases from a single central point. The combination of these three things ­ scalability, much faster recovery and backup, and manageability ­ will inevitably reduce TCO of data storage in the near future.

These benefits make a revolution in data storage inevitable ­ in fact, it is well underway. Storage networking, whether network attached storage (NAS) or storage area network (SAN), has become essential to mission-critical IT environments. It is the dominant trend in data storage. Gartner estimates that end-user expenditures on storage networking solutions will exceed $65 billion in 2003. For database managers, the burning question now is: How can we get the greatest return from a major investment in storage network architecture?

This brings us to the debate about the relative value of NAS and SAN technologies. A NAS system is essentially a plug-and-play storage solution based on industry- standard networking architecture and attached to a local, metropolitan or wide area network. Its inherent design eliminates server I/O bottlenecks on heterogeneous networks by providing cost-effective storage and file sharing. On the other hand, we can describe a SAN as the vision of an open, scalable, fibre channel architecture that interconnects storage systems, backup devices and servers. It absorbs LAN traffic to improve network performance. Figure 1 highlights the prime differentiator between NAS and SAN technology.

In NAS, a single central file system resides in the architecture of the NAS appliance and controls data storage. In a SAN, the file systems are part of the operating systems that reside on multiple servers. The existence of multiple servers implies that data must be stored in a variety of file systems and formats, which makes true data sharing a challenge. NAS appliances, on the other hand, communicate with different operating systems environments (e.g., UNIX and NT) to achieve true single-copy data sharing.

Figure 1 also explains why NAS is a reality today and SANs remain a vision. NAS, which is based on standard protocols, connects to the LAN, based on reliable standards developed over the last 15 years. On the SAN side, the interoperability technology between different servers and storage units remains largely undefined as the standards and protocols are still being developed.

Figure 1: Prime Differentiator Between NAS and SAN Technology

However, both NAS and SAN solutions offer significant performance advantages over traditional server- attached storage in a networking environment. Some see them as competing, mutually exclusive or orthogonal; but it has already been pointed out in this article that SAN and NAS are complementary, interoperable and that each has its place in corporate network architectures. The view that they are orthogonal is outmoded. To understand where they're going, we have to examine another trend in storage networking technology: convergence.

Destined to Converge

It is now clear that the distinctions between NAS and SANs will ultimately bring them together as parts of a single solution. NAS and SANs, like all networking technologies, have architectures that include challenges, although NAS technology is clearly meeting important customer needs. Here are some of the issues affecting NAS and SANs:

Where does this leave companies that need to improve the TCO of their business solutions, including their IP infrastructures, and are in search of the ideal storage networking solution? They will drive the convergence of NAS and SANs for these reasons:

Trends in networking technology also support the concept of NAS/SAN convergence. Technologies are beginning to borrow from each other fairly heavily. Ethernet achieved gigabit speeds by leveraging gigabit fibre channel. When fibre channel reaches 10GB speeds, it will do so by leveraging the work that led to 10GB Ethernet. We can expect this kind of convergence to apply to the different types of physical media used today for storage networking, whether they involve storage-oriented protocols or network-oriented protocols. They will ultimately merge into a single physical infrastructure from a networking standpoint.

Another significant technology trend that favors NAS/SAN convergence is the rapid rise in networking speed as depicted in Figure 2. Network speeds are now surpassing storage speeds, blurring the lines between NAS and SANs from the standpoint of underlying technology. Customers will soon begin to focus more on the value proposition of storage networks and worry less about GbE versus fibre channel architectures.

InfiniBand exemplifies the way technology is helping to drive this convergence. Infiniband is being designed to take advantage of well-established existing media such as the enormous dark fibre networks already in place. It's already clear to the companies driving the InfiniBand standard that InfiniBand must be congruent with NAS/SAN convergence. InfiniBand is being designed to accommodate it ­ which, in turn, is helping to drive the convergence.

Finally, a recent industry development that favors convergence is the open standards initiative of direct access file system (DAFS) by a consortium of companies (see www.dafscollaborative.org). DAFS is a file system protocol based on virtual interface (VI) standards that has been fostered by Intel in recent years. DAFS eliminates storage I/O latencies by using direct memory access (DMA) operations between servers and storage memory architectures. DAFS will blur the lines between NAS and SANs even further because it is aimed primarily at the storage network environment and is agnostic to GbE or fibre channel mediums.

The storage networking technology built around this convergence trend will contain both NAS and SAN technologies in the same product. Any other approach would be less than optimal because companies are not likely to buy two products when one will do. Picture this solution as an intelligent box. On one side are storage technologies and storage protocols to manage disk systems. On the other side are network-oriented protocols for delivering the data from storage to the outside world, including database servers. These same network protocols are used to provide high-speed access to a common automated tape library. This is true cost-effective NAS/SAN convergence, and it exists today.

OSN: Catalyst for Convergence

The most powerful force driving NAS/SAN convergence, however, is open storage networking (OSN). OSN is based on the concept that linking pools of storage with networking technologies is essential to giving companies the data they need whenever they want it, with perfect data integrity. It is our conviction, based on considerable experience, that companies will soon be making it an absolute requirement that these integrated NAS/SAN solutions be delivered as open industry standard solutions. The OSN design concept allows companies to construct best-of-breed data access and management solutions that readily integrate legacy equipment, as well as new and emerging storage and networking technologies. By implementing solutions that are truly open, companies fully leverage existing management tools, staff, knowledge and equipment resources while benefiting from the best offerings of a large, mature and diverse base of suppliers. Organizations can focus on their business problems and opportunities ­ not on evaluating, testing and integrating new technologies.

The storage network of the future will be based on open standards ­ open protocols, open hardware technologies and open software technologies ­ that are plug-and-play interoperable and can be combined to match the customers' needs. The breathtaking ascent of far faster networking technology and rising volumes of data demand it. Networking technology is expected to unify the LAN/MAN/WAN environments by 2003. By focusing so clearly on leveraging existing networking technology, OSN delivers reliable, affordable access to any type of data anywhere, anytime.

Figure 2: Rapid Rise of Networking Speed with NAS/SAN Convergence

This open architecture facilitates the convergence of NAS and SANs. Customers will be able to adapt storage resources to drive applications to greater levels of scalability, performance and data availability. Data availability, in fact, may well be the critical issue that moves many companies to adopt best-of-breed solutions that include NAS and SANs in an OSN environment. InfiniBand will also drive NAS/SAN convergence and the adoption of OSN. It will be adapted to OSN and enthusiastically embraced by customers.

The industry is now using OSN to enhance the connectivity layer of SANs and combine them with the power of NAS to deliver an improved storage-over-a-network business solution. The solution architecture spans both NAS and SAN hardware and software technology, and incorporates a powerful data management capability to eliminate the shortcomings of both SANs and NAS. The result is mature SAN management functionality, leveraging existing standards such as simple network management protocol (SNMP). The solution, which is commercially available today, uses OSN as the only acceptable standard to integrate best-of-breed multi vendor technologies that dramatically reduce TCO and protect customer investment.

To Sum Up

File and Block Virtualization Convergence

Wedding these two complimentary approaches

By Christine Taylor Chudnow

In its broadest sense, storage virtualization is nothing new. It has deep roots in computer virtual memory and networking technologies like host-based logical volume managers. But with the growth in storage networking, virtualization became quite a bit more complicated. It may be limited to single servers and arrays or expanded to domains and even full SANs. It may affect disk-based arrays or tape libraries, and virtualization engines can be based within the SAN or outside of it. Virtualization also differs according to file or block-level virtualization procedures: although all computer data rests on blocks, virtualizing files adds different management elements.

Sorting the Blocks

All computer data is basically made up of blocks, units of information that move quickly between servers and their storage. Many applications make sense out of blocks, creating files with added definitions and file headers, which in turn requires more sophisticated processing at the server and storage levels. Virtualizing file-based data requires file systems that can interpret the packets and store file metadata, while virtualizing block-based data works closer to the raw disk levels. Block-level storage virtualization distributes blocks of data to virtual storage pools across multiple storage devices, which helps to manage space and control devices at the hardware level. File-level virtualization manages the stored objects—the files—which may be scattered around different physical storage devices.

SANs are ideal for sharing block data over open systems networks such as Fibre Channel or iSCSI. Block-level virtualization generally refers to aggregating storage devices in a storage area network, increasing storage space to applications while increasing flexibility and ease of management. Block-level virtualization technologies often include data-protection technologies such as replication, snapshots, and local and remote mirroring. Since the SAN offloads these computer-intensive operations from the local network, this results in better storage consolidation and improved storage resource management (SRM). Physically consolidating storage helps control over-provisioning and improves ROI, while simplified SRM allows storage administrators to efficiently manage shared block storage as a single volume. By separating storage management and allocation from the physical hardware and specific application servers, storage administrators can manage and control escalating storage costs.

Although SANs are by nature shared storage environments, virtualization does not automatically follow. Even in a storage area network, there are technical limitations to servers sharing their storage devices over Fibre Channel or IP. Servers still communicate with their storage devices using SCSI protocols and must meet certain SCSI requirements. For example, servers do not want to be confronted by a large pool of amorphous storage. They expect to see specific targets with addresses containing a target ID and logical unit number (LUN). In addition, some hosts will grab any LUN they can see, regardless of what the storage administrator meant to do. Such hosts may also compete for the same device, unwittingly overwriting each other’s data. Because of this, storage administrators commonly zone their Fibre Channel switches to block other servers from seeing the same devices. A related approach is called LUN masking, which renders certain LUNs invisible to other servers even in the same zone. This keeps the storage intact and secure, but whittles away at the SAN’s reason for existence: the ability to share storage among application servers.

Virtualization

To get around these limitations, block virtualization presents virtual layers that appear between the servers and physical storage devices. Actual blocks may be stored across different storage devices, while the storage administrators create virtual devices by virtually partitioning a single disk, or aggregating multiple disks to widen the storage pool. The servers no longer see (and try to grab) specific physical targets, but instead “discover” logical volumes for their exclusive use. The servers send their storage directly to the virtual volumes, happily thinking they are their direct-attached property. In fact, these logical volumes are highly flexible.

For example, Fujitsu Softek’s Virtualization application, which is built on a DataCore engine, builds a transparent layer between the application server and storage devices. The virtualized layer shows the application server a set of devices optimized for its needs. Meanwhile the virtualization engine maps the virtual devices to actual physical devices. As is common with these types of virtualization schemes, the engine also uses advanced caching, local and remote mirroring, and snapshot capabilities to optimize and protect the virtualized storage.

File-level virtualization requires global distributed file systems, or what passes for global file systems: proxy-like file systems that push data through a centralized server. These file systems process file requests from different operating systems and translate them into common commands for the target storage device. File virtualization (which can work in DAS and NAS as well as SAN), provides a virtual file system layer between files and block level storage. This virtual file system allows the administrator to manage files on virtual volumes, while in fact the files are scattered across different physical devices. Note that simply using a distributed file system does not equal virtualization—for example, NAS appliances and clusters often use such file systems to allow different applications or operating systems to access the same files. But if the global file system additionally allows the storage administrator to create and administer a storage pool, it is a virtualization technology.

File- and block-based virtualization are complementary, and it’s not unusual to wed the two in converged environments. Two such implementations are virtual file systems in SANs and storage clusters. Block-based virtualization is already big on the SAN, and competition rages around where to locate the virtualization engine: single hosts or arrays, single-vendor or multi-vendor domains, or SAN-wide implementations. (The question there is in-band or out-of-band virtualization engines.) But SAN-based file serving suffers from a unique circumstance: SAN application servers are limited to a single storage device, represented by a unique LUN, so file virtualization systems must create file-based storage pools by striping multiple LUNs into a single logical volume.

Some SAN-based virtual file systems already exist, including Sistina’s Linux storage cluster software. More storage companies are developing virtual file systems for multiple operating systems, which would allow users to store their files on the SAN as well as easing file management issues for storage administrators. For example, IBM’s Storage Tank project would virtualize enterprise storage by adding an installable file system to storage system clients such as Windows 2000, Linux, and Unix. Files enter the SAN from the IP network, where Storage Tank logs metadata information for file attributes and locations, and enables file locking.