In their efforts to simplify life for their customers, the heavy hitters of the storage industry are heading for complicated waters. Tim Stammers reports.
Blessed by the relentless growth in the volume of data that its customers must store, the storage industry bounced back rapidly from the dot-com implosion, and is now one of IT's fastest growing sectors. But for the same reason, storage is now also cursed to live in what might be described as 'interesting times.'
The mantra of IBM's storage division reflects this perfectly. Andy Monshaw, general manager of IBM's storage division, says: "Our strategy has evolved from a focus on price and feature-functionality to get us back into the market four or five years ago, to the on-demand computing approach we have now."
The market re-entry that Monshaw is talking about happened when IBM returned to strength in the market for high-end storage hardware in 2001, when its flagship Shark disk arrays at last became competitive with the best that EMC had to offer. Together with EMC and Hitachi Data Systems, IBM is now part of the triad of suppliers that dominates the market for these expensive monolithic storage devices.
Joe Tucci, EMC president and CEO
But high-end disk arrays incorporating sophisticated data replication functions are no longer the principal area of competition among big storage suppliers. On their own, they no longer solve the problems that businesses face when applying essentially flat IT budgets to the task of maintaining swelling volumes of data, while meeting increasingly sophisticated requirements for access and protection. The ballooning of data is very real, and shows no sign of slowing. IDC estimates that the capacity of worldwide external disk arrays shipped last year was up 64% on 2003.
"CIOs are being faced with tremendous complexity in handling exploding data volumes. Added to this are regulatory pressures about data retention, and a need to turn raw data into useful information. That is what our on demand strategy is about," Monshaw says.
All of IBM's large storage competitors use the same language. Hewlett-Packard promises to help customers "focus on using the information they have stored, rather than on managing the storage itself," and talks repeatedly about reducing complexity. EMC says that "smart use of information has truly become a differentiator," and promises to "simplify complexity" in order to allow customers to get on with that differentiation.
Software is the broad front on which the industry has already begun attacking the problem. One of the most salient technologies is storage virtualisation, or to be technically accurate, a new breed of storage virtualisation software that was pioneered by start-up software developers such as FalconStor Software and StoreAge, around five years ago.
But the hour is finally approaching for virtualisation, which by definition masks complexity, and will soon be part of almost every large vendor's portfolio. Volume market adoption is still some way off, but when it happens, virtualisation will hugely simplify the day-to-day operation of storage systems. Virtualisation automates what are currently complex but very routine disk configuration tasks, and allows data to be moved around virtual pools of disk capacity at will.
Virtualisation will also have a major effect on the way storage suppliers operate, by allowing customers to couple together disk arrays made by different vendors, so freeing them from a long-standing vendor lock-in. It also threatens to make redundant the expensive array-based software customers currently use when mirroring or snapshotting data for protection purposes. This effectively turns high-end disk arrays into zombies under the control of virtualisation systems, and completes their commoditisation. "When it is taken up, virtualisation is going to open up the market a lot," says Martin Davies, managing consultant at integrator and consultant Morse.
IBM is currently leading the virtualisation field with the SAN Volume Controller system that it launched in 2003, and which has recently chalked up 1000 customer implementations.
HP fumbled an early lead in the technology, and is now the only storage big-hitter not shipping or soon to be shipping a home-grown virtualisation system.
HP is keeping quiet about how it will recover its virtualisation momentum. It does at least offer its customers the high-end virtualisation system developed by Hitachi, rebranded as an HP product. This is Hitachi's Lightning TagmaStore array, which was launched late last year. The TagmaStore hosts virtualisation software that allows it to take control of and pool the capacity of third-party arrays attached to it.
EMC is set to ship its Invista virtualisation product this summer. This will run on the new breed of smart SAN switches from Brocade Communications Systems, McData and Cisco, that can host third-party software. This is the same platform for virtualisation chosen by Veritas, which just over a year ago launched a Cisco smart switch-hosted version of its Volume Manager software. However, as of March this year, there had been no production implementations of the product.
The biggest reason for this less than stunning sales performance is a market blockade that has demonstrated the size of the threat posed by virtualisation. The large storage OEMs that dominate sales of SAN gear - HP, EMC, IBM and Hitachi - all refused last year to offer the Veritas product to their customers, on the grounds of limited demand. In reality, it was because it competed with their conventional array-based software.
Gary Bloom, Veritas chairman, president and CEO
Virtualisation will be an important tool to help customers embrace another universal theme in storage: information lifecycle management. ILM involves the use of multiple tiers of storage, each with different levels of performance and protection against disaster, and each storing data of different value. As data ages or changes in value, or applications change priority, virtualisation tools will move data between storage tiers without disrupting applications.
The acronym first came to prominence in 2003, when EMC nailed its colours to the ILM mast, making its name temporarily almost synonymous with ILM. Within months, tape and disk giant Storage Technology threatened to trademark the term, which it claimed to have invented. Eventually that plan was dropped, but StorageTek still boasts that it is the originator of ILM.
Now ILM is beginning to move from vendors' marketing talk towards real implementation. "All our customers are talking about ILM. Evaluating the data they have got and putting it on the right sort of storage is important, for both SMBs and enterprises," says Davies. According to surveys completed by researcher TheInfoPro, ILM last year leapt from 17th to second place among the priorities of US CIOs.
EMC's giant purchases of back-up software specialist Legato Software and content management maven Documentum were ostensibly inspired by its commitment to ILM. But for customers, ILM has far less to do with software than with strategy. "Of the effort involved in implementing ILM, 90% is about strategy and procedures, and only 10% is about technology," says Sue Clarke, analyst at the Butler Group.
That view is endorsed entirely by EMC's VP of platform marketing, Chuck Hollis. "ILM is a strategy, not an uber-system or a point product," he says.
So why did EMC spend so much money and resources absorbing Legato and Documentum? The software acquisitions certainly bolstered EMC's profit margins against the ravages of hardware commoditisation, but ILM was also a major reason for the purchases. Without Documentum and Legato, EMC could not truly claim to be an ILM supplier. Even their products were not enough to complete the picture. Once famously disdainful of tape storage, EMC last year was forced to engage in a reselling deal with tape vendor ADIC, because tape is a key component of ILM.
Other vendors have not hit the acquisition trail in quite the same way as EMC, but ILM has still forced them to broaden their businesses. Soon after EMC bought Legato and Documentum, HP declared an 'ILM initiative' which saw it partner with a raft of software companies specialising in areas such as e-mail archiving and content management, and including businesses such as CommVault, Ixos, KVS and even EMC-Legato.
HP did adopt EMC's approach, however, when it bought Persist Technologies, whose technology now underpins HP's answer to the Centera CAS device, which EMC has sold so successfully into the compliance and ILM markets. As EMC did with the Centera, HP has opened its box to third-party applications that play a major part in its use.
Veritas subsequently bought e-mail archiving specialist KVS in order to bolster its ILM credentials, and Hitachi signed an OEM deal with Ixos, which was later bought by Open Text. IBM has been spared from this need to buy or partner, by the extensive nature of its software catalogue. But that does not mean that IBM has not been forced to open new fronts in the war on complexity.
Just one example of this was the renewed commitment it made to network attached storage (NAS) earlier this year, when it announced an OEM deal with NAS king, Network Appliance. IBM was not the only company that has very recently had to fully embrace NAS, which for some time has been given only lip service by almost all of the big hitters in storage.
The huge growth of file-based and desktop generated reference data that has helped drive customers towards ILM, has also made the hitherto small market for NAS too important to ignore. In the same month that IBM announced its NetApp deal, Hitachi launched the first of what it says will be a range of in-house-developed NAS devices, and HP unveiled high-end clustered NAS systems based on software OEMed from a start-up.
Elsewhere, data retention regulations are pushing many vendors towards the new but also fast-growing market for compliant storage systems, most notably CAS, or content addressed storage. EMC pioneered the CAS market in 2002, and was followed by HP with its CAS-based RISS e-mail archiving system in 2004. Already this year StorageTek and Sun Microsystems have promised to develop their own CAS systems.
Beyond the need to compete in an ever growing number of arenas, the pure-play storage suppliers are facing a much larger long-term threat, which is that storage is heading towards a convergence with server and network management systems. Veritas and EMC are already attempting to anticipate this development. Late in 2002, Veritas announced its purchase of application monitoring start-up Precise Software Solutions, and server virtualisation specialist Jareva Technologies. Last year it opened its wallet again and bought Ejasent, a start-up whose software migrates applications between servers.
These purchases are part of Veritas' promise to develop 'utility' computing systems, which at their most sophisticated will allow server and storage resources to be automatically allocated to applications on the fly, as if computing power were as simple a resource to consume as utility-supplied electricity or gas. Veritas argues that its utility plans will in no way be distracted by its coming merger with Symantec Software, and that the union will simply add security to the utility software stack.
EMC has been making similar purchases to Veritas, although it has yet to declare any sort of utility plan. It claimed that ILM was the reason for its purchase of server virtualisation specialist VMWare last year, and of network diagnostics developer Smarts this year. But server virtualisation and network diagnosis have a less than tangential connection with tiered storage or ILM, and EMC is obviously heading towards converged systems management.
Veritas and EMC are headed for competition in this field with IBM and HP, which both have utility and grid-based computing on their roadmaps. This is why Veritas and EMC are moving this way. If they do not, in the long term, IBM and HP will steal their market share.
The days are gone when big storage suppliers were specialists, or specialist divisions of larger businesses. Software is now the mainstay of storage technology, and the tool with which vendors are promising to reduce the complexity which businesses face in maintaining storage systems.
Ironically, the competition to eliminate that complexity is making life more complex for the storage vendors themselves, by generating an increasing number of technology arenas. But these arenas are not niches, and their interdependencies are likely to push the storage industry towards consolidation. How much that threatens competition and innovation remains to be seen.