What happened to ILM?

By Steve Tongish.

Frequently featuring as a theme at storage conferences around the world, information lifecycle management (ILM) has been one of the hottest topics in the storage industry over the last few years. Riding the surge of interest, storage technology vendors of all sizes scrambled to present their products as part of an ILM strategy or, in some cases, as a complete ILM solution. However, in recent months we have heard very much less about ILM. What explains ILM’s apparent disappearance from the storage industry’s radar?

Those who have spent some time observing the industry suspect that ILM has been yet another in a long line of utopian visions, over-hyped by both the storage industry and the press. Some go so far as to suggest that the sudden popularity of ILM amongst the vendor community was the ability it gave them to put a fresh coat of paint on old marketing materials and sell more of their existing products. However, as with other trends whose substance didn’t quite match expectations, customers realised that ‘the emperor has no clothes', and ILM retreated into the shadows.

Not surprisingly, these assertions are vigorously contested within sections of the storage industry. The initial goal was to raise market awareness of the benefits of ILM strategies so that the concept could be fully assimilated into the IT consciousness. Once that had been achieved, solutions able to support the ILM theory would follow and practical solutions would be delivered to the market. In that context, they say, everything has gone exactly to plan.

As with many issues that polarise opinion, the truth probably lies somewhere in the middle. ILM was clearly over promoted, especially when you consider that, even today, few storage vendors can deliver on ILM’s vision of full data control and resource optimisation. The inability to meet customer expectations that the industry itself set, has almost certainly tarnished its own reputation. On the other hand, the process of hype and lagging fulfilment is a very familiar cycle. Because of the technical complexities that lie behind a heterogeneous ILM architecture, it stands to reason that an operational and cost-effective solution would follow the formulation of the ILM proposition by months or even years.

In the context of this middle-way, ILM is no different to many other technology strategies that have come before and which we have eventually accepted as part of our daily business, even if the final incarnation deviates significantly from the original concept. ILM has not disappeared. In fact, there is plenty of evidence to support the position that ILM has simply moved onto the next stage of its evolution.

ILM has been championed by large and small vendors as a way of solving the problem of storing unstructured data. Effectively managing data sets that are very difficult to classify can have huge cost and operational implications if you cannot optimise the distribution of the files across a range of storage hardware over the life of the data. The Holy Grail is to understand the data well enough to place it on the correct storage technology at the right point in time. Magnetic disk, tape and optical storage each have their own strengths and weaknesses and being able to capitalise on the strengths can provide tremendous business benefits. All of the following concepts have practical, effective and available solutions that go some way to achieve the lofty goals of ILM:

Information classification and management (ICM)
The first step in the ILM process is the effective classification of unstructured data, known as information classification and management (ICM). As is so often the case, smaller storage companies have developed some of the most interesting technology. A handful of start-ups have emerged as leaders in ICM. These companies employ software and/or hardware to identify and apply classification to previously unstructured data. There are two main architectural approaches, in-band that applies data classification dynamically as it is created, or out-of-band that classifies existing data sets using batch processes. Some vendors use a combination of the two, but no matter the technique, the objective is the same; to apply a degree of intelligence to unstructured data so it can be managed more effectively.

Electronic content management (ECM) and digital asset management (DAM)
So what is so new about data classification? Isn’t this something that has been going on for years in the electronic content management (ECM) and digital asset management (DAM) markets? ECM and DAM solutions are excellent examples of how data classification can dramatically improve business operations. These solutions typically address closed-loop applications such as invoice processing and content creation. They can also include vertical industry applications like PACS (picture archive and communication systems) and HIS (hospital information systems) in the medical industry.

In all these cases, the value and workflow of the content is much better understood. ECM and DAM applications manage the information through its workflow and over its entire life. In this sense data classification is not a new concept. However, even for those companies that have deployed ECM and DAM systems, they often have a very large amount of data that doesn’t fall neatly into a specific application area. It is this unstructured data that ICM is designed to address within the context of ILM.

It is also interesting to note that very few ECM or DAM solutions are storage device aware. This means that while they understand the data, they do not use their data classification intelligence to ensure the data is staged to the most appropriate storage device. In many cases, ECM and DAM vendors go out of their way to remain device agnostic. While this may make business sense, it creates a potential disconnect between logical classification and physical storage which is key to creating a truly optimized system.

Data lifecycle management (DLM)
One successful subset of ILM is data lifecycle management (DLM). This technology approach can go a long way in bridging the gap between classification and physical storage. So what is the difference between DLM and ILM? The vision of ILM is based on the ability to drill into unstructured data and identify attributes that are vital to business operations. As noted earlier, this can be pretty tricky. DLM provides a slightly less ambitious approach to data classification, focusing on the already known attributes of unstructured data such as file type, size, owner, last accessed time, etc. Using these attributes, the unstructured data can be categorized sufficiently to optimize physical storage over the life of the data.

This may sound suspiciously like the old hierarchical storage management (HSM) model and in some ways it is. However, today’s DLM solutions have expanded on HSM and incorporated analysis and reporting and tools from storage resource management (SRM) solutions. DLM capabilities include functionality to virtualize the storage devices, replicate and move data as required, de-duplicate (single instance) redundant data, encrypt files for protection and have enabled advanced search tools to more easily locate and leverage the value of unstructured data.

Summary
In practice, it can be difficult to draw clear lines between ILM, ICM, ECM, DAM, DLM and other technology categories as they often overlap. However, one thing is clear; ILM has not disappeared. There is a huge amount of technology and financial resource being applied to the principles of ILM. Dozens of small storage companies are focusing on different pieces of the ILM puzzle and some are being acquired by larger vendors to fill out their own strategies. Industry organizations like SNIA are also working behind the scene to put standards in place that enable technology interoperability, furthering the cause of the ILM vision.

The potential benefit to customers and vendors is very compelling. In a perfect world, ILM will allow companies to intelligently manage their unstructured data to meet business objectives while optimizing storage resources. This vision promises major business efficiencies and bottom-line financial gains.

The press and the storage industry did themselves no favours in over-promoting ILM as a solution that could be delivered immediately. It is a complex, multi-faceted model that incorporates a wide range of disciplines. ILM has moved from hype into a more exciting and realistic phase in its evolution. Key pieces of technology are now being delivered to the market, and customers are responding to this by taking a more pragmatic approach to ILM. Few companies will claim to have an ILM strategy, but they are deploying carefully selected technology that allows them to begin to tackle the challenges of their unstructured data. The ILM vision is alive and well.

Steve Tongish is director of marketing (EMEA) for Plasmon Data.

•Date: 5th October 2007• Region: World •Type: Article •Topic: IT continuity
Rate this article or make a comment - click here