Managing Archival Data

It is becoming more difficult to manage an enterprise’s archived data
because it involves structured and unstructured components. Solution providers
should offer an archival platform that is an ideal combination of intelligent
storage hardware and software. This combination can be effectively used with ISO’s
Reference Model for Open Archival Information Systems (OAIS).

In today’s business environment, information and data have transformed to
become the most important corporate assets. This has given a huge impetus to the
storage solutions market. Vendors in this space have come up with many
innovative products and solutions that help organizations store and manage
corporate data.

However, with an exponential growth of data, it is becoming increasingly
difficult to store and manage archival data. It has become all the more
problematic because both structured and unstructured data have become an
integral part of today’s business.

STRUCTURED AND UNSTRUCTURED

The traditional definition of structured data is that which is organized by
the well-defined structure provided by databases. These database sizes are
growing so fast that it is impeding application performance and artificially
inflating the total cost of operations.

However, if we look at unstructured data, the growth of unstructured data has
far surpassed the growth of structured data. This is virtually due to the
inherent nature of unstructured data.

Unstructured data typically comprises of documents, spreadsheets, graphics,
still and motion images and various other formats. Going further, messages and
e-mail can be classified as semi-structured data as they form a framework for
further classifying unstructured data. According to industry estimates over 50%
of data residing in data centers falls into these categories.

STUMPED BY EMAILS

Emails are now a well-established form of correspondence and its usage is
growing phenomenally. Users normally get messages such as "Mailbox size
limit exceeded". Adding to the problem are new regulations which are
forcing corporations to retain their e-mails for a specified period of time and
to be able to produce them on demand.

In such a scenario, solution providers should get their corporate customers
to adopt techniques such as data life cycle management (DLCM). This is done by
effectively by managing all the data that is considered to be a corporate asset,
by matching availability and retrieval time with the data’s value which varies
throughout the data lifecycle. In adopting DLCM techniques, organizations can
elevate the efficiency and responsiveness of the total storage environment and
utilize available capacity optimally.

While it is fundamental that partners continue to ensure capacity
requirements are met for critical enterprise applications, there is a further
demand for more effectively managing digital assets by moving them to a
different class of media based on their current value.

The idea is to take advantage of waning requirements for retrieval time and
availability by moving less valuable, less-likely to be accessed data to less
expensive storage. Doing so necessitates greater intelligence for managing
storage devices and automatically moving data within the overall storage
environment from the time it is created until its expiry.

WHY IT MAKES SENSE

Since more and more information is generated out of business activities, it
is outside the periphery of structured bounds and retrieval mechanisms. This
gives rise to the need to quickly catalogue, search and retrieve this
unstructured information into the storage environment itself.

At the same time, solutions must encompass varying classes of storage devices
and media arranged in tiers in order to balance the cost of storing any
particular data asset with its current value from the time of creation to
end-of-life.

Therefore, the archival platform solution should be an ideal combination of
intelligent storage and an open and collaborative approach to storage software.
This combination can be most effectively used with ISO’s Reference Model for
Open Archival Information Systems (OAIS). OAIS is a proven foundation for
archive systems, having served as the underpinnings of some of the largest data
archives in existence.

Conclusively, the archival storage architecture should be based on an open,
ISO-compliant architecture that implements DLCM as a complement to mainstream
storage and business continuity practices. This openness allows enterprises to
participate in an inter-operable environment where the right data is always
available at the right time, and there is no need for the special-purpose
storage management software and devices used by other solutions.

Sudhakar Rao is Director-Technical
Consultant at Hitachi Data Systems