Metadata Management: Helping Tame IT's Chaos
By Don Tirsell
India, Dec 10, 2007

Corporations, like cities, grow and contract organically. Infrastructure is added in response to demand, and in general, until a certain critical mass is reached, the need for infrastructure management is unrealized. Thus, organizations that have gathered data over the years are now, in fact, overflowing with data.

This data is stored in various silos, and is spread across the enterprise. However, the data is redundant, inconsistent, fragmented, in different locations, in different contexts, and hard to leverage outside of its single context. Companies have not so much managed data, as to gather and store it.

A major contributor to breaking down barriers and helping IT "tame chaos" is metadata management. Metadata management is usually found in two forms, either as a discrete shared Business/IT function or as a function within an IT domain like Data Integration or Business Intelligence. But what is metadata and how can managing it help?

Metadata defined

Metadata is descriptive information about data. For example, the metadata for a book includes the author, publisher, copyright date, edition, and other such information. In fact, the card catalog in a library can be considered one of the first metadata management systems invented.

Metadata can be broadly divided into 3 different types: technical, operational, and business. Technical metadata is information about the structure of data, such as physical definitions, e.g. format, data type, field names, and table names. This technical metadata provides visibility into data stores, identifies and locates data, and helps to improve the management of data systems.

Operational metadata is in the form of transformation rules, statistics on process execution and error/change information.

A specific example is when a job was run, load times for data processed and data rejected in the process.

This higher order metadata can be sourced from Data Integration and Business Intelligence repositories, and applications, such as CRM or ERP. This metadata can provide insight into the business processes within a corporation.

Business metadata provides context. For example, what's the definition of "profit" or "northern region"? The semantics of data is of particular interest to business users. Strictly speaking, since metadata in its "raw" form is technical metadata, business metadata must be abstracted from or associated from technical metadata.

Business metadata is required to reconcile data across the enterprise, simplify audits, help business analysts find the data they need, and help drive correct decision making.

To conclude, metadata management is the practice of cataloging, associating, analyzing and managing Technical, Operational and Business metadata. Its value is clear for data integration and business intelligence; they cannot function efficiently without metadata.

But organizations that tackle metadata management at a higher level and connect Business and IT together through a metadata management initiative are taking active steps towards unraveling the "data chaos" within their boundaries and increasing the value of data as a productive asset.

- The writer is the senior director (product marketing) of Informatica Corporation