From the March 2010 issue of Treasury & Risk magazine

Catching Data Degradation

While accuracy in their financial data has always been a top concern, more companies are now fretting about the quality of all the information that flows through their systems. According to a recent survey by Aberdeen Group, a Boston consultancy, "85% of mid- to large-size organizations indicated that they struggle with information quality," says Russ Klein, group director of technology research at Aberdeen.

Input errors are the most common source of problems, followed by "the challenge of data integration, especially across enterprises, with trading partners and other external stakeholders," Klein says. "You have situations where it's difficult to normalize data because the structure of the data is different."

"Most organizations recognize they do have a data quality problem, but they've pursued it in very tactical ways," says Merv Adrian, a principal at consultancy IT Market Strategy, adding that many solutions attack the symptoms rather than the cause. Companies that take that approach can spend considerable sums resolving a problem, only to see it reoccur at a later date, he says.

One of the new products IBM rolled out last month helps companies keep a watchful eye on that data. InfoSphere Business Information Monitor "provides early alerts about a degradation in quality," says Steven Adler, IBM's program director for information governance.

Adler says the quality of data can be compromised as it moves through a company's systems. The channels through which data flows--what Adler calls the information supply chain--are in poor shape, he says. "They're often creaky, old and badly monitored."

He gives the example of a company that combines two different databases that contain addresses and Social Security numbers, one of which has zeros for the Social Security numbers. The company could regard the integration as successful in the absence of a tool that checks the new records, he says.

InfoSphere can sample data to detect problems, Adler says. "We set up sensors, we set up agents that monitor transactions and we set up certain rules that check the quality." InfoSphere combines technology IBM acquired with its 2009 purchase of Guardium with proprietary research. InfoSphere has connections to a number of databases and ERP and business intelligence systems.

While companies don't have to use technology to make their data more trustworthy, it can help, especially when there's a lot of information or time is a factor, says Aberdeen's Klein. "If you can get your decision-maker the information that is needed in hours rather than days or minutes rather than hours, that translates to real savings."