Data validation and reconciliation is a continual process that goes on within almost all organizations. These processes for the most part are manually and redundant. Countless departments reconcile data back to the General Ledger to confirm that they are working with a complete set of transactions. This problem is not isolated to any industry or content, although highly regulated industries such as finance and healthcare must have a more robust set of procedures in place. For these verticals, documentation of the control and evidence that the control is being verified add to the complexity and overhead. MetaGovernance estimates that this effort consumes an average 20% of time for a typical data-intensive role. The time spent insuring accuracy of data and information is, unfortunately, time not spent on analysis to generate business revenue or activities more in line with the value stream of the company.
Tackling the inefficiencies of continual validation and reconciliation are an integral component of an effective Information Governance program. What is needed is an automated means for reconciling data between systems. But not just reconciling data in databases from system 1 to system 2. Reconciliations need to extend to derived information such as Net Income or Total Sales as found in data warehouse and countless reports. Reconciliations must also extend to spreadsheets, which are so often the system of record for financial data.
Data inconsistency issues are not just across systems. Often discrepancies occur within the same database. We once had a client that had 26 different fields representing accrued interest in one database. These fields were added over the years, one column at a time, due to a lack of awareness and control. Often the data in these fields did not agree. This issue was quickly found through an Information Governance initiative that focused on sources and uses of data to track down known copies of the business attribute accrued interest.
MetaGovernance utilizes a Reconciliation Control Framework® for the purpose of validating the consistency of data and information. This technology-enabled framework crosses system databases and spreadsheets. Companies want reconciliations across transactional system or data warehouses. As the diagram shows, there is a strong need to reconcile between these systems and the General Ledger to prove that the sub-ledger and ledgers are in agreement. These systems are often at different levels of detail, increasing the need for technology-enabled solutions. A Reconciliation Control Framework can be used for ongoing business operations, fraud detection, during data conversions, as part of system parallel or QA testing, or during times of mergers and acquisitions.
While there are many automated reconciliation tools on the marketplace, it is rare to find an approach that is tightly integrated into an Information Governance framework. MetaGovernance uses the concept of registered governance stakeholder. These stakeholders are known and documented departments that have one of the key Information Governance or Data Governance roles of owner, steward, consumers, or custodians over a domain of data. Previous MetaGovernance blogs have talked about the Information Governance Architecture and awareness of the source and use of data and information assets. Organizational knowledge of registered governance stakeholder is part of the key set of information to effectively manage an organization’s information assets.
To be effective, any reconciliation engine needs to be tightly linked into the business rules (metadata) that provide clear visibility into which departments and people need to be aware of an issue found during ongoing data reconciliations. Of what value is knowing there are data issues, if they cannot be dynamically communicated with the stakeholders? Of what value is a listing of stakeholders that is outdated due to reorganizations? Automated data reconciliation tools are one very powerful component in the Information Governance arsenal. However, to serve the business needs, they must be inclusive of the broad set of business rules that drive Information Governance as captured in the overall governance architecture.