Case Study: Migrate

The Problem

The customer, a large multi-national, had regular and ongoing mergers and acquisitions activity. This created an ongoing requirement to migrate data from each company’s systems into their own applications. Their inventory management system was implemented in Oracle on a UNIX platform, and is the subject of this case study. Most of the source systems were in different platforms, often IBM mainframes, so the migration typically involved a transfer between environments, in addition to the typical data quality and data transformation tasks.

Due to a lack of tools, it had been their practice to require the IT department of the other company to be responsible for data conversion to match their system. Inevitably, they had to get involved in the process, resulting in significant project over-runs and duplication of effort. In many cases the other company would contract with third parties to handle the data conversion, at significant additional cost to the organization.

As well, since the other company typically took many months to prepare the data conversion, many data quality issues were usually not found until late in the process, which inevitably resulted in further migration delays.

The customer wanted to be able to handle the data conversion directly themselves, in order to:

• improve the quality of the task
• finish projects on schedule, and
• reduce the migration costs

The Solution

Using the Arbutus technology they were able to take control of the data migration process. Arbutus was chosen after demonstrating the ability not only to access mainframe data directly, but also to provide comprehensive data quality assessments on data from most environments. This was achieved by staging the data onto a machine that had the Arbutus Windows Engine installed for analysis and processing.

In addition, they were able to contract with Arbutus to supply specialized data conversion expertise as required, where the demands of the task were beyond their capabilities.

The Process

Each merger or acquisition represents a totally separate data migration project. The target is always the same, but each source system is unique. As there was a backlog in data migration, there were a number of candidates, at various stages of completion, to choose from.

First, they identified an existing project nearing completion, and used Arbutus Migrate to verify the results of the work they had already completed. This was done as a part of their evaluation process for Arbutus, to verify the data quality capabilities. They read the finalized files that had been prepared by the other company and that were ready for loading into Oracle. The process of creating the metadata took very little time, based on the support provided by Arbutus’s Data Definition Wizard expert system.

Within days they were able to confirm the quality of the converted data, and were actually able to identify a significant number of non-critical data quality issues that had been missed during their previous process.

To test a complete migration process, they repeated the process with a company at an early stage of the conversion. In this step they actually read the customer’s source data, and created files to load Oracle, but otherwise performed the same data quality tasks as above.

Again, the metadata creation progressed quickly, due to the Data Definition Wizard’s ability to automatically convert the source system’s COBOL copybooks. The metadata for the twenty five relevant source files for this system were defined and reviewed within the first day, once the other IT department had provided the location of the copybooks. They utilized on-site consultants from Arbutus to ensure this process ran smoothly.

On the second day, the initial reports on critical data quality issues were completed. This information was forwarded to the company’s IT department to address. They also noticed a significant number of non-critical data quality issues that were also forwarded to the IT department. As they reviewed the source data they realized that the documentation they had been provided was significantly out of date, and were able to adjust their initial conversion mapping based on these results.

The last step in this process was to create the files to load Oracle. This involved the creation of a number of procedures, to automate the conversion process. The transformations were by and large very straightforward, and presented no particular problems.

The customer only had one opportunity to address data quality with the other IT department, so only a portion of the non-critical data quality issues were addressed before the conversion. Even so, they were able to set up a separate project to correct the remaining data quality issues in their system, based on the results of the pre-load data.

Results

With the Arbutus technology they were able to achieve their initial goals. As they had hoped:

• Using the Arbutus tools they were able to migrate the data far more easily than they had previously, and at a fraction of the cost
• They were able to finish their data conversion project significantly ahead of their original schedule. It would have been even quicker, but most of the delays were beyond their control in the other IT department.
• They were able to provide much more comprehensive data quality testing, and at a much earlier stage, and were much more aware of the remaining issues

They were also able to realize unexpected benefits. These included:

• They were able to do a much better job of mapping the source data to their system because they had an opportunity to work with the data at a much earlier stage.
• They discovered that the inevitable late stage project changes had a minimal impact on their conversion process due to the flexibility of the Arbutus technology.