Do’s and Don’ts in Document Migrations

By: Brian J. Stewart

A successful document migration safeguards the integrity of documents and metadata, ensures compliance with regulations, and assures the return on investment (ROI) is realized. This article provides several recommendations on what to do and not do with document migrations.

Do

Consider all systems that integrate with the legacy and target systems – It is important to consider how systems that integrate with the legacy system will be impacted by the data migration. In addition, to improve data accuracy and integrity, as well as streamline business processes, it is critical to view data beyond application boundaries. Migrating to a new system enables organizations to evaluate how to integrate in new ways with additional external systems to reduce data redundancy.
Map data to harmonize data dictionary and Master Data Management (MDM) – Migrating data to a new system provides an opportunity to harmonize data dictionaries across systems and align with an organization’s Master Data Management strategy. Harmonizing data enables new integration opportunities, improves data reporting, and increases data accuracy.
Generate a post-migration report – A report should be generated post-migration that includes the source and mapped metadata, as well as the migration status for each document. Ideally this document should be in a familiar format, such as a spreadsheet to provide a convenient mechanism to facilitate the manual migration verification activity.
Ensure the migrated document in the target system contains legacy source name and legacy object identifier – Storing the legacy identifier information as part of metadata in the target system enables system administrators and users to clearly identify the source of each document. This is useful in post-migration verification and reporting as well as correcting data post-migration should an issue surface.
Create post-migration tool to automatically verify content integrity – Create a checksum tool to verify integrity of the migrated content. A checksum provides the ability to compare content at the byte/block level to ensure no errors occurred during the transfer of content from source to target.
Involve business in verification of migration – Verifying document migrations isn’t as glamorous as testing a new application. Data verification is a time-consuming and tedious activity, but not one that should be left to only technology professionals. Certainly technical verification is needed, but too often the activity is viewed as a technical activity. Businesses users who know the data are more likely to discover data discrepancies and issues.
Use a staging area to review content data and metadata when appropriate – In some migrations, content or metadata is changed significantly from the source system to the target system. This may be a result of complex data mapping rules, data harmonization and standardization, or reformatting of content. In these scenarios, it is best to implement a staging repository and system where documents can be reviewed and approved for data migration. The content can be automatically migrated to the target system upon business verification and approval.

Don’t

Don’t start the data migration activities too late in a project – Too often the data migration activities aren’t started until after development of the target system is complete or nearly complete. This is too late in the project for two main reasons. First, the review of source data may uncover flaws or oversights in the target system design. The identification of these flaws too late causes expensive and time-consuming development rework which could affect the project schedule and budget. The second reason to start early is the fact that data migration activities are often underestimated. A good time to start the migration planning is after initial design or prototype phase of the target system.
Don’t just migrate all documents to the target system – The default approach for many document migrations is to migrate all documents to the new target system. It is important to recognize the costs associated with migrating documents, such as the time for data mapping, verification, actual migration, as well as continued storage and management costs.
Don’t underestimate the document mapping activity – Data mapping is a complex, tedious, and time-consuming activity which is typically iterative in nature. Often additional rules are identified the further data is analyzed. It is not uncommon to discover missing rules after migrating documents to the target system. To save time and money, it is better to build a pre-migration tool that generates a report containing the source and target (after applying data mapping rules) metadata as it will exist post-migration. This enables the business to correct data in multiple iterations without the expensive and time-consuming effort re-migrating documents to a test system.
Don’t underestimate the document verification activity – Document migrations, especially large document migrations, are prone to issues with mapping and other complexities. Robust verification methods should be employed to verify data, including report generation and analysis, automated and programmatic verifications, as well as manual inspection in the target system.
Don’t only use a subset of documents to test migration tools and processes – It is important to run a full production document migration in the test environment prior to the actual production migration. Too often only a subset of documents are migrated to the test environment, this limited approach prevents the identification of issues until after the production migration.
Don’t leave new attributes blank for migrated documents – It is advantageous to implement a process and mechanism to capture metadata values for documents that will be migrated from a source system which does not contain specific fields or attributes. Blank attributes decreases the effectiveness of locating documents, as well document reporting. Blank attribute values may also impact functionality where the attributes are expected to be non-blank.
Don’t perform data cleanup post-migration – In an attempt to reduce project timelines, too often a decision is made to defer data cleanup to post-migration. This is frequently shortsighted as data is seldom “corrected” post-migration. It should be done as part of the document migration process and should be clearly defined in the Document Migration Plan.

Summary

A successful document migration is dependent on implementing specific processes, activities, and design decisions to ensure data integrity and compliance, as well as avoiding common mistakes made with document migrations. Follow these do’s and don’ts to guarantee a successful document migration.

Related Article(s)

  1. Recommendations for Successful Document Migrations