What Is Data Migration

What is data migration?

Data migration is the planned movement of data from one location, platform, schema, or storage format to another. It may happen during a cloud migration, a database upgrade, an application rewrite, a merger, a backup restoration, or a move from one vendor to another. A simple example is moving customer records from an old ecommerce database into a new commerce platform. A more complex example is moving logs, user profiles, payment references, and content objects across several cloud services while the public site continues to run.

Good data migration is not just copying files. The goal is to preserve meaning, ownership, integrity, security, and operational continuity. A record that arrives in the new system with the wrong encoding, missing timestamps, broken permissions, or a changed customer identifier may be technically present but operationally wrong. For that reason, migration work needs both engineering and governance: what data exists, who owns it, how it is protected, how it is validated, and when the old copy can be retired.

Where migration shows up

Data migration often appears as part of a larger technology change. An organisation may rehost applications, consolidate databases, adopt a managed storage service, or split a monolith into smaller services. Each change can create a data movement problem even when the project is described as infrastructure work.

Migration can also happen continuously. Analytics pipelines, customer data platforms, object storage replication, disaster recovery systems, and search indexes move data on schedules or in near real time. These migrations need the same controls as one-off projects, because repeated silent errors can be harder to detect than a single failed cutover.

Common migration types include storage migration, database migration, application data migration, cloud migration, and data centre migration. Some projects combine several types. For example, a public website may move media assets to object storage, user accounts to a new identity system, and logs to a cloud analytics platform at the same time.

Planning the migration

Start with an inventory. Identify datasets, schemas, file stores, indexes, credentials, retention rules, owners, and systems that read or write each data source. Include hidden dependencies such as scheduled exports, reporting jobs, API consumers, cache warmers, and support tools. Migration plans often fail because the main database is understood but surrounding data flows are not.

Next, define the migration method. A big-bang migration moves everything in a short window and then switches users to the new environment. It is simple to reason about but risky for large or sensitive systems. A phased migration moves one dataset, region, tenant, or workflow at a time. It reduces blast radius but requires more compatibility work. A parallel-run migration keeps old and new systems active while teams compare outputs before cutover.

Define rollback before migration starts. Rollback is not always just switching traffic back. If users have changed data in the new system, teams need a plan for replaying, reconciling, or discarding those writes. Without that plan, the first serious defect can turn into a data loss decision made under incident pressure.

Validation and data quality

Validation should check more than row counts. Useful checks include record counts by category, checksums, referential integrity, field-level sampling, permission comparisons, timestamp accuracy, encoding, null handling, duplicate detection, and application-level test journeys. For public web applications, teams should also confirm that caches, search results, customer sessions, and API responses behave correctly after the move.

Business validation matters because technically valid data can still be wrong. A billing system may accept a migrated invoice that uses the wrong tax code. A support tool may show a customer record but lose its case history. A content system may migrate files but break access controls. The people who understand the business process should help define what "correct" means.

Migration evidence should be retained. Keep logs, comparison reports, exception lists, approval records, and cutover timelines. These artifacts are useful for audits, incident response, and future migration planning. They also help teams distinguish a migration defect from an unrelated application bug after the new system is live.

Security and compliance considerations

Data migration can temporarily increase exposure. Data may be staged in intermediate systems, exported to files, transferred across networks, or accessed by contractors and migration tools. Each step should have controls for encryption, authentication, least privilege, logging, and retention. Temporary storage should not become a forgotten permanent copy.

Sensitive data needs special handling. Personal information, payment data, health information, credentials, security logs, and proprietary business records may be subject to regulatory or contractual obligations. Teams should confirm where the data is stored, which jurisdictions apply, who can access it, and whether masking or tokenisation is required during testing.

Network and application security controls still matter during migration. If a public application points to a new backend, request filtering, API validation, bot controls, rate limits, and monitoring should follow the data path. A migration that bypasses normal edge, access, or logging controls can create a short but serious window of risk.

Common failure modes

One common mistake is treating migration as a storage task rather than a service change. Users experience data through applications, APIs, reports, and support workflows. If those paths are not tested, the project can pass technical checks while failing in production.

Another failure mode is underestimating data shape. Old systems often contain inconsistent values, deprecated fields, duplicate records, or undocumented relationships. New systems may enforce rules that the old system did not. These differences should be discovered in trial migrations, not during the final cutover.

Performance can also surprise teams. A migrated database may be correct but slower because indexes, connection pools, network distance, cache behaviour, or query plans changed. Data movement can also create high egress costs or saturate links if throughput is not modelled.

Finally, teams sometimes keep the old environment alive indefinitely "just in case." That may feel safe, but it can leave stale data, duplicate permissions, and unclear ownership. A complete migration includes a retirement plan for old copies, old credentials, and old integrations.

How to know the migration is complete

A migration is complete when the new system serves the required workflows, validation defects are resolved or accepted, monitoring is active, rollback risk is closed, and old data paths are retired according to policy. Completion should be based on evidence, not on the fact that data was copied.

For security and operations teams, the strongest signal is normal service behaviour under real traffic: expected errors, stable latency, correct access control, understandable logs, and no unexplained divergence between old and new records. The final result should leave the organisation with a clearer data estate than it had before the move.