Database Migration From AWS (mysql RDS) to GCP Cloud Sql

[ShivendraPatel]

I am migrating the database from AWS to GCP Cloud SQL using the database migrations service of
GCP.
Source Database info: 2vCpu,4GB RAM, and 150GB Data Size. MySQL Version: 5.7.42
Destination Database Info: 2vCpu, 3.75GB RAM and 200GB Storage. MySQL Version: 5.7.44  

We have two options for migration,  * one-time migration and * Continuous migration.
I am using the Continuous migration option for migration and it was working fine. but when it started the migration. The migration speed was good. in the initial one hour,48GB of data was migrated but after that, it was migrating data with a very low speed of 1GB/ Hour.  this is my development database and it does not have any traffic. I am using the public IP network option for it. 

I am unable to debug it why does the speed go slow after 48GB of data transfer?
Attaching a screenshot of the overall database migration status.

[ms4446 · Accepted solution]

The speed of a continuous database migration can be influenced by several factors, including:

1. Network Bandwidth: Limited network bandwidth between AWS and GCP, especially when using public IP networks, can cause significant slowdowns in data transfer. Consider using a dedicated network connection to enhance speed.

2. Database Resources: Insufficient resources (CPU, memory, disk I/O) in either the source or destination database can impede migration. Monitor resource utilization on both databases and increase resources if necessary.

3. Database Contention and Activity: Concurrent activities or processes on the source or destination databases might be affecting the migration speed. Review activity logs to identify and mitigate any unrelated database activities during migration.

4. Database Locks: Locks on the source database during migration can lead to delays. Ensure that the database is appropriately configured to handle migration processes without significant locking.

5. Incremental Migration: Utilize a migration tool that supports incremental migration to ensure that ongoing changes in the source database are continuously captured and migrated.

6. Data Type and Structure: The nature of the data being migrated, such as large BLOBs or complex table structures, can affect migration speed. Assess the data structure for elements that might require more time to migrate.

7. Service Limits and Quotas: Check for any AWS or GCP service-specific limits or quotas that might be impacting the migration process.

To improve the migration speed, consider the following tips:

• Use a dedicated network connection for more consistent and faster data transfer.
• Ensure both databases are adequately resourced in terms of CPU, memory, and disk I/O.
• Schedule the migration during off-peak hours to minimize contention and locks.
• Opt for a migration tool that supports incremental data transfer.
• Regularly monitor the migration process for any errors or warnings in the logs.