Best Practices

Here you will find our technical and procedural recommendations for a successful migration, based on our collective experiences.

General

1. Over-provision the destination for the duration of the migration process

2. Build indexes later in the CDC phase

3. Use UAT and DEV post-migration average object size to get accurate estimates for PROD data size and disk sizes

4. Monitor the process

   1. CPU, Memory, Disk IO on the source and destination

   2. CPU and Memory for Dsync process

   3. Dsync throughput and replication lag

5. Check all the components before the cutover time, especially when the source and the destination been in the CDC mode for a prolonged period of time

   1. Source database and services health

   2. Destination database and services health

   3. Dsync process(es) health and replication lag

      1. When running Dsync in a single-binary mode, any errors and replication lag are reflected in the CLI progress view and the dsync log file

      2. When running the Enterprise version, in addition to the web progress dashboard, it's a good idea to check the status of workflows and tasks in Temporal, as well as check for any reported errors in SigNoz log collector

Database-specific recommendations

Cosmos DB RU destinations

1. Enable server-side retries in Cosmos DB configuration

2. Cosmos DB RU consumes a lot of resources for batch writes (~3-5 RU per write). When using as a destination for a large migration, make sure to provision at least 8K RUs for default settings

MongoDB destinations:

• For large (multi-TB) migrations use w:majority write concern (default on Atlas) and increase replWriterThreadCount if there's CPU and disk headroom (e.g. try 64 for M200)

• Disable backups

• Set oplog to a fixed size in Atlas

• Use network compression in the connection string when Atlas is in a different region or availability zone - this will help save network costs

Dsynct Enterprise:

• Use a dedicated disk for Temporal database (recommended 3000 IOPS, 125 MBps)