Replication Overview
Data Backup and Recovery Solutions / Replication / Replication Overview

Replication Overview

Delphix allows you to replicate data objects between Delphix Engines. These engines must be running identical Delphix versions, but otherwise they can be asymmetric in terms of engine configuration. In the event of a failure that destroys the source engine, you can bring up the target engine in a state matching that of the source. In addition, you can provision VDBs from replicated objects, allowing for geographical distribution of data and remote provisioning.

You can run replication ad hoc, but it is typically run according to a predefined schedule. After the initial update, each subsequent update sends only the changes incurred since the previous update. Replication does not provide synchronous semantics, which would otherwise guarantee that all data is preserved on the target engine. When there is a failover to a replication target, some data is lost, equivalent to the last time a replication update was sent.

Replication is generally not suited for high-availability configurations where rapid failover (and failback) is a requirement. Failing over a replication target requires a non-trivial amount of time and is a one-way operation; to fail back requires replicating all data back to the original source. For cases where high availability is necessary, it is best to leverage features of the underlying hypervisor or storage platform. For more information on how to evaluate the use of Delphix Engine replication for your data recovery requirements, see the topics under Backup and Recovery Strategies for the Delphix Engine.

Replication Features

As virtual appliances, it is possible to backup, restore, replicate, and migrate data objects between Delphix platforms using features of VMWare and the underlying storage infrastructure. Data objects include groups, dSources, VDBs, Self-service (Jet Stream) data templates and data containers, and associated dependencies. In addition to the replication capabilities provided by this infrastructure, native Delphix platform replication provides further capabilities, such as the ability to replicate a subset of objects, replicate multiple sources to a single target, and provision VDBs from replicated dSources and VDBs without affecting ongoing updates. The topics under Backup and Recovery Strategies for the Delphix Engine provide more information on how to evaluate features of the Delphix platform in relation to your backup and recovery requirements. 

Replication is configured on the source Delphix platform and copies a subset of dSources and VDBs to a target Delphix platform. It then sends incremental updates manually or according to a schedule. For more information on configuring replication, see Configuring Replication.

You can use replicated dSources and VDBs to provision new VDBs on the target side. You can refresh these VDBs to data sent as part of an incremental replication update, as long as you do not destroy the parent object on the replication source. For more information, see Provisioning from Replicated Data Sources or VDBs.

During replication, replicated dSources and VDBs are maintained in an alternate replica and are not active on the target side. In the event of a disaster, a failover operation can break the replication relationship. For more information on how to activate replicated objects, see Replicas and Failover.

Replication Details

When you select objects for replication, the engine will automatically include any dependencies, including parent objects, such as groups, and data dependencies such as VDB sources. This means that replicating a VDB will automatically include its group, the parent dSource, and the group of the dSource, as well as any environments associated with those databases. When replicating an entire engine, all environments will be included. When replicating a database or group, only those environments with the replicated databases are included. 

During replication, the Delphix Engine will negotiate an SSL connection with its server peer to use SSL_RSA_WITH_RC4_128_MD5 as the cipher suite, and TLSv1 as the protocol.

Only database objects and their dependencies are copied as part of a backup or replication operation, including:

  • dSources
  • VDBs
  • Groups
  • JetStream Data Templates and Data Containers
  • Environments
  • Environment configuration (users, database instances, and installations)

The following objects are NOT copied as part of a backup or replication operation:

  • Users and roles
  • Policies
  • VDB (init.ora) configuration templates
  • Events and faults
  • Job history
  • System services settings, such as SMTP

After failover, you must recreate these settings on the target.

Selective Data Distribution

Selective data distribution replication enhances the current replication feature by allowing you to restart large, time-consuming initial replication or incremental updates from an intermediate point. A single replication instance can fail for a number of environmental and internal reasons. Previously, when you restarted a failed replication instance, replication required a full resend of all data transmitted prior to the failure. With resumable replication, no data is retransmitted. Replication is resumable across machine reboot, stack restart, and network partitions. The resumable replication feature is fully automated and does not require or allow any user intervention.

For example, suppose a replication profile has already been configured from a source to a target. A large, full send begins between the two that is expected to take weeks to complete. Halfway through, a power outage at the datacenter that houses the source causes the source machine to go down and only come back up after a few hours. On startup, the source will detect a replication was ongoing, automatically re-contact the target, and resume the replication where it left off. In the user interface (UI) on the source, the same replication send job will appear as active and continue to update its progress. However, in the UI of the target, a new replication receive job will appear but will track its progress as a percentage of the entire replication.

In 4.1 and earlier releases, the replication component would always clean up after failed jobs to ensure that the Delphix platform was kept in a consistent state and that no storage was wasted on unused data. With the addition of resumability, the target and source can choose to retain partial replication state following a failure to allow future replications to complete from that intermediate point. In the current release, the target and source will only choose to retain partial replication state following failures that leave them out of network contact with each other – for example, source restart, target restart, or network partition. Once network contact is re-established, the ongoing replication will be automatically detected and resumed. 

Replication will not resume after failures that leave the source and target connected. For example, if a storage failure on the target, such as out-of-space errors, causes a replication to fail, the source and target remain connected. As a result, the Engine will discard state data associated with the failed Replication operation. Nonetheless, resumable replication would begin during a source reboot, a target reboot and a network partition.

Replicating Jet Stream Templates

Templates can now be replicated and accessed on the target engine via Jet Stream. Replicated templates can be replicated into the target space with or without their containers. On the new target engine, the newly created replicated template can be used to create new containers that are assigned to users. You cannot change the replicated template’s name or the names of the containers from which it was replicated.

Any containers that were  replicated over with the template can not be used to start, stop etc until they are disconnected to their parent containers in the source engine during the failover operation.

Related Links