Delphix Storage Migration
System Administration / Capacity and Resource Management / Delphix Storage Migration

Getting Started

Delphix storage migration allows you to remove storage devices from your Delphix Engine, provided there is sufficient unused space, thus allowing you to repurpose the removed storage device to a different server. You can also use this feature to migrate your Delphix Engine to different storage by adding the new storage devices and then removing the old storage devices.

Note: This feature is only compatible for Delphix Engine Releases 5.0.4 and later.  

Please be aware of the following technical bulletins:

TB034 Delphix Engine May Fail To Restart Following Device Removal

TB033 Device Removal May Lead To Delphix Engine Crash

Understanding Delphix Storage Migration

Delphix storage migration is a multi-step process:

  1. Identify a storage device for removal. The device you choose will depend on your use case.  

    1. To remove extra storage that is unused, you can select any device for removal. For best performance, select the device with the least allocated space; typically, this is the device that you added most recently. The allocated space is defined by the usedSize property of the storage device:

       areece-test1.dcenter 'Disk10:2'> ls 
 Properties 
     type: ConfiguredStorageDevice 
     name: Disk10:2 
     bootDevice: false 
     configured: true 
     expandableSize: 0B 
     model: Virtual disk 
     reference:  STORAGE_DEVICE-6000c293733774b7bb0e4aea83513b36 
     serial: 6000c293733774b7bb0e4aea83513b36 
       size: 8GB 
       usedSize: 7.56MB   
     vendor: VMware

           

    2. To migrate the Delphix Engine to new storage, add storage devices backed by the new storage to the Delphix Engine. Then remove all the devices on the old storage.
  2. Use the Delphix command-line interface (CLI) to initiate the removal of your selected device. 
  3. Data will be migrated from the selected storage device to the other configured storage devices. This process will take longer the more data there is to move; for very large disks, it could potentially take hours. You can cancel this step if necessary. 
  4. The status of the device changes from configured to unconfigured and an alert is generated to inform you that you can safely detach the storage device from the Delphix Engine. After this point, it is not possible to undo the removal, although it is possible to add the storage device back to the Delphix Engine.
  5. Use the hypervisor to detach the storage device from the Delphix Engine. After this point, the Delphix Engine is no longer using the storage device, and you can safely re-use or destroy it.

Limitations of Delphix Storage Migration

After removal, the Delphix Engine uses memory to track the removed data. In the worst-case scenario, this could be as much as 1GB of memory per 1TB of used storage. Note that this is used storage; the overhead of removing a 1TB device with only 500MB of data on it will be much lower than the overhead of removing a 10GB device with 5GB of data on it.

User Interface

Delphix storage migration is currently available exclusively via the CLI. There are 3 entry points:

  • storage/remove  –  The status of the current or most recent removal, including the total memory used by all removals up to this point
  • storage/device “$device”/removeVerify –  Returns the predicted effect of removing the selected device, or an error if the device cannot be removed
  • storage/device “$device”/remove –  Begins the evacuation and removal of the selected device

Device Removal for Storage Migration

  1. Identify which device you want to remove.

    1. If you are using a VMware RDM disk, note the UUID of the device by looking at its name in the vSphere GUI. For more information, see this Getting the UUID of an RDM Disk from VMware, via the vSphere GUI.  

    2. If you are using a VMware virtual disk, note the UUID of the device via the vSphere API. See the section of this VMware KB article on how to get the UUID of your virtual disk. 

    3. In EC2, note the attachment point – for example,  /dev/sdf.

    4. In KVM, note the UUID.

  2. Login to the Delphix CLI as a sysadmin user and navigate to storage/device.

  3. Type cd storage/device.

  4. Select your device: 

    areece-test1.dcenter storage device> ls
Objects
NAME      CONFIGURED  SIZE  EXPANDABLESIZE
Disk10:2  true        8GB   0B
Disk10:0  true        24GB  0B
Disk10:1  true        8GB   0B 
Disk10:3  true        8GB   0B
areece-test1.dcenter.dcenter storage device> select Disk10:2

  5. (VMware only) Confirm that your disk selection is correct by validating that the serial matches your UUID:

    areece-test1.dcenter storage device 'Disk10:2'> ls
Properties
   type: ConfiguredStorageDevice
   name: Disk10:2
   bootDevice: false
   configured: true
   expandableSize: 0B
   model: Virtual disk
   reference: STORAGE_DEVICE-6000c2909ccd9d3e4b5d62d733c5112f
   serial: 6000c2909ccd9d3e4b5d62d733c5112f
   size: 8GB
   usedSize: 8.02MB
   vendor: VMware

  6. Execute removeVerify to confirm that removal will succeed. Validate the amount of memory/storage used by the removal:

    areece-test1 storage device 'Disk10:2'> removeVerify
areece-test1 storage device 'Disk10:2' removeVerify *> commit
   type: StorageDeviceRemovalVerifyResult
   newFreeBytes: 15.85GB
   newMappingMemory: 3.14KB
   oldFreeBytes: 23.79GB
   oldMappingMemory: 0B

  7. Execute remove to start the device evacuation:

    areece-test1 storage device 'Disk10:2'> remove
areece-test1 storage device 'Disk10:2' remove *> commit
   Dispatched job JOB-1
   STORAGE_DEVICE_REMOVAL job started for "Disk10:2".
   0.63GB out of 0.68GB remaining.
   0.59GB out of 0.68GB remaining.
   ...

  8. Wait for device evacuation to complete. Alternatively, you can cancel the evacuation. 

    Do NOT detach the device from the Delphix Engine in your hypervisor until after the data evacuation has completed.


     The screenshot below illustrates a progress bar for completion. 


  9. Once the device evacuation has completed, the job will finish and a fault will be generated. Detach the disk from your hypervisor and the fault will clear on its own. An example of the fault created as seen below.


When using VMDKs, deleting the wrong VMDK could cause data loss. Therefore, it is highly advisabletodetachthedevice, then verify that the Delphix Engine continues to operate correctly, andlastlydeletetheVMDK.

Getting the UUID of a RDM Disk from VMware, via the vSphere GUI

In the ESX graphical user interface (GUI), select your VM. 

  1. Click Edit settings.
  2. If not already displayed, select the Hardware tab.
  3. Select the device you want to remove.

  4. Click Manage Paths. 
     


The UUID of the device appears in the title bar, as seen below.

Getting the UUID of a VMDK from VMware, via ssh to the ESX server

  1. ssh onto the ESX server as the root user
  2. Navigate to the directory containing the .vmdk files for the Delphix VM
  3. Use the 'vmkfstools -J getuuid <.vmdk filename>' command to obtain the UUID, for example
    /vmfs/volumes/25894daa-f7b2b044/delphix01-2356 # vmkfstools -J getuuid delphix01-2356_1.vmdk
UUID is 60 00 C2 91 01 bc 8e 72-31 a4 cd b0 b3 f6 e5 74

Related Links