Overview

In this guide, we will walk through deploying a Delphix Engine, starting with configuring  Source, Staging (aka Validated Sync), and Target database environments. We will then create a dSource, and provision a VDB. It assumes that you are working in a Lab/Dev setting and attempting to quickly test Delphix functionality. It assumes you will use the VMware Hypervisor. 

Deploy OVA on VMware

Use the Delphix-supplied OVA file to install the Delphix Engine. The OVA file is configured with many of the minimum system requirements and deploys to one 300GB hard disk with 8 vCPUs and 64GB RAM. The underlying storage for the install is assumed to be redundant SAN storage.

1. Download the OVA file from https://download.delphix.com . You will need a support login from your sales team or a welcome letter.

1. 1. Navigate to "Virtual Appliance" and download the appropriate OVA. If you are unsure, use the HWv8_Standard type.

2. Login using the vSphere client to the vSphere server (or vCenter Server) where you want to install the Delphix Engine.

3. In the vSphere Client, click File .

4. Select Deploy OVA Template .

5. Browse to the OVA file.

6. Click Next .

7. Select a hostname for the Delphix Engine.
   This hostname will also be used in configuring the Delphix Engine network.

8. Select the data center where the Delphix Engine will be located.

9. Select the cluster and the ESX host .

10. Select one (1) datastore for the Delphix OS . This datastore can be thin-provisioned and must have enough free space to accommodate the 300GB comprising the Delphix operating system.

11. Select four (4) or more datastores for Database Storage for the Delphix Engine. The Delphix Engine will stripe all of the Database Storage across these VMDKs, so for optimal I/O performance, each VMDK must be equal in size and be configured Thick Provisioned - Lazy Zeroed . Additionally, these VMDKs should be distributed as evenly as possible across all four SCSI I/O controllers.

12. Select the virtual network you want to use.
    If using multiple physical NICs for link aggregation, you must use vSphere NIC teaming. Do not add multiple virtual NICs to the Delphix Engine itself. The Delphix Engine should use a single virtual network. For more information, see Optimal Network Architecture for the Delphix Engine .

13. Click Finish .

    The installation will begin and the Delphix Engine will be created in the location you specified.

Setup Network Access to the Delphix Engine

1. Power on the Delphix Engine and open the Console.

2. Wait for the Delphix Management Service and Delphix Boot Service to come online.

   This might take up to 10 minutes during the first boot. Wait for the large orange box to turn green.

3. Press any key to access the sysadmin console.

4. Enter sysadmin@SYSTEM for the username and sysadmin for the password.

5. You will be presented with a description of available network settings and instructions for editing.

   Delphix Engine Network Setup
   To access the system setup through the browser, the system must first be configured for networking in your environment. From here, you can configure the primary interface, DNS, hostname, and default route. When DHCP is configured, all other properties are derived from DHCP settings.
   
   
   To see the current settings, run "get." To change a property, run "set <property>=<value>." To commit your changes, run "commit." To exit this setup and return to the standard CLI, run "discard."
          defaultRoute       IP address of the gateway for the default route
       -- for example, "1.2.3.4."
   
   
             dhcp                        Boolean value indicating whether DHCP should be used for the primary interface. Setting this value
   to "true" will cause all other properties (address, hostname, and DNS) to be derived from the DHCP response
            dnsDomain              DNS Domain -- for example, "delphix.com" dnsServers       DNS server(s) as a list of IP addresses -- for example, "1.2.3.4,5.6.7.8."
   
   
            hostname                Canonical system hostname, used in alert and other logs -- for example, "myserver"
   primaryAddress Static address for the primary interface in CIDR notation -- for example, "1.2.3.4/22"
   
   
   Current settings:
   
   
   defaultRoute: 192.168.1.1 dhcp: false
   dnsDomain: example.com
   
   
   

6. Configure the hostname. If you are using DHCP you can skip this step.  

   delphix network setup update *> set hostname=<hostname>

   Use the same hostname you entered during the server installation.

7. Configure DNS. If you are using DHCP, you can skip this step.

   delphix network setup update *> set dnsDomain=<domain>
   
   delphix network setup update *> set dnsServers=<server1-ip>[,<server2-ip>,...]

8. Configure either a static or DHCP address.

   DHCP Configuration

   delphix network setup update *> set dhcp=true

   Static Configuration

   delphix network setup update *> set dhcp=false
   
   delphix network setup update *> set primaryAddress=<address>/<prefix-len>

   The static IP address must be specified in CIDR notation (for example, 192.168.1.2/24)

9. Configure a default gateway. If you are using DHCP, you can skip this step.

   delphix network setup update *> set defaultRoute=<gateway-ip>

10. Commit your changes. Note that you can use the get command prior to committing to verify your desired configuration.

    delphix network setup update *> commit
    
    Successfully committed network settings. Further setup can be done through the browser at:
    
           http://<address>
    
    Type "exit" to disconnect, or any other commands to continue using the CLI.

11. Check that you can now access the Delphix Engine through a Web browser by navigating to the displayed IP address, or hostname if using DNS.

12. Exit setup.

delphix> exit

Setting Up the Delphix Engine

Once you setup the network access for Delphix Engine, navigate to the Delphix Engine URL in your browser for server setup.

The setup procedure uses a wizard process to take you through eight configuration screens:

• Administrators

• System Time

• Network

• Storage

• Serviceability

• Authentication

• Registration

• Summary

1. Connect to the Delphix Engine at HTTP:// <Delphix Engine> /login/ index.html#serverSetup.

   The Delphix Setup application will launch when you connect to the server.

   Enter your sysadmin login credentials, which initially defaults to the username sysadmin , with the initial default password of sysadmin . On the first login, you will be prompted to change the initial default password.

2. Click Next. 

Administrators

The Delphix Engine supports two types of administrators:

• System Administrator ( sysadmin ) - this is the engine system administrator. The sysadmin password is defined here .

• Engine Administrator ( admin) - this is typically a DBA who will administer all the data managed by the engine.

  The default domain user created on Delphix Engines is now admin instead of delphix_admin. When engines created before 5.3.1 are upgraded to 5.3.1 or later they will retain their old username 'delphix_admin'. To avoid complications Delphix recommends creating users with an admin role and then Disabling delphix_admin.

System Time

Choose your option to setup system time in this section.

For a Quick Start, simply set the time and your timezone. You can change this later.

Network

The initial out-of-the-box network configuration in the OVA file is set to use a single VMXNET3 network adapter.

You have already configured this in the initial configuration. Delphix supports more advanced configurations, but you can enable those later.

Storage

You should see the data storage VMDKs or RDMs you created during the OVA installation. Click Next to configure these for data storag e.

Serviceability

Choose your options to configure serviceability settings.

For a Quick Start, accept the defaults. You can change this later.

Registration

If the Delphix Engine has access to the external Internet (either directly or through a web proxy), then you can auto-register the Delphix Engine:

1. Enter your Support Username and Support Password .
2. Click Register. 
   If external connectivity is not immediately available, you must perform manual registration.
3. Copy the Delphix registration code in one of two ways:
   1. Manually highlight the registration code and copy it to the clipboard. Or,
   2. Click Copy Registration Code to Clipboard .
4. Transfer the Delphix Engine's registration code to a workstation with access to the external network Internet. For example, you could e-mail the registration code to an externally accessible email account.
5. On a machine with access to the external Internet, please use your browser to navigate to the Delphix Registration Portal at http://register.delphix.com .
6. Login with your Delphix support credentials (username and password).
7. Paste the Registration Code .
8. Click Register .
   

To regenerate the registration code for a Delphix Engine please refer to,  Regenerating the Delphix Engine Registration Code. Delphix strongly recommends that you regenerate this code and re-register the engine regularly to maximize the Support Security of the Delphix Engine. Delphix recommends doing this every six months.

Summary

The final summary tab will enable you to review your configurations for System Time, Network,  Storage, Serviceability, and Authentication.

1. Click the Back button to go back and to change the configuration for any of these server settings.
2. If you are ready to proceed, then click Submit .
3. Click Yes to confirm that you want to save the configuration.
4. Click Setup to acknowledge the successful configuration.
5. There will be a wait of several minutes as the Delphix Engine completes configuration.

Source and Target  Environment Requirements

Each DB2 Source host (master) must meet these requirements:

• IBM DB2 installed and an instance created on the machine
• HADR settings for each database to be used with the standby server should be preset before the linking process begins

Requirements for DB2 Staging and Target Hosts and Instances

• The staging environment that the Delphix Engine uses must have access to an existing full backup of the source database on disk to create the first full copy. Delphix recommends using compressed backups as that will reduce storage needs and speed up ingest.
• The staging and target DB2 instances that you wish to use must already exist on the host and contain no existing databases.
• The available instances on each host can be verified by going to the databases tab for the environment in question.

Additional Environment Requirements

• There must be an operating system user ( delphix_os ) with these privileges:

• • Ability to login to the target environment via SSH

  • Ability to run mount, umount, mkdir, and rmdir as a super-user. If the target host is an AIX system, permission to run the nfso command as a super-user.  See Sudo Privilege Requirements for DB2 Environments for further explanation of the commands and Sudo File Configuration Examples for DB2 Environments for examples of the /etc/sudoers file on different operating systems.

• There must be a directory on the staging and target environment where you can install the Delphix Engine Toolkit – for example,  /var/opt/delphix/toolkit .

• • The delphix_os user must own the directory.

  • The directory must have permissions -rwxrwx--- (0770), but you can also use more permissive settings.

  • If delphix os user and instance users (responsible for DE operations such as linking and provisioning) are not sharing any common group, then toolkit directory must have -rwxrwxrwx (0777) permissions.

  • The delphix_os user must have read and execute permissions on each directory in the path leading to the toolkit directory. For example, when the toolkit is stored in /var/opt/delphix/toolkit, the permissions on /var, /var/opt, and /var/opt/delphix should allow read and execute for "others," such as -rwxr-xr-x.

  • The directory should have 1.5GB of available storage: 400MB for the toolkit and 400MB for the set of logs generated by each DB2 instance that runs on the host.

  • In DB2 Toolkit: toolkit directory space will be used as the base location for the mount point.

• The Delphix Engine must be able to initiate an SSH connection to the target environment

• NFS client services must be running on the target environment

Instance User Requirements

• The instance owner of each instance you wish to use within a staging or a target host must be added as an environment user within the Delphix engine. See Managing DB2 Users and Instance Owners .
• For HADR synced dSources the staging instance owner must be able to "read" the ingested database contents as Delphix will check the validity of the database by querying tables before each dSource snapshot.

Database Container Requirements

• All DB2 database containers types are fully supported with the exception of DB2 raw containers.  NOTE: If a container is added or deleted, the dSource will have to be resynced.

Instance level configuration values such as the bufferpool value will need to be managed by the customer independent of Delphix. The instances used for staging and target environments must be compatible with the source DB2 instance. The Delphix DB2 DB Level toolkit supports managing dSources with database level granularity.

Sudo Privilege Requirements for DB2 Environments

This topic describes the rationale behind specific sudo privilege requirements for virtualizing the DB2 Databases.

Adding a DB2 Environment

Prerequisites

• Make sure that the staging environment in question meets the requirements described in Requirements for DB2 Hosts and Databases.

Procedure

1. Login to the Delphix Management application.
2. Click Manage .
3. Select Environments .
4. Next, to Environments , click the Actions (...) menu and select Add Environment .
5. In the Add Environment dialog, select Unix/Linux in the menu.
6. Select Standalone Host .
7. Click Next .
8. Enter a Name for the environment.
9. Enter the Host IP address or hostname .
10. Enter the SSH port .
11. The default value is 22.
12. Enter an OS Username for the environment. For more information about the environment user requirements, see Requirements for DB2 Hosts and Databases .
13. Select Login Type.
14. For Password, enter the password associated with the user in step 9.
15. Using Public Key Authentication:
16. If you want to use public key encryption for logging into your environment:
    1. Select Public Key for the Login Type.
    2. Click View Public Key.
    3. Copy the public key that is displayed, and append it to the end of your ~/.ssh/authorized_keys file. If this file does not exist, you will need to create it.
       1. Run chmod 600 authorized_keys to enable read and write privileges for your user.
       2. Run chmod 755 ~ to make your home directory writable only by your user.
       3. The public key needs to be added only once per user and per environment.
       4. You can also add public key authentication to an environment user's profile by using the command line interface, as explained in the topic CLI Cookbook: Setting Up SSH Key Authentication for UNIX Environment Users .
17. For Password Login, click Verify Credentials to test the username and password.
18. Enter a Toolkit Path (make sure toolkit path does not have spaces).
19. For more information about the toolkit directory requirements, see Requirements for DB2 Hosts and Databases .
20. Click Submit .

As the new environment is added, you will see two jobs running in the Delphix platform Job History, one to Create and Discover an environment, and another to Create an environment. When the jobs are complete, you will see the new environment added to the list in the Environments tab. If you do not see it, click the Refresh icon in your browser.

Managing DB Instances

When you add an environment with the Delphix Management Application, all existing DB2 instances on the host are automatically discovered by Delphix. A list of all instances and databases are available to Delphix based on the environment discovery process.

View Instances

1. Login to the Delphix Management application.
2. Click Manage .
3. Select Environments .
4. In the Environments panel, click on the name of the environment to you want to refresh.
5. Select the Databases tab to see a list of all DB2 instances found in the environment.

Linking a Data Source (dSource)

Prerequisites

Be sure that the source and staging instances meet the host requirements and the databases meet the container requirements described in Requirements for DB2 Hosts and Databases .

Source Database Preparation

Instance Owner Permissions

Delphix uses the DB2 instance owner account on the dSource for many things, including verifying the data inside the databases. For ingesting database on the staging server with different instance we need permissions on the source database to do restore on the staging server. For example in the source if we have an instance named auto1051 and database name delphix and  if we want to create a dSource on the auto1052 instance on staging server then you must explicitly grant DBADM and SECADM to the dSource instance auto1052 on the source instance using the following steps :

1. Connect to the source database as the source instance owner.
   1. connect to <DB_NAME> user <INSTANCE_OWNER>
2. Issue database grant command.
   1. grant DBADM, SECADM on database to user <DSOURCE_INSTANCE_OWNER>
3. Repeat step 2 for every database to be included in the dSource, on the corresponding source database.

Determine if your dSource will be a non-HADR instance, an HADR single standby instance, or an HADR multiple standby instance. Non-HADR dSources can only be updated via a full dSource resync from a newer backup file

Non-HADR Database

HADR Single Standby Database

1. All items in Non-HADR Database section above.
2. The following database configuration settings must be set:
1. update db cfg for <DB_NAME> using HADR_LOCAL_HOST <PRIMARY_IP> HADR_LOCAL_SVC <PRIMARY_PORT > immediate
2. update db cfg for <DB_NAME> using HADR_REMOTE_HOST <STANDBY_IP> HADR_REMOTE_SVC <STANDBY_PORT> immediate
3. update db cfg for <DB_NAME> using HADR_REMOTE_INST <STANDBY_INSTANCE_NAME> immediate
4. update db cfg for <DB_NAME> using HADR_SYNCMODE SUPERASYNC immediate
3. If database configuration parameter LOGINDEXBUILD is set to OFF, do the following:
1. update db cfg for <DB_NAME> using LOGINDEXBUILD ON
2. Force off all connections to the database and reactivate the database
4. If database configuration parameter LOGARCHMETH1 is set to OFF, do the following:
1. update db cfg for <DB_NAME> using LOGARCHMETH1 XXXX (must be a valid log archiving method)
2. Take an offline backup
5. If LOGARCHMETH1 points to a third-party backup server (i.e. TSM or Netbackup) define LOGARCHMETH2 to disk
   1. Log files in the directory must be available from the time of the backup until the restore has successfully completed on the dSource.
1. update db cfg for <DB_NAME> using LOGARCHMETH2 DISK:<full path to archive log directory>
6. db2 start hadr on db <DB_NAME> as primary by force
7. Take a full online backup as defined in the "Backup Source Database" section below.
8. Record the following information, as it must be entered on the Delphix Engine while creating the dSource.
1. HADR Primary hostname
2. HADR Primary SVC
3. HADR Standby SVC (auxiliary standby port)

HADR Multiple Standby Databases

This assumes a single standby database HADR setup already exists. The existing standby will be referred to as the main standby. The new delphix standby will be referred to as the auxiliary standby.

1. The following database configuration settings must be set on the primary database:
   1. You may have up to two auxiliary standbys defined separated by a '|'; one of which must be the delphix dSource.
1. update db cfg for <DB_NAME> using HADR_SYNCMODE <SYNC MODE> immediate – set whichever sync mode you wish to use on your main standby.
2. update db cfg for <DB_NAME> using HADR_TARGET_LIST "<MAIN_STANDBY_IP:MAIN_STANDBY_PORT|AUXILIARY_STANDBY_IP:AUXILIARY_STANDBY_PORT>" immediate
2. stop hadr on db <DB_NAME>
3. start hadr on db <DB_NAME> as primary by force
4. Take a full online backup as defined in the "Backup Source Database" section below. While this backup is running, you may continue with step 5.
5. The following database configuration settings must be set on the existing main standby database:
1. update db cfg for <DB_NAME> using HADR_SYNCMODE <same mode as defined in 1.a above.> – It must be the same value used for primary database.
2. update db cfg for <DB_NAME> using HADR_TARGET_LIST "<PRIMARY_IP:PRIMARY_PORT|MAIN_STANDBY_IP:MAIN_STANDBY_PORT>"
6. stop hadr on db <DB_NAME>
7. start hadr on db <DB_NAME> as standby
8. Record the following information, as it must be entered on the Delphix Engine while creating the dSource (the auxiliary standby database):
1. HADR Primary hostname
2. HADR Primary SVC
3. HADR Standby SVC (auxiliary standby port)
4. HADR_TARGET_LIST <PRIMARY_IP:PRIMARY_PORT|MAIN_STANDBY_IP:MAIN_STANDBY_PORT>

Backup Source Database

In order to complete the linking process, the Standby dSource must have access to a full backup of the source DB2 databases on disk. This should be a compressed online DB2 backup and must be accessible to the dSource instance owner on disk. Delphix is currently not setup to accept DB2 backups taken using third-party sources such as Netbackup or TSM. Both HADR and Non-HADR backups must also include logs .

Example backup command: db2 backup database <DB_NAME> online compress include logs.

Procedure

1. Login to the Delphix Management Application using Delphix Engine credentials or as the owner of the database from which you want to provision the dSource.

2. On the  Databases  tab of Environment Management screen, add a source config against discovered staging instance.

3. Then, click Manage.

4. Select Datasets.

5. Click the Plus (+) icon and select Add dSource, you’ll get a list of available source configs using which you can go for dsource creation.

6. In the Add dSource wizard, select the required source configuration.  

7. If you are working with an HADR setup, please leave the HADR checkbox checked.

8. The database name is mandatory and must be unique for a given instance. This is the name that the database was on the instance it was restored from. 

9. Enter the complete Backup Path where the database backup file resides. If no value is entered, the default value used is the instance home directory. If there are multiple backup files for a database on the backup path, the most current one will be used.

10. Enter the Log Archive Method1 you wish to use for the database. If no value is entered, the default value used is DISK:/mountpoint/dbname/arch.

11. Optionally, users can set the database configuration parameters during the linking operation in the Config Settings section.

12. If the dSource is to use HADR please enter the following fields. If it will not use HADR skip ahead to step 13. For more information about HADR please view  Linking a dSource from a DB2 Database: An Overview .

    a. Enter a fully qualified HADR Primary Hostname. This is a required field for HADR and must match the value set for  HADR_LOCAL_HOST  on the master.

    b. Enter the port or /etc/services name for the HADR Primary SVC. This is a required field for HADR and uses the value set for  HADR_LOCAL_SVC  on the master.

    c. Enter the port or /etc/services name for the HADR Standby SVC. This is a required field for HADR and uses the value set for  HADR_REMOTE_SVC  on the master.

13. Click  Next .

14. Select a  dSource Name  and  Database Group  for the dSource.

15. Click Next.

    You will get Data Management section where you need to specify the staging environment and user which will be used for dsource creation.

16. Set the Staging Environment to be the same as the dSource host.

17. Select the  Staging Environment User  to be the same as the instance owner of the dSource instance.

    Changing the Environment User

    If you need to change or add an environment user for the dSource instance, see Managing DB2 Users and Instance Owners .
18. Then, click Next and you’ll get Policies section. Set the desired Snapsync Policy for the dSource. For more information on policies see  Advanced Data Management Settings for DB2 dSources .
    
19. Click Next.
20. Specify any desired pre- and post-scripts. For details on pre- and post-scripts, refer to  Customizing DB2 Management with Hook Operations .
21. Click Next.
22. Review the dSource Configuration and Data Management information in the summary section.
23. Click Submit.

The Delphix Engine will initiate two jobs to create the dSource: DB_Link and DB_Sync. You can monitor these jobs by clicking Active Jobs in the top menu bar, or by selecting System > Event Viewer. When the jobs have completed successfully, the database icon will change to a dSource icon on the Environments > Host > Databases screen, and the dSource will also appear in the list of Datasets under its assigned group.

Provisioning a Virtual Database (VDB)

Prerequisites

• You will need to have linked a dSource from a staging instance, as described in Linking a DB2 dSource or have created a VDB from which you want to provision another VDB

• You should have set up the DB2 target environment with necessary requirements as described in Requirements for DB2 Hosts and Databases

• Make sure you have the required Instance Owner permissions on the target instance and environment as described in Managing DB2 Users and Instance Owners

• The method for Database Permissions for Provisioned DB2 VDBs is decided before the provisioning

Procedure

1. Login to the Delphix Admin application.

2. Click Manage.

3. Select Datasets.

4. Select a dSource.

5. Select a snapshot from which you want to provision.

6. Click Provision VDB icon to open Provision VDB wizard.

7. Select a target environment from the left pane.

8. Select an Installation to use from the dropdown list of available DB2 instances on that environment.

9. Set the Environment User to be the Instance Owner. Note: The picking of instance owner is only possible if you have multiple environment users set on that host.

10. Provide VDB Name as database name as parameter.

11. Optionally, set the database configuration parameters for the VDB.

12. Click  Next .

13. Select a Target Group for the VDB.

    Click the green Plus icon to add a new group, if necessary.

14. Select a Snapshot Policy for the VDB.

15. Click Next.

16. Specify any desired hook operations. For details, on-hook operations, refer to Customizing DB2 Management with Hook Operations.

17. Click  Next .

18. Review the Provisioning Configuration and Data Management information.

19. Click Submit.

    When provisioning starts, you can review the progress of the job in the Databases panel, or in the Job History panel of the Dashboard. When provisioning is complete, the VDB will be included in the group you designated and listed in the Databases panel. If you select the VDB in the Databases panel and click the Open icon, you can view its card, which contains information about the database and its Data Management settings.

    Once the VDB provisioning has successfully completed, if the source and target instance ids are not the same, you may want to grant secadm and dbadm on the database to the target instance id. Refer to Database Permissions for Provisioned DB2 VDBs for more information.

Next Steps

Congratulations! You have provisioned your first virtual database!

Now, perform some simple functional tests with your application. You can connect your app to the VDB using standard TNS/JDBC techniques. Delphix has already registered the VDB for you on the target listener.

We suggest the following next steps:

1. Drop a table and use the VDB Rewind feature to test the recovery of your VDB.
2. Take a snapshot of your dSource and refresh your VDB to quickly get fresh production data.
3. Provision a new VDB from your VDB to test sharing data quickly with other teams.
4. Mask your new VDB to protect sensitive data. Provision new VDBs from that masked VDB to quickly provide safe data to development and QA teams.