This quick start guide, which is excerpted from the larger User Guide, is intended to provide you with a focused overview of working with SAP ASE database objects in the Delphix Dynamic Data Platform. It does not cover advanced configuration options. It assumes you have already setup the Delphix Engine.


Overview

In this guide, we will walk through configuring SAP ASE Source and Target environments, creating a dSource, and provisioning a VDB.

The following diagram describes the platform topology for SAP ASE environments. It illustrates the required and recommended ports to open from Delphix to remote services, to the Delphix Engine, and to the Target Environments. A detailed list can be referred to in the Network and Connectivity Requirements for SAP ASE Environments section of the manual.

Delphix stays in sync with source databases by monitoring the ASE backup server log. When it sees a new database dump or transaction log has been created, it attempts to load it into the ASE staging instance.

Validated sync ModeSnapShot for FULL BACKUPSnapShot for TLOGSPoint-in-Time Restore
truncate log on chkpt = trueYesNoNo
truncate log on chkpt = falseNoYesNo
truncate log on chkpt = true + Log SyncYesNoNo
truncate log on chkpt = false + LogSyncNoYesYes

Before proceeding with setting up environments, decide where the databases will be located relative to the ASE staging instance. In the following diagram, we have two ASE hosts (production and staging).


If the database dumps (or transaction logs) are available locally to the staging Backup Server, Delphix considers them “local”. Whereas if they are available to a Backup Server on production, they are “remote”. If the files will be local to the staging server, there are many options to get them from the production host to the staging host including but not limited to:

  • Sharing the directory on the production host over NFS to the staging host.

  • Using Network Attached Storage (NAS) to replicate the files.

  • Using scp to copy the files from one host to the other.

If the files are only available on the production host, Delphix will login to the ASE staging instance and issue the “LOAD DATABASE” command using the remote server syntax (for example: LOAD DATABASE pubs2 FROM "/dumps/pubs2.full.9_5_16" AT Prod_BS).

Source Host Requirements

There must be an operating system user, such as delphix_os, that meets the following requirements:

  • The $SYBASE environment variable is defined for non-interactive shells (such as via the .bashrc configuration file). 

    • Set the PermitUserEnvironment configuration parameter to "yes" in the sshd_config file

    • Add the variable to the user's .ssh/environment file

    • Restart the SSH daemon

  • To test this requirement:

ssh delphix_user@ase_hostname env | grep SYBASE


  • Can login to the source host via SSH (TCP port 22)

  • Delphix requires superuser permission to run pargs in order to discover Solaris ASE instances. For more information, see Sudo Privilege Requirements for SAP ASE Environments.

  • Has read access for the Sybase ASE Backup Server log files.

  • There must be a directory on the source host where you can install the Delphix platform toolkit, for example: /var/opt/delphix/Toolkit

    • The delphix_os user must own the directory

    • The directory must have permissions 0770, for example, -rwxrwx---. However, you can also use more permissive settings.

    • The directory should have 256MB of available storage.

Source Database Requirements


When adding a source ASE environment to Delphix, you may use a single login to discover the ASE instances and link the source databases OR you may use a single login to discover all of the ASE instances and separate logins to link each dSource.

  • Delphix uses a single database user for discovery of all ASE instances and their databases for each environment added to Delphix.

    • The discovery database user (delphix_disc for example) must have SELECT privileges on the following tables for each ASE instance on the source host:

      • sysdatabases

      • sysservers

      • syslisteners

  • Another user must be specified when linking each dSource (delphix_link for example) that has SELECT privileges on the above tables.

    • If you will select New Full Backup when linking, this user must also have privileges to take a new full database dump of the source database. For more information about linking options, see Linking an SAP ASE Data Source.

    • The link database user can be different for each instance and database on the source host.

If the source database is resized and trunc log on chkpt is disabled, take a transaction log dump immediately after the resize operation completes. If trunc log on chkpt is enabled, take a full database dump immediately after the resize operation completes. If multiple resizing operations are performed without taking transaction log dumps between each operation it may be necessary to manually sync the dSource with a new full database dump for Delphix to be able to continue ingesting source database dumps.

Target Host Requirements

  • The operating system on the target environment must be the same as, or binary compatible with, the operating system on the source environment.

  • The SAP ASE major version on the target environment must be the same as the version on the source environment. However, EBF/SP version on target environment can be higher. If the target is used as a staging server, the ASE version must match at the patch/sp level also.

  • There must be an operating system user, such as delphix_os, that meets the following requirements:

    • The $SYBASE environment variable is set for non-interactive shells (such as via the .bashrc configuration file).  Set the variable as follows:

      1. Set the PermitUserEnvironment configuration parameter to "yes" in the sshd_config file

      2. Add the variable to the user's .ssh/environment file

      3. Restart the SSH daemon

    • To test this requirement:

ssh delphix_user@ase_hostname env | grep SYBASE


  • Can login to the target host via Secure Shell (SSH)

  • Can login to ASE instances using isql with LANG=C set

  • Has write permission for the mount-point directory

  • The following permissions are usually granted via sudo authorization of the commands. See Sudo Privilege Requirements for SAP ASE Environments for further explanation of this requirement, and Sudo File Configuration Examples for SAP ASE Environments for example file configurations.

    • Permission to run mount and umount as super-user.

    • On Solaris, permission to run pargs on Solaris

    • On AIX, permission to run the nfso command as super-user.

    • (Optional) On AIX and Linux, permission to run ps as super-user.

    • Disable tty for the delphix_os user for mount and umount.

  • There must be a directory on the source host where you can install the Delphix platform toolkit, for example: /var/opt/delphix/Toolkit

    • The delphix_os user must own the directory

    • The directory must have permissions 0770, for example, -rwxrwx--. However, you can also use more permissive settings.

    • The directory should have 1GB of available storage

    • Avoid using the home directory of the delphix_os user

    • If you intend to use the LogSync feature, it is recommended to make the toolkit directory as short as possible to keep the full path to the transaction log file names under ASE's 127 character limit. For example, create the toolkit directory as /tk.

Target Database Requirements


    • There must be a database user, such as delphix_db, with the sa_role on each instance on the target environment

    • The database user such as delphix_db for any staging instances must also have the sybase_ts_role

    • If the target host will be used as a staging target environment (see Managing SAP ASE Environments Overview), at least one of the following two options must be configured:

      • You must use sp_addserver to add the staging ASE instance's Backup Server to sysservers on the source ASE instance (so that remote database dump/load works)

        OR

      • Full and transaction dump files from the source database must be available locally to the staging database (over NFS, replication, scp, etc.)

    • Staging and target ASE instances should have disk mirroring disabled.


sp_configure "disable disk mirroring" – run value should be 1, which is the default.  If it is 0, change it using



sp_configure "disable disk mirroring", 1  – this parameter is static so the ASE instance must be restarted for this change to take effect.


    • Delphix will mirror the number of devices used on the source database for the staging database (dSource) and each VDB created from that source database. The number of devices parameter should be scaled appropriately based on the max number of virtual databases that will be provisioned to the ASE instance. This parameter can be changed using: sp_configure "number of devices", <max devices>.


ASE 15.7.0 SP100 and later releases support the shrink command. In some cases Delphix must increase the number of devices used for databases if this command is used. Delphix creates a minimum of the same number of devices as the source database for the staging database (dSource) and each VDB and will add more devices for every 4TB of fragment holes. See SAP ASE issue CR#799273 for additional details.


To support multiple VDBs and the staging databases, you may need to increase the parameter number of alarms.


Delphix uses ASE operations which use alarm structures such as MOUNT and UNMOUNT. The number of alarms limit the number of these operations which can be run concurrently. Various ASE instance failures can occur if the available alarm structures are exhausted. The amount of memory consumed by increasing the number of alarm structures is small. Delphix recommends that the number of alarms value is increased to at least 4096.

Adding SAP ASE Source and Target Environments


Follow the steps below to add both source and target environments for SAP ASE.


  1. Login to the Delphix Management application using Delphix Admin credentials.
  2. Click Manage.
  3. Select Environments.
  4. Click the Plus icon next to Environments.
  5. In the Add Environment dialog, select Unix/Linux.
  6. Select Standalone Host.
  7. Enter the Host IP address.
  8. Enter a Name for the environment.
  9. Enter the SSH port.
  10. The default value is 22.
  11. Enter a Username for the environment.
  12. Select a Login Type.
  13. For Password, enter the password associated with the user in Step 10.
  14. For Password Login, click Verify Credentials to test the username and password.
  15. Enter a Toolkit Path.
  16. The toolkit directory stores scripts used for Delphix Engine operations. It must have a persistent working directory rather than a temporary one. The toolkit directory will have a separate sub-directory for each database instance. The toolkit path must have 0770 permissions.
  17. Click the Discover SAP ASE checkbox.
  18. Enter the database Username for an ASE instance on the environment.
  19. Enter the database Password associated with the database user in Step 15.
  20. Click Submit.

ASE Manual Discovery

When an environment is added, Delphix discovers your ASE instances. Manual discovery allows users to add instances that were not automatically discovered. This feature is currently only supported via the CLI.

This topic describes how to use CLI commands to manually add ASE repositories to a SAP ASE environment.  Discovery is the process by which the Delphix platform identifies data sources and data dependencies on a remote environment. ASE repository discovery is done automatically when an environment is added to the Delphix platform or when an already added environment is refreshed. In some cases, automatic discovery does not discover all of the repositories in a SAP ASE environment. These repositories may be added using manual discovery.

Unlike automatically discovered instances, manually discovered instances are not automatically deleted if the environment is refreshed when the instance isn't running. Manually discovered instances are not updated during an environment refresh either. So for example, if you upgrade ASE to a new version or change the listener port, you must manually update the repository.

To manually discover an ASE repository you will need to:

    • Add an the environment to Delphix

    • Use CLI to manually discover a repository

Creating an ASE Environment

Please refer to Adding an SAP ASE Environment for detailed steps.


Manually discovering a repository

In the following example we are using sc-dev3.dc2 as our example environment.


    1. Log into CLI and cd to repository menu:  

$ ssh delphix_admin@sc-dev3.dc2
Password:
sc-dev3.dc2> cd repository
sc-dev3.dc2 repository>


   2. Add (manually discover) an ASE repository instance:

The values used in the following code block are specific to the example instance we are adding.

sc-dev3.dc2 repository> create
sc-dev3.dc2 repository create *> ls
Properties
  type: ASEInstance
  credentials: (unset)
  dbUser: (unset)
  environment: (required)
  installationPath: (required)
  instanceName: (required)
  instanceOwner: (required)
  ports: (required)
  version: (unset)
sc-dev3.dc2 repository create *> set credentials.password=sybase
sc-dev3.dc2 repository create *> set dbUser=sa
sc-dev3.dc2 repository create *> set environment=sc-rhel64-sybase-ase-0
sc-dev3.dc2 repository create *> set installationPath=/opt/sybase/15-7
sc-dev3.dc2 repository create *> set instanceName=ASE1570_S1
sc-dev3.dc2 repository create *> set instanceOwner=sybase
sc-dev3.dc2 repository create *> set ports=5100
sc-dev3.dc2 repository create *> ls
Properties
  type: ASEInstance
  credentials:
      type: PasswordCredential (*)
      password: ******** (*)
  dbUser: sa (*)
  environment: sc-rhel64-sybase-ase-0 (*)
  installationPath: /opt/sybase/15-7 (*)
  instanceName: ASE1570_S1 (*)
  instanceOwner: sybase (*)
  ports: 5100 (*)
  version: (unset)
sc-dev3.dc2 repository create *> commit
  `ASE_INSTANCE-22
sc-dev3.dc2 repository>


Updating a repository

Adding onto the above, the following example illustrates updating an ASE instance's version after upgrading ASE:

sc-dev3.dc2> repository
sc-dev3.dc2 repository> select ASE1570_S1
sc-dev3.dc2 repository 'ASE1570_S1'> update
sc-dev3.dc2 repository 'ASE1570_S1' update *> set version="15.7 SP138"
sc-dev3.dc2 repository 'ASE1570_S1' update *> ls
Properties
   type: ASEInstance
   credentials:
       type: PasswordCredential
       password: ********
   dbUser: sa
   installationPath: /opt/sybase/15-7
   instanceOwner: sybase
   linkingEnabled: true
   ports: 5100
   provisioningEnabled: true
   servicePrincipalName: (unset)
   staging: false
   version: 15.7 SP138 (*)
sc-dev3.dc2 repository 'ASE1570_S1' update *> commit

Be careful when setting the version string. Make sure that it matches output as displayed by the "select @@version" query all the way out to the patch level (PL). For example "15.7 SP138" or "16.0 SP02 PL01".



Linking a SAP ASE Data Source

Prerequisites

Ensure that you have correctly set up the source and target environments, as described in Managing SAP ASE Environments.

Dump file requirements

    • Database and transaction log dumps that the Delphix platform will use must be taken using native ASE format.

    • Dump devices are not supported. Database and transaction dumps the Delphix platform will use must be taken to filesystem files.

    • If ASE dump compression is being used, the dumps must be generated using the compression = compress_level syntax. The older compress::compress_level syntax is supported in Delphix version 4.3.4 or higher.


Procedure

    1. Login to the Delphix Management application using Delphix Admin credentials.

    2. Click Manage.

    3. Select Datasets.

    4. Click the Plus icon.

    5. Select a Add dSource.

    6. Alternatively, on the Environment Management screen, you can click the Add dSource link next to a database name to start the dSource creation process.

    7. In the Add dSource wizard, from the Requirements tab select Next.


      Changing the Environment User

      If you need to change or add an environment user for the source database, see Managing SAP ASE Environment Users.
    8. Select a data source.

    9. Enter your login credentials for the source database.

    10. Click Verify Credentials.


    11. Click Next.

    12. In the dSource configuration tab select:

      dSource Name

      Target Group

      Adding a dSource to a database group lets you set Delphix Domain user permissions for that database and its objects, such as snapshots. For more information, see the topics under Users, Permissions, and Policies.


    13. Click Next.

    14. From the Data management tab select Initial Load option and enter any additional settings needed. There are three different options for the initial load of the dSource:

      1. New Full Backup – Lets Delphix create a new full backup file and load it. Note that when Delphix creates the backup, it is dumped to Delphix storage, not the Backup Location specified in the next step. It will dump to the staging database's NFS mounted "temp" directory on the server hosting the staging ASE instance. This means that this option will work under two scenarios:

        1. If the source ASE instance resides on the same server as the staging ASE instance, the staging database's NFS mounted "temp" directory will be present for the source database to write to in response to the "DUMP DATABASE" command.

        2. If the source and staging ASE instances are configured to allow remote access to the backup servers and the dSource is linked using the "Remote Server" option as described below. Delphix will then issue the "DUMP DATABASE" command and append the "AT <staging_backup_server_name>" clause so that the dump is written to the staging backup server.

      2. Most Recent Existing Full Backup – Find the most recent existing full backup file in the Backup Location and load it.

      3. Specific Existing Full Backup – Specify which existing backup files in the Backup Location you want to load.


      When using a dump taken with the deprecated compression syntax, select the Specific Existing Full Backup option for Initial Load and, for each stripe, type compress::<file name> into the text box.
    15. Enter the Backup Location. This is the directory where the database backups are stored. Delphix recursively searches this location, so the database backups or transaction logs can reside in any subdirectories below the path entered.

    16. Select Staging Environment and ASE instance name.

    17. Enable or disable Validated Sync Mode. Validated Sync Mode (also known as ValidatedSync) is the background process that monitors the ASE Backup Server's log file for new database or transaction log dumps. When Delphix detects a new dump is available it attempts to load it into the staging ASE database.

    18. Enable or disable LogSync. LogSync copies the transaction logs to Delphix storage which enables provisioning VDBs from a specific point in time in rather than just a particular backup.


      LogSync support limitations

      Due to ASE CR 800569, Delphix can only support transaction logs generated in intervals greater than one minute apart in ASE versions 16.0 SP02 through ASE 16.0 SP02 PL04. This ASE bug inadvertently removed the second and millisecond precision from the dump header sequence dates preventing Delphix from knowing what order to apply the transaction logs in when there are multiple transaction logs dumped within the same minute.



    19. Select Backup Location Type.

    20. Click Advanced to edit Source of Production Dump, External Data Directory, Retention policies, or Dump Password.




      External Data Directory

      The External Data Directory feature is not currently used with ASE dSources and is targeted for removal in a future release of Delphix.





      Remote Server should be selected when database dumps are not accessible on the Staging Environment. This options can be used with any of the initial load selections (New Full Backup, Most Recent Existing Full Backup or Specific Existing Full Backup). If selected, fill out additional settings as needed:

      1. Enter the Remote Server Name. This is the name of the backup server used when the dump was created.

      2. Select the Remote Host and Remote User that the backup server is located on.

      3. As noted, the interfaces file on both the staging and remote environments must be modified to point at each other's backup servers.



      Caution

      The Create Dump Password sets a dump password for the dSource. Select this only if the dump password option was used to create a password on the database dump or transaction log dump files. Setting this option causes Delphix to add the "WITH passwd=" clause to the "LOAD" commands.
    21. Click Next.

    22. Specify any Pre and Post Scripts.

    23. Review the dSource Configuration and Data Management information.

    24. Click Submit.

The Delphix platform will initiate two jobs, DB_Link and DB_Sync, to create the dSource. You can monitor these jobs by clicking Actions in the top menu bar, or by selecting System > Event Viewer. When the jobs have successfully completed, the database icon will change to a dSource icon on the Environments > Databases screen, and the dSource will be added to the list of My Datasets under its assigned group.


Dataset details

After you have created a dSource, you can view and edit information about it in the Dataset Details view by selecting the dataset from the dataset list. The center of the screen will show information about the status, timeflow, and configuration of the dSource.


Provisioning a SAP ASE VDB

Prerequisites

Procedure

    1. Login to the Delphix Management application.

    2. Click Manage

    3. Select Datasets.

    4. Select a dSource.

    5. Select a means of provisioning.

      For more information, see Provisioning by Snapshot and LogSync.

    6. Click Provision.

      The Provision VDB panel will open, and the Instance and Database Name fields will auto-populate with information from the dSource.

    7. Select whether to enable Truncate Log on Checkpoint database option for the VDB.

    8. Click Next.

    9. Select a Target Group for the VDB.

      Click the green Plus icon to add a new group, if necessary.

    10. Select a Snapshot Policy for the VDB.

      Click the green Plus icon to create a new policy, if necessary.

    11. Click Auto VDB Restart to enable VDBs to be automatically restarted when staging/target host gets rebooted, if necessary.

    12. Specify any Hooks to be used during the provisioning process.

      For more information, see Customizing SAP ASE Management with Hook Operations.

    13. If your Delphix Engine system administrator has configured the Delphix Engine to communicate with an SMTP server, you will be able to specify one or more people to notify when the provisioning is done. You can choose other Delphix Engine users or enter email addresses.

    14. Click Submit.

When provisioning starts, the VDB will appear in the Datasets panel. Select the VDB and navigate to the Status tab to see the progress of the job. When provisioning is complete, more information can be seen on the Configuration tab.

Automatic VDB Restart on Target Server After Reboot

The Delphix platform now automatically detects whether a target server has been rebooted, and proactively restarts any VBD on that server that was previously up and running. This is independent of data platform. It is done as if you realized a target server was restarted and issued a start command from the Delphix platform. This feature is compatible with Jet Stream ordering dependencies and is limited to non-clustered VDBs.

To enable automatic restart, complete the following steps:

    1. When provisioning a new VDB in the VDB Provisioning wizard, check the Auto VDB Restart box.

Once the VDB has been provisioned, you will be able to turn Automatic VDB Restart on.

    1. In the Datasets panel, select the VDB.

    2. Select the Configuration tab.

    3. Select the Source sub-tab.

    4. Select Database edit.


Next Steps

Congratulations! You have provisioned your first virtual database!

Now, perform some simple functional tests with your application. You can connect your app to the VDB using standard TNS/JDBC techniques. Delphix has already registered the VDB for you on the target listener.

We suggest the following next steps:

    1. Drop a table and use the VDB Rewind feature to test recovery of your VDB.
    2. Take a snapshot of your dSource and refresh your VDB to quickly get fresh production data.
    3. Provision a new VDB from your VDB to test sharing data quickly with other teams.
    4. Mask your new VDB to protect sensitive data. Provision new VDBs from that masked VDB to quickly provide safe data to development and QA teams.

Appendix

ASE Setup Checklist


UNIX Systems Administrator Tasks

Operating System User

File permissions

Operating System

Network Administrator Tasks

DBA Tasks

Source ASE instance

      • Grant SELECT to sysdatabases, sysusers and syslisteners

Target and Staging ASE instances

      • Grant sa_role and sybase_ts_role

Delphix Administrator Tasks