Release 5.2.5

This release has several significant new features that will change the way our customers use our products. 

5.2.5 New Features

Multiple NIC Support for Oracle and ASE

Delphix previously only allowed one IP Address to be specified per host. Now, customers can specify a list of IP addresses for Oracle and ASE instances that will accept mount traffic from Delphix. For more information please see, Adding an SAP ASE Environment and Adding an Oracle Single Instance or RAC Environment. 

NetBackup Ingestion Support for SQL Server

In 5.2.5, Delphix introduces the ability to use NetBackup backups to sync SQL Server databases. Existing NetBackup backups can be used as sources to sync an initial backup and to stay current with the production data source. NetBackup integration will now be part of the environment discovery workflow. This feature will allow for much faster SQL server deployments. For more information please see, Linking a dSource from a NetBackup SQL Server Backup.

Multiple Backup Locations Support for SQL Server

With this release, Delphix enables users to enter multiple backup locations to search for backups when adding new data sources. Many customers store their full and transaction log backups in different locations and this will allow them to more easily get data into Delphix. For more information please see, Delphix as a Backup Solution to SQL Server and Linking a SQL Server dSource

Enhanced TLS Support

In 5.2.4 Delphix added the ability for customers to upload and manage their own HTTPS certificates using the CLI. Starting in 5.2.5 Delphix is introducing the ability to upload and manage the host’s DSP certificates. An intuitive UI has be added for customers to manage all certificates. For more information please see, Certificate Management.

Terminating long-running EBS appsTier processes

In 5.2.5, customers get a configurable option to define a timeout value to terminate running appsTier processes if the value is exceeded.

Upgrade and downgrade Testing for DB2 Fix Packs

Starting in 5.2.5, Delphix allows customers to perform upgrade/downgrade testing between the fix packs of a major DB2 version. Upgrade testing allows our customers to validate a new or untested DB2 fix pack version before deploying it in Production or before an application rollout. The downgrade testing is a less common scenario. Customers who are facing issues after an upgrade to a newer fix pack can easily downgrade and test the previous, stable DB2 fix pack version in their non-production environments before making the changes in production.

Support for different ASE patch level versions between the source and staging hosts

Starting in 5.2.5, for all currently supported ASE versions, Delphix allows customers to use different ASE patch level versions between the source and staging hosts. The major version of ASE must still be the same. The only caveat is for ASE 15.7 - where the source and staging patch levels must both be either any version below SP64 or both be any version above SP100.

Key Product Updates

• The Virtualization Engine deployment with Masking update to 5.2.5.0 requires both the Virtualization and Masking Engines to be upgraded to 5.2.5.
• AWS r4.2xlarge, r4.4xlarge, and r4.8xlarge Support: Delphix Dynamic Platform can run on AWS memory-optimized R4 Instances starting with this release. Customers with existing AWS accounts can run our engines on the latest released AWS R4 instances. R4 instances are more cost-effective than the earlier-generation R3 instances. Migration from R3 to R4 is a one-time effort.
• AWS i3.2x, i3.4xlarge, i3.8xlarge Support: We will be adding support for the next generation of the storage-optimized i3 AWS instance types. The cost and performance benefits of the i3 instances make it a natural transition for customers currently using the i2 instances.
• Custom Algorithm (Mapplet) Manageability: Custom algorithms (also commonly referred to as mapplets) are extensions to the main Masking product that allow Delphix Professional Services to create custom functionality for customers. These custom algorithms are not supported by Delphix Support but rather by our services team. One issue that has arisen in the past is that we do not clearly indicate in the UI or in our logs when a custom algorithm is being used. In 5.2.5 we are aiming to make it much clearer in the UI and in the logs when a custom algorithm is being used.
• Enhanced ingestion for DB2 databases: In the past, customers have reported considerably slower performance during ingestion of larger DB2 databases. In 5.2.5, we are making product improvements to enable faster ingestion of DB2 databases in Delphix. The performance will be noticeably better for larger DB2 databases.
• Update of Target APPS Password: During EBS provisioning and refresh operations, we had a security issue where the target APPS password was not updated with the user-supplied password but instead was incorrectly assigned with the production password. This issue was fixed to ensure that the user-supplied password is updated in the Target during provisioning and persists through the Delphix operations.

New Certifications:

In each release, Delphix aims to keep up to date on the latest database platforms. As a reminder, we’ve currently shipped all relevant SQL Server certifications, so there will be fewer updates here.

• Oracle 12c R2 Masking:  Users will be able to specify the particular PDB they would like to connect to through the Advanced Oracle Connector (using the JDBC URL parameter).

Release 5.2.4

This release has several significant new features that will change the way our customers use our products. 

5.2.4 New Features

TLS Certificate Upload

In 5.2.4 Delphix supports uploading TLS certificates via the virtualization command line. For more information please refer to Replacing the HTTPS (HTTP Secure) Certificate.

Customize DB2 dSource and VDB Parameters

Customers’ production systems are normally performance-optimized servers configured with large amounts of memory, buffer size, etc. However, their non-production systems are often resource-optimized and differ significantly in configuration. With 5.2.4, customers have the ability to modify different dSource and VDB parameters such that they match the resources available on the staging and target hosts. These settings can either be made during initial setup and deployment time or be changed for an active dSource or VDB.

Expanded Masking File Format Upload

In addition to relational databases, the Masking Engine supports masking files. For the Masking Engine to properly interpret a file’s structure, users upload a format file. In 5.2.4, we have expanded support for uploading format files using a web browser to XML and VSAM copybooks. For more information please refer to Managing File Formats.

Using DB2 Databases with Raw Devices

Several customers use raw devices-based tablespaces for source DB2 databases in their environments. To leverage these environments with Delphix, we have built a workflow using DB2s native tools that allow us to discover and convert a raw device-based tablespace into an automatic storage-based tablespace during ingestion. Once ingested, customers will be able to provision VDBs of the automatic storage-based database.

Key Product Updates

Delphix has made a number of product changes that merit mentioning:

Jackson JSON Library Update

There were several well-documented vulnerabilities in the open source Jackson library which we used in our SQL Server Virtualization connector. In this release, Delphix has updated to the latest stable version, Jackson Release 2.9.4.

Masked Provisioning User Experience Improvement:

In this release, Delphix has introduced the ability to mark VDBs as masked without selecting a masking job at the time of provisioning. This enables customers to provision masked VDBs and use Selective Data Distribution (SDD) without referencing a previously defined masking job. This simplifies the user workflow of masking a database using a hook script and allows customers using third-party masking tools to setup masked provisioning without requiring a Masking Engine.

Multiple Recovery Paths for SQL Server

When a SQL Server database is restored, as in the case of disaster recovery or an error in the database instance, SQL Server generates a new recovery path.  Support for multiple recovery paths enables customers to roll back SQL Server source databases and refresh VDBs from those sources within Delphix.

User-click Analytics with Virtualization 

• In Delphix 5.2.4, we are releasing the User-click Analytics feature that is a lightweight method to capture how users interact with our product’s user interfaces.

• The goal behind capturing this data is to get a better understanding of product usage, engagement, and user behavior, and use this data to improve our product and customer experience.

• We will collect browser-based, user-click analytics data. Essentially, we will selectively track actions that a user takes in the product UI. We will not be collecting, transmitting or storing any personally identifiable information (PII) which for Enterprise customers includes information such as email address, IP address, username, etc

• This feature setting will match an Engine’s Phone Home Service setting:

• If the Phone Home Service is enabled on an Engine, then this feature will be enabled by default.  

• If the Phone Home Service is disabled on an Engine, then this feature will be disabled by default.

• If you want to opt-out please see these CLI instructions - CLI Cookbook: Disabling User-click Analytics. The opt-out is not currently available in the GUI. Please note that we would like to encourage our customers to share this non-PII data with us.

New Virtualization Certifications

The following Support Matrices have been updated to reflect updated certification of data sources and OS versions:

• Oracle 12.1 on Solaris 11 U3 SPARC

Release 5.2.2

Delphix Dynamic Data Platform 5.2 introduces several major changes that will dramatically improve our customers’ experience and expand functionality

Core Virtualization Capabilities

• New Delphix Management UI: A completely revamped GUI simplifies the user experience. Highlights include:
  • Flash no longer required for the user interface.
  • Improved navigation and page layout.
  • Enhanced Timeflow that allows easier review of snapshots.
  • Redesigned Capacity Management screen: Quickly view which VDBs and snapshots are taking the most disk space for easier capacity management.
  • Dataset performance screen: Clearer, easier way to monitor the health and performance of VDBs.
  • Consistent Performance: UI performance as good or better than the 5.1 release

• Improved Upgrade: Several new features make upgrade much faster and simpler, progressing towards the long-term goal of self-service. Note: these features will only be available when upgrading from 5.2 to a future release, and our first rollout will still be driven with the help of Delphix Support.

  • Automated rollback to a stable point on failed upgrade.
  • Automated check process to prevent upgrading if Delphix is using a hotfix that the target upgrade does not contain.

  • Built-in checks prior to upgrade to help ensure the Delphix platform is configured and prepared properly to successfully upgrade.

• TCP Congestion Control Algorithm Modernization: Increased throughput for workloads such as replication and Snapsync where there are usually high latency and congested network paths.

• Filesystem (ZFS) improvements: A number of ZFS-level changes will improve I/O performance, particularly on VDBs which have heavy write workloads:

  • Lower latency for multiple synchronous write, especially under heavy workloads. Most writes to VDBs are synchronous.

  • Improved performance for high-IOPS workloads, which are common on all-SSD systems. Helps especially with a high number of vCPUs, and on cached reads and asynchronous writes (most writes to dSources are asynchronous).

• Password Security: Ability for customers to define login password requirements along multiple variables including length, capitalization, and symbols.

• vSphere 6.5: Running the Delphix platform on VMware vSphere  6.5 is now fully supported.

Data Virtualization Connectors

• Oracle

  • Removing support for Oracle 9i: Version 5.2 will no longer support Oracle 9i.

  • Introducing Virtual CDB (vCDB) Support: Traditional support for existing 12c MT operations required a physical CDB to be created in the target host (by the DBA) to provision vPDBs into. In addition to the manual overhead of creating a physical CDB, users could encounter provisioning errors due to incompabilities, storage consumption on the target host, and administrative overhead for creating and managing the physical CDBs. By introducing the concept of a virtual CDB (vCDB) Delphix enables customers to provision a vPDB along with its underlying vCDB in a single provisioning workflow. 

  • PDB Attach/Detach: There are several common scenarios where a source environment may move, such as host failover or disaster recovery. Within Oracle 12c MT, there is additional functionality to move a PDB from one CDB to another. Without Detach/Attach, there is no way to instruct the Delphix platform to follow the PDB: the “moved” PDB must be relinked to create a new dSource. In addition to being an expensive operation, relinking a PDB complicates database management, reduces storage savings and prolongs sync time. 

    PDBs can now be managed in such a way that they can be “moved” to a different environment and the Delphix platform will track those movements automatically. PDB Attach/Detach enables customers pointing the Delphix platform to the new location to gather incremental data for the PDB dSource and have continuity in their operations. A PDB may “move” in many different ways. Some examples are below:

• • • The PDB is unplugged from its CDB and plugged into a different CDB
    • The PDB and its constituent CDB are failed over to a different host that was previously a standby database
    • Swap linking from DataGuard primary PDB to DataGuard standby PDB

• • Automatic vCDB Restart on Target Host Reboot: Similar to the VDB auto-restart functionality, we will be adding support for the automatic vCDB restart for customers using multi-tenant configurations. 

  • Remove dependency on 32-bit glibc and move to 64-bit glibc support: Oracle periodically releases patches for Exadata. These patches can uninstall the non-white listed packages such as 32-bit glibc which results in Delphix failures. We will be eliminating the need for 32-bit glibc packages and moving to 64-bit glibc support.

  • Enhanced Local Listener Support: An Oracle listener is a server process that listens for incoming connections on one or more endpoints. Oracle database instances and services register with one or more listeners (via their endpoints) on database startup. To connect to these databases, clients first connect to one of the listener endpoints, whereupon the listener forwards the request accordingly. Frequently, security protocol requires using only the IPC protocol to set up their local_listener init parameter. This is a measure customers use to restrict registering their Oracle DB (including their VDBs provisioned from Delphix) to only use IPC local protocol to contact the local listener. This can help them to avoid dealing with the 'Oracle TNS Listener Attack' security bug. This feature adds support for IPC local listeners for the single instance (non-RAC) case. In total, we will support the discovery of TCP, TCPS, and IPC.

  • Discovery over DSP: The long-term goal for Delphix is to use DSP for all source and target operations. This addresses the concerns of our security-conscious customers who would like to leverage DSP for Delphix operations. As a first step, we built out support for SnapSync on DSP. In the 5.2 timeframe, we are adding support for Discovery over DSP. 

• DB2
• Instance to DB-level Support: DB2 on Delphix was initially designed to work at the Instance-level: an instance is a collection of DB2 databases. In addition to deviating from the design of our other supported data sources, this design had several challenges including management overhead in terms of setup and operations as well as technical limitations in areas such as provisioning, renaming of VDBs, etc. For 5.2, we will be removing this limitation and moving DB2 operations to the database-level. This enhancement would provide our customers with simplified administration and operations of the DB2 VDBs. It is important to note that we will not be providing customers with an upgrade path between the instance and the DB level which may lead to data loss, and impact on business continuity. In order to ensure our customers have a smooth transition, we will assist in our customers in this migration journey.
• DB2 BLU: DB2 BLU is a collection of technologies for analytical database workloads. BLU Acceleration design integrates different technologies including in-memory processing of columnar data, Actionable Compression, CPU Acceleration, Data Skipping. Virtualization of DB2 BLU with Delphix provides customers with not only storage savings but with agility, integration and federated DB support. Customers have expressed use cases where they leverage the self-service capabilities of JetStream to integrate data from different data sources such as DB2 (BLU and Non-BLU), SQL Server, etc.
• Deprecation: In 5.2, we will be deprecating support for DB2 10.1 and AIX 6.1 as IBM has removed support for these versions.
• Certifications: In all Delphix releases, we introduce support for new data source / operating system versions. The following combinations are certified in 5.2:
  • DB2 10.5 with RHEL 6.9
  • DB2 10.5 with RHEL 7.4
  • DB2 10.5 with AIX 7.2
  • DB2 11.1 with RHEL 6.9
  • DB2 11.1 with AIX 7.

• SQL Server

  • Removing support for Windows Server 2003: Version 5.2 will no longer support Windows Server 2003.

  • Compressed Copy-Only Backups: Customers can now compress their copy-only backups when linking a dSource or taking a snapshot, allowing them to save even more space on their backups taken with Delphix.

• Oracle EBS

  • EBS Apps Password Management: This feature would address the long-standing issue of password management with EBS in Delphix. The end to end workflow consists of provisioning from production, changing the APPS password in the source and updating Delphix's copy of the password. The solution consists of obtaining the apps password for dSource snapshot and saving it as per-snapshot metadata. We would explicitly test that the password provided is valid as most EBS preclone utilities don't require a password or don't validate it. When provisioning, Delphix would use the password from a snapshot

  • Validate EBS credentials at the beginning of operations: EBS provision/snapshot would sometimes not validate the credentials provided (ex. APPS, Weblogic) until it is actually being used which might belong down the way. In some cases, the scripts would fail with a cryptic error or abruptly. This Feature would validate the credentials provided earlier in the process and error out with an understandable message to the end user.

• SAP ASE

  • Removing support for ASE 12.5: Version 5.2 will no longer support ASE 12.5.

  • Certifications: In all Delphix releases, we introduce support for new data source / operating system versions. The following combination is certified in 5.2:

  • ASE 15.7 with AIX 7.2

Delphix Self-Service (formerly known as Jet Stream)

• Improvements of Error Messaging: users will be provided with more clear error messages. These improvements will help end users understand what error occurred and what steps they should take to resolve their issue.

• Automatic Retries for Self-Service Operations: In order to make self-service operations more robust, we have implemented an automatic retry feature when there are failures. This feature will improve the user experience by significantly reducing the probability of containers left in an error after using them.

• Bookmark Default Expiry Policy: admins are now able to set a default expiration date for all new bookmarks. This feature enables users to better manage their space utilization by making sure the expiration date is auto-filled when creating new bookmarks.

Delphix Masking Capabilities

• New Masking Wizard: We’re introducing a new masking wizard that will be the first experience users have when they use Delphix. The goals of this wizard include:
  • Enabling first-time users to easily complete their initial masking job and see the results of sensitive fields being masked.
  • Enabling seasoned users to get jobs done faster.
  • Taking a first step towards dramatically simplifying and automating the process of data masking.
• Completely New API: We’re introducing a completely new masking API that will allow programmatic control and masking at scale. This API is a major step forward to enabling true integration of masking into enterprise development and CI / CD workflows and to more easily deploy masking enterprise-wide. The API functionality that will be shipping in 5.2 includes:

  • Masking/Profile Job Control

  • Database/File Connector Control

  • Environment Control

  • Application Control

  • Profile Set/Profile Expression Control

  • Domain Control

  • Ruleset Control

• Masking Algorithm Synchronization: One of the major features of our masking product is our ability to maintain referential integrity (for example first name “Tom” is always changed to “Kevin”) while masking. Until now, this has only been the case for single masking engine deployments, not when deploying multiple masking engines. In 5.2 we are introducing the ability to have consistent masking output across multiple engines (no limit to the quantity) when using the following algorithms:

  • Secure Lookup

  • Binary Lookup

  • Date Shift Discrete

  • Tokenization

  • Segmented Mapping

  • Some custom algorithms (mapplets) (Please work with your services team to verify)

• Other Masking Features

  • New Welcome Screen: We’re introducing new masking login and welcome screens. These new screens have a modern design and offer a set of tools to help users get started with masking. These new screens include:

  • New instructional video - walking users through the masking wizard as well as the traditional masking UI.

  • Links to important documentation - getting started guide and quick start guide.

  • Link directly to the new Masking Wizard - letting users immediately get started creating a masking job.

  • New forgotten password workflow: Allows users to reset forgotten passwords without needing to contact their administrator (or Delphix).

  • Disable Constraints for SAP ASE (Sybase): The “Disable Constraints” option will now be available when running an ASE (Sybase) masking job. This makes it possible to mask tables with certain foreign key constraints.

  • Split application logs: Applications logs have now been split into different user-visible messages and developer-targeted logs. This enhancement makes it easier for support and users to find the log data they need to diagnose a problem.

  • Removal of connector limits: It is now possible to create an unlimited amount of connectors. Previously, this required opening a support case to go over 100.

  • Performance optimizations for identity columns without an index: There have been several performance improvements across relational databases.

  • SQL Server Deadlock: There have been several performance improvements for Microsoft SQL Server to avoid deadlock

  • VSAM support for custom algorithms (mapplets): We have added the ability to apply custom algorithms (mapplets) to VSAM files.

  • Removal of outdated custom algorithms (mapplets): We determined that some mapplets in the product are outdated and should not be used by customers.The following mapplets will be removed from the product:

  • DATE_SHIFT_VARCHART_DATE

  • FREE TEXT MASKING

  • FREE_TEXT_REDACTOR

  • INCLUSION_MAPPLET

  • Removal of unsupported connectors: We will remove many legacy, unsupported connectors from the Masking UI to correspond with our Masking Support Matrix. This discrepancy between our support matrix and the connectors listed in the product has previously caused confusion.

  • SMTP Settings in UI: Users will now be able to configure their SMTP directly through the Masking Admin UI instead of having to contact support.

  • General Security Enhancements: Variety of security enhancements around components like user passwords and connector credentials.