This page provides definitions of major concepts.

Components of the Delphix Dynamic Data Platform

TermExplanation
Delphix Dynamic Data Platform

The combined suite of products for virtualizing, masking, and working with your data.

Delphix ManagementFormerly called the Admin App. The interface for Delphix administrators to adjust settings and policies.
Delphix Self-ServiceFormerly called Jet Stream. Enables application development teams to get data on demand.
Delphix ReportingFormerly called Mission Control. A centralized reporting and auditing appliance for overseeing multiple Delphix deployments.
Delphix MaskingThe Masking capability of the Delphix Dynamic Data Platform represents an automated approach to protecting non-production environments, replacing confidential information such as social security numbers, patient records, and credit card information with fictitious, yet realistic data. 

Terms for Using the Delphix Engine

Ways to Access the Delphix Engine

TermExplanation
Application Programming Interface (API)

A method by which you can access a Delphix Engine programmatically.


Command Line Interface (CLI)A method by which you can access a Delphix Engine using SSH, which supports input of text commands.
Graphical User Interface (GUI)A method by which you can access a Delphix Engine using a web browser.

Delphix Virtualization Concepts

TermExplanation
Blocks or data blocksPhysical chunks of data that contain Relational Database Management System (RDBMS) rows. RDBMS databases are made up of files containing data blocks, which are backed up and restored with traditional database tools or virtualized with Delphix.
Dataset

A generic term for Delphix virtualized objects from which you can create virtual databases (VDBs).

A dataset can be a dSource, VDB, or vFiles.

Data source

The system, typically an RDBMS, that feeds information to the Delphix Engine, and from which virtual objects are derived. A data source can be a database, an application, or a set of unstructured files.

Not to be confused with a dSource, which is a virtualized, compressed duplicate of this database. (See below.)

Dataset Groups

Object groups are arbitrary collections of dSources or VDBs used for organization.

Note: Objects cannot be moved between groups.

Delphix ConnectorA service that runs on a Windows proxy host and enables communication between the Delphix Engine and the Windows Target Environment where it is installed.

dSource

Is an object on the Delphix Engine that is a virtualized representation of a database that the Delphix Engine uses to create and update virtual copies of your database. As a virtualized representation of your source data, it cannot be managed, manipulated, or examined by database tools. Because dSources are simply source data, you must provision a VDB in order to distribute/clone/test the data being pulled in. VDBs can also later be refreshed from the same or other points in time synced from the dSource.

Environment

Connection details, host information, and data source information for Delphix to communicate with a host or cluster.


HooksDelphix initiated calls to external scripts used to automate tasks, primarily on VDBs and dSources.
HostThe physical or logical machine that contains database instances. A host can be distinguished from an environment because the host has a physical reference point in its IP address. For example, you can specify a host (by referring to its host name or IP address) that a Delphix Environment points to.
HostChecker

A standalone program which validates that host machines are configured correctly before the Delphix Engine uses them for syncing from data sources or provisioning VDBs.

HostChecker is a Delphix script that you should run before adding any Environment to your Delphix Engine. It is available to download from the HostChecker subfolder at download.delphix.com.

LogSyncThis feature which enables the ingestion and retention of more granular (log-based) source change data. This more granular change data allows for VDB point-in-time provision, refresh, or rollback.
Replicas

Copies of the source Delphix Engine information on the target Delphix Engine, which can include objects such as dSources, VDBs and vFiles. Replicas preserve object relationships and naming nomenclatures. Replicas do noot copy platform specific configurations such as users, permissions etc.

Replication

You can replicate datasets between two instances of the Delphix Engine. Replication consists of a profile-replica pair. It is configured on the source Delphix Engine and copies specified objects to a target Delphix Engine. The source engine then sends incremental updates manually or according to a schedule. In addition, you can provision VDBs from replicated objects, allowing for geographical distribution of data and remote provisioning.

Replication profileReplication configuration on the source Delphix Engine. Formerly called "Replication spec."
Snapshots

Snapshots represent the state of a dataset at a specific moment in time. They accumulate over time from policies or are generated by manual creation. Snapshots allow you to choose a point in time from which to provision, refresh, or rollback.

If you have LogSync enabled, you can provision refresh or rollback from a point in time between the snapshots endpoints.

Note: Point in time provisioning is also dependent on log retention.

SnapSync

The standard process for importing data from a linked source into the Delphix Engine. An initial SnapSync is performed to create a dSource on the Delphix Engine. Incremental SnapSyncs are performed to provide additional points in time to the dSource on the Delphix Engine.

Source database

The original (either physical or virtual) database. It is commonly a production database, although it could be any database that the user designates as a source. Delphix creates a dSource from the source database.

Source EnvironmentAn environment from which the Delphix Engine can capture data.
Staging EnvironmentAn environment suitable for facilitating resource-intensive portions of the linking process and SnapSync.
Target EnvironmentAn environment on which the Delphix Engine can create VDBs.
TimeflowThe collection of Snapshots created by SnapSync policies or, in the case of SQL Server, the pre-provisioning process. When you provision or refresh a VDB, you pick a point in the Timeflow of a dSource or another VDB from which to provision.
Unstructured filesData stored in a filesystem that is NOT usually accessed by a DBMS or similar software. Unstructured files can consist of anything from a simple directory to the root of a complex application like Oracle E-Business Suite. It is a dataset that is treated as simply a directory tree full of files. Like with other data types, you can configure a dSource to sync periodically with a set of unstructured files external to the Delphix Engine. Virtualized unstructured files are called vFiles (see below).
Validated SyncThe process that runs on a staging database within a Staging Environment, and which executes either before a snapshot is taken (SQL Server) or after a snapshot is taken (Oracle).
vFiles

Virtual unstructured files. A virtual copy of data files created and managed by Delphix. vFiles are fully functional read/write copies of the original unstructured files source. You can mount vFiles across one target environment or several.

vFiles (Empty)Creating an Empty vFiles places an initially-empty mount on Target Environments. You can then create data directly on Delphix. This is useful when you have no existing files to copy into the Delphix Engine, but you do have files which you will generate, track, and copy with vFiles. For more information see Creating Empty vFiles from the Delphix Engine.
Virtual Database (VDB)A database provisioned from either a dSource or another VDB which is a full read/write copy of the source data. A VDB is created and managed by the Delphix Engine.
Virtual datasetComprehensive term that includes VDBs and vFiles.
V2PVirtual to Physical. This refers to the process of moving a VDB to a physical database, for example in a disaster recovery situation.

Data Operations

The terms below describe actions you can perform on a Delphix Engine.

TermExplanation
LinkThe process of establishing a relationship between a data source and the Delphix Engine. After linking a data source, the Delphix Engine can import data periodically and manage it as it evolves over time. In the GUI, synonymous with "Add dSource."
MaskMasking replaces sensitive data with fictitious data in non-prod environments (such as VDBs). It provides realistic data with which to work while reducing security risks. For more details about masking, see Masking Terms below.
Migrating a VDBMoving a VDB to a new Target Environment.
ProvisionCreate a new VDB from a dSource or VDB.
Refresh

Refreshing a VDB will re-provision it from the dSource. As with the normal provisioning process, you can choose to refresh the VDB from a Snapshot or a specific point in time. Refreshing a VDB will delete any changes that have been made to the VDB prior to the refresh operation; you are essentially resetting it to the state you select during the refresh process.

Rewind

Rewinding a VDB rolls it back to a previous point in its Timeflow. The VDB will no longer contain changes that occurred after the rewind point. This also creates an alternative Timeflow.

Delphix Virtualization Users and Privileges

ObjectUser PrivilegesGroup Privileges
Reader

Access statistics on the dSource, VDB, or Snapshot such as usage, history, and space consumption

Access statistics on all dSources, VDBs, or Snapshots in the group such as usage, history, and space consumption

Provisioner
  • Access statistics on the dSource, VDB, or Snapshot such as usage, history, and space consumption
  • Provision VDBs from owned dSources and VDBs
  • Access statistics on all dSources, VDBs, or Snapshots in the group such as usage, history, and space consumption
  • Provision VDBs from all dSources and VDBs in the group
Owner
  • Provision VDBs from owned dSources and VDBs
  • Perform Virtual to Physical (V2P) from owned dSources
  • Access the same statistics as an Reader
  • Refresh or Rollback VDBs
  • Take Snapshots of dSources and VDBs
  • Provision VDBs from all dSources and VDBs in the group
  • Refresh or Rollback all VDBs in the group
  • Snapshot all dSources and VDBs in the group
  • Perform Virtual to Physical (V2P) from owned dSources
  • Apply custom policies to dSources and VDBs
  • Create template policies for the group
  • Assign Owner privileges for dSources and VDBs
  • Access the same statistics as an Provisioner, Data Operator, or Reader
Data operator
  • Access statistics on the dSource, VDB, or snapshot such as usage, history, and space consumption
  • Refresh or rollback VDBs
  • Access statistics on all dSources, VDBs, or snapshots in the group such as usage, history, and space consumption
  • Refresh or rollback all VDBs in the group
sysadmin userCan perform typical system administration duties such as: modifying NTP, SNMP, SMTP settings; managing storage; downloading support logs for the Delphix Engine; and performing upgrades and patches. The sysadmin user launches the initial Delphix Setup configuration application and has access to the Command Line Interface (CLI).Has privileges for storage, upgrades, network etc.

Types of Notification

TypeNotification
EventCompletion of some action in the Delphix Engine. Examples include user-initiated tasks such as snapshots or VDB provisioning, policy-based tasks, and background monitoring and maintenance tasks.
Alert

Caused by a single event on a Delphix Engine. Also known as a System Event, and viewable through the System Event Viewer. Examples include warnings on source/target environment settings, recoverable errors, or incorrect connection settings.

Alert Levels:  Informational, Warning, Critical

Fault

A persistent event on a Delphix Engine that remains until the issue is resolved. The fault may be marked resolved automatically or require that it be resolved manually. Selecting to Ignore a fault will also ignore future faults of that exact type against the same object.

System faults describe states and configurations that may negatively impact the functionality of the Delphix Engine and which can only be resolved through active user intervention.

Examples:  Delphix Engine storage failure, Communication failures between the Delphix Engine and a source or target environment/host

Fault Levels:  Warning, Critical

 

Delphix Self-Service (Jet Stream) Terms

TermExplanation
Administrator

Has full access to all report data and can configure and administer Delphix Self-Service. Additionally, can use the Delphix Engine to:

  • add/delete reports
  • add/delete users
  • change tunable settings
  • add/delete tags
  • create and assign data templates and containers
Bookmark

A logical reference to a point in time on a branch. You can use it as a point from which to fork new branches. It can also be the target of policies – for example, you can arrange to keep this bookmark for two years.

Bookmarks are a way to mark and name a particular moment of data on a timeline. You can restore the active branch's timeline to the moment of data marked with a bookmark. You can also share bookmarks with other Delphix Self-Service users, which allows them to restore their own active branches to the moment of data in your container. The data represented by a bookmark is protected and will not be deleted until the bookmark is deleted.

Branches

Branches are task-specific groupings you can create within a data container. A branch is used to track a logical task, and contains a timeline of the historical data for that task. As you work within your data container, you can create more branches over time to run or complete separate tasks.

Branches represent a logical sequence of activity, separate from the underlying data lineage. This is the main concept introduced in the core engine and forms the basis of many higher level primitives. Branches:

  • Can have only one timeline active at any time
  • Can be user-visible (e.g. exported to a user target) or implementation (e.g. just a staging source to run a series of transformations)
Branch group / target groupA collection of multiple Branches that are treated as a single entity. The system can determine compatibility automatically, or a template can be used to create more complex orchestration.
Branch timelineA dynamic point-in-time interface for user actions within the Branch. Common activities include re-setting Data Sources to run a test, refreshing the Data Container with the most current source data, and bookmarking data to share or track interesting moments of time along the branch timeline.
Data container

Consists of one or more Data Sources, such as databases, application binaries, or other application data. Allows users to:

  • Undo any changes to their application data in seconds or minutes

  • Have immediate access to any version of their data over the course of their project

  • Share their data with other people on their team, without needing to relinquish control of their own container

  • Refresh their data from production data without waiting for an overworked DBA
Data templateCreated by the Delphix Administrator, data templates consist of the data sources users need in order to manage their data playground and their testing and/or development environments. Data templates serve as the parent for a set of data containers that the administrator assigns to Delphix Self-Service users. Additionally, data templates enforce the boundaries for how data is shared. Data can only be shared directly with other users whose containers were created from the same parent data template.
Data UserDelphix Self-Service data users have access to production data provided in a data container. The data container provides these users with a playground in which to work with data using the Self-Service Toolbar.

Delphix Reporting (Mission Control) Terms

TermExplanation
Admin user

Admin users have full access to all report data and can configure the Delphix Reporting appliance. (They can also assign auditor users a set of tags to restrict which report data they can view. There is no default auditor account. The first Delphix Administrator will need to create the auditor users and will be responsible for creating their User IDs and Passwords.)

An Admin user can:

  • Add/delete reports
  • Add/delete users
  • Change tunable settings
  • Add/delete tags
Auditor userAuditor users can only view report data.
Reports

Reports present aggregated data across all connected Delphix Engine .

Interactive reports such as Storage Breakdown and History display interactive graphical representations of historical and current storage usage across all Delphix Engines you are monitoring. These visualizations of storage and disk capacity enable you to analyze and mediate storage across Delphix Engines from multiple perspectives.

TaggingYou can tag Delphix Engines in Delphix Reporting with a set of arbitrary text strings. You can then filter reports to show only data from Delphix Engines with a certain tag. You can also use tags to restrict auditor users so that they can only view data from Delphix Engines with that tag.

Delphix Masking Engine Terms

TermExplanation
ApplicationThe IT assets (programs, data, processes) that support a business function. For example, if a bank offers payroll services to its clients, there would be an application in its IT division to support that business.
ConnectorWhere the Delphix Engine stores JDBC database connection information. Builds a connection between the source database and the Delphix interface for data masking.
DomainThe Domain represents the correlation between various sensitive data categories and the masking algorithm which will be applied to them
EngineSyncThe ability to coordinate the use of masking algorithms across multiple Delphix Engines.
Masking EnvironmentDefines a collection of masking constructs (connectors, rule sets / inventories, and jobs) that support masking for a given application environment. In order to mask databases and files within the Delphix Engine, you first need to create an environment in which the Delphix Engine will store the connection information and masking rules for those data stores. An Environment can contain multiple database connections and multiple file connections. Environments are connected to applications for informational purposes.
In-place masking"Mask data in place" refers to updating a database with masked data. This includes reading data from the table defined in the Rule Set, masking the data with the Delphix Engine, and updating the tables with the masked data.
InventoryThe Delphix Engine automatically stores the masking rules for each sensitive column in the repository database in the Environment's "inventory." When you select a table to mask, its columns will appear and you can select them for masking. Afterwards, you can edit the columns with an appropriate algorithm required for masking.
Masked VDBA virtual database with masked data.
On-the-fly maskingWith on-the-fly masking, you specify the source of the information to be masked, and where the masked data will be loaded. On-the-fly masking is an Extract Transform Load (ETL) process.
Profile dataA way to identify the location of Non-Public Information (NPI) or sensitive data if you are unsure of what data needs to be masked in the first place. Profiling data is not necessary when you have already identified the sensitive data you need to mask.
Rule SetA Rule Set points to a collection of tables or flat files that Delphix uses for masking data. The Rule Set allows you to identify, select, and configure which tables you need to mask. For those tables that do not have a primary key defined, you can define a logical key with a combination of columns (or ROWID for Oracle database).
Selective Data Distribution (SDD)Permits the distribution of masked data between Delphix Engines. The sources received on a target Delphix Engine do not include and secure the original parent source, thereby making the original source inaccessible from the target.

Masking Algorithms

AlgorithmDescription
Secure LookupThe most commonly used type of algorithm. It is easy to generate and works with different languages. This algorithm replaces real, sensitive data with fictional data, it is possible that it will create repeating data patterns, known as “collisions.” For example, the names “Tom” and “Peter” could both be masked as “Matt.” Because names and addresses naturally recur in real data, this mimics an actual Data Set. However, if you want the Delphix Engine to mask all data into unique outputs, you should use segmented mapping, described below.
Segmented MappingProduces no overlaps or repetitions in the masked data. You can mask up to a maximum of 36 values using segmented mapping. You might use this method if you need columns with unique values, such as Social Security Numbers, Primary Key columns, or foreign key columns. You can set the algorithm to produce alphanumeric results or only numbers.
Mapping

Allows you to state what values will replace the original data. There will be no collisions in the masked data, because it always matches the same input to the same output. For example “David” will always become “Ragu,” and “Melissa” will always become “Jasmine.” The algorithm checks whether an input has already been mapped; if so, the algorithm changes the data to its designated output. You can use a Mapping Algorithm on any set of values, of any length, but you must know how many values you plan to mask.

Note: When you use a Mapping Aalgorithm, you cannot mask more than one table at a time. You must mask tables serially.
Binary LookupReplaces images that appear in object columns. For example, if a bank has an object column that stores images of checks, you can use a binary lookup algorithm to mask those images. The Delphix Engine cannot change data within images themselves, such as the names on X-rays or driver’s licenses. However, you can replace all such images with a new, fictional image. This fictional image is provided by the owner of the original data.
Tokenization

The only type of algorithm that allows you to reverse its masking. For example, you can use a tokenization algorithm to mask data before you send it to an external vendor for analysis. The vendor can then identify accounts that need attention without having any access to the original, sensitive data. Once you have the vendor’s feedback, you can reverse the masking and take action on the appropriate accounts.

Like mapping, a tokenization algorithm creates a unique token for each input such as “David” or “Melissa.” The Delphix Engine stores both the token and the original so that you can reverse masking later.

Min MaxValues that are extremely high or low in certain categories allow viewers to infer someone’s identity, even if their name has been masked. For example, a salary of $1M suggests a company’s CEO, and some age ranges suggest higher insurance risk. You can use a Min Max algorithm to move all values of this kind into the midrange.
Data CleansingData Cleansing does not perform any masking. Instead, it standardizes varied spellings, misspellings, and abbreviations for the same name. For example, “Ariz,” “Az,” and “Arizona” can all be cleansed to “AZ.”
Free Text Redaction

Helps remove sensitive data that appears in free-text columns such as “Notes.” This type of algorithm requires some expertise to use, because you must set it to recognize sensitive data within a block of text.

One challenge is that individual words might not be sensitive on their own, but together they can be. The algorithm uses profiler sets to determine what information it needs to mask. You can decide which expressions the algorithm uses to search for material such as addresses. For example, you can set the algorithm to look for “St,” “Cir,” “Blvd,” and other words that suggest an address. You can also use pattern matching to identify potentially sensitive information. For example, a number that takes the form 123-45-6789 is likely to be a Social Security Number.

You can use a free text redaction algorithm to show or hide information by displaying either a “black list” or a “white list.”