Release Information

Introduction

Getting Started

Delphix Engine System Administration

Virtual Database Management with the Delphix Engine

Data Backup and Recovery

Delphix Modernization Engine

Delphix Masking

Virtualizing Unstructured Files in the Delphix Engine

Virtualizing Oracle E-Business Suite in the Delphix Engine

JetStream

Mission Control

Delphix Express User Guide

Reference


Documentation for Previous Versions of the Delphix Engine

Delphix Server 3.0 Documentation
Delphix Server 3.1 Documentation
Delphix Engine 3.2 Documentation
Delphix Engine 4.0 Documentation
Delphix Engine 4.1 Documentation
Delphix Engine 4.2 Documentation
Delphix Engine 4.3 Documentation

Skip to end of metadata
Go to start of metadata

This section describes how to create and manage jobs. Tokenization uses reversible algorithms so that the data can be returned to it's original state. Tokenization is a form of encryption where the actual data – such as names and addresses – are converted into tokens that have similar properties to the original data (text, length, etc.) but no longer convey any meaning.

Creating a Tokenization Algorithm

  1. From the Home page, click Settings.
  2. Add Algorithm. You will see the popup below:
  3. Select Tokenization Algorithm.
  4. Enter a name and description.

Create a Domain

Once you have created an algorithm, you will need to associate it with a domain.

  1. From the Home page, click Settings.
  2. Select Domains.
  3. Click Add Domain. You will see the popup below:
  4. Enter a domain name and associate it with your algorithm.

Create a Tokenization Environment 

  1. From the Home page, click Environments.
  2. Click Add Environment. You will see the popup below:
  3. Select Tokenize/Re-Identify as the purpose.
  4. Click Save.

Note

This environment will be used to re-identify your data when required.

Create a Connection and Rule Set

At this point, you can proceed in the same fashion as if you were using Delphix to perform normal masking. You have made all the changes needed to use Tokenization (reversible) algorithms instead of Masking (irreversible) algorithms. Note it is possible to create two different environments for the same application one for Masking and one for Tokenization.

  1. From the Home page, click Environments.
  2. Click Connector.
  3. Click Create Connection. You will see the popup below:

Create the Rule Set and Apply File Format

If you are masking a file you will need to create a Rule Set and the associated File Format.

  1. From the Home page, click Environments.
  2. Click Rule Set.
  3.  Click Edit Rule Set. You will see the popup below:

Apply the Tokenization Algorithm

In Step 1, you created a tokenization algorithm you can now apply to the field. You will see the popup below:

Create and Execute a Tokenization Job

  1. From the Home page, click Environments.
  2. Click Tokenize.  
  3. Set up a Tokenize job using tokenization method. Execute the job.
  4. You will be prompted for the following information:
    1. Job Name — A free-form name for the job you are creating.
    2. Tokenization Method — Select Tokenization Method.
    3. Multi Tenant — Check box if the job is for a multi-tenant database.
    4. Rule Set — Select a rule set that this job will execute against.
    5. Generator
    6. No. of Streams — The number of parallel streams to use when running the jobs. For example, you can select two streams to run two tables in the ruleset concurrently in the job instead of one table at a time. (This option only appears if you select DMsuite as the Generator.)
    7. Remote Server — (optional) The remote server that will execute the jobs. This option lets you choose to execute jobs on a remote server, rather than on the local Delphix instance. Note: This is an add-on feature for Delphix Standard Edition. (This option only appears if you select DMsuite as the Generator.)
    8. Min Memory (MB) — (optional) Minimum amount of memory to allocate for the job, in megabytes. (This option only appears if you select DMsuite as the Generator.)
    9. Max memory (MB) — (optional) Maximum amount of memory to allocate for the job, in megabytes. (This option only appears if you select DMsuite as the Generator.)
    10. Commit Size — (optional) The number of rows to process before issuing a commit to the database.
    11. Feedback Size — (optional) The number of rows to process before writing a message to the logs. Set this parameter to the appropriate level of detail required for monitoring your job. For example, if you set this number significantly higher than the actual number of rows in a job, the progress for that job will only show 0 or 100%.
    12. Disable Constraint — (optional) Whether to automatically disable database constraints. The default is for this check box to be clear and therefore not perform automatic disabling of constraints. For more information about database constraints, see Enabling and Disabling Database Constraints.
    13. Batch Update — (optional) Enable or disable use of a batch for updates. A job's statements can either be executed individually, or can be put in a batch file and executed at once, which is faster.
    14. Truncate — (optional) Whether to truncate target tables before loading them with data. If this box is selected, the tables will be "cleared" before the operation. If this box is clear, data is appended to tables, which potentially can cause primary key violations. This box is clear by default.
    15. Disable Trigger — (optional) Whether to automatically disable database triggers. The default is for this check box to be clear and therefore not perform automatic disabling of triggers.
    16. Drop Index — (optional) Whether to automatically drop indexes on columns which are being masked and automatically re-create the index when the masking job is completed. The default is for this check box to be clear and therefore not perform automatic dropping of indexes.
    17. Prescript — (optional) Specify the full pathname of a file containing SQL statements to be run before the job starts, or click Browse to specify a file. If you are editing the job and a prescript file is already specified, you can click the Delete button to remove the file. (The Delete button only appears if a prescript file was already specified.) For information about creating your own prescript files, see Creating SQL Statements to Run Before and After Jobs.
    18. Postscript — (optional) Specify the full pathname of a file containing SQL statements to be run after the job finishes, or click Browse to specify a file. If you are editing the job and a postscript file is already specified, you can click the Delete button to remove the file. (The Delete button only appears if a postscript file was already specified.) For information about creating your own postscript files, see Creating SQL Statements to Run Before and After Jobs.
    19. Comments — (optional) Add comments related to this provisioning job.
    20. Email — (optional) Add e-mail address(es) to which to send status messages.
  5. When you are finished, click Save.

Result Snapshot

Here is a snapshot of the data before and after Tokenization to give you an idea of what the it will look like.

Before Tokenization

After Tokenization


Steps to Re-Identify

Use the Tokenize/Re-Identify environment.

  1. From the Home page, click Environments.
  2. Click Re-Identify.  
  3. Create a re-Identify job and execute.

    You will be prompted for the following information:
    1. Job Name — A free-form name for the job you are creating.
    2. Multi Tenant — Check box if the job is for a multi-tenant database.
    3. Rule Set — Select a rule set that this job will execute against.
    4. Generator — The default value is Delphix.
    5. No. of Streams — The number of parallel streams to use when running the jobs. For example, you can select two streams to run two tables in the ruleset concurrently in the job instead of one table at a time. (This option only appears if you select DMsuite as the Generator.)
    6. Remote Server—(optional) The remote server that will execute the jobs. This option lets you choose to execute jobs on a remote server, rather than on the local Delphix instance. Note: This is an add-on feature for Delphix Standard Edition. (This option only appears if you select DMsuite as the Generator.)
    7. Min Memory (MB) — (optional) Minimum amount of memory to allocate for the job, in megabytes. (This option only appears if you select DMsuite as the Generator.)
    8. Max memory (MB) — (optional) Maximum amount of memory to allocate for the job, in megabytes. (This option only appears if you select DMsuite as the Generator.)
    9. Commit Size — (optional) The number of rows to process before issuing a commit to the database.
    10. Feedback Size — (optional) The number of rows to process before writing a message to the logs. Set this parameter to the appropriate level of detail required for monitoring your job. For example, if you set this number significantly higher than the actual number of rows in a job, the progress for that job will only show 0 or 100%.
    11. Bulk Data — (optional) For In-Place masking only. The default is for this check box to be clear. If you are masking very large tables in-place and require performance improvements, check this box. Delphix will mask data to a flat file, and then use inserts instead of updates to bulk load the target table. (This option only appears if you select DMsuite as the Generator.)
    12. Batch Update — (optional) Enable or disable use of a batch for updates. A job's statements can either be executed individually, or can be put in a batch file and executed at once, which is faster.
    13. Drop Index — (optional) Whether to automatically drop indexes on columns which are being masked and automatically re-create the index when the masking job is completed. The default is for this check box to be clear and therefore not perform automatic dropping of indexes.
    14. Prescript — (optional) Specify the full pathname of a file containing SQL statements to be run before the job starts, or click Browse to specify a file. If you are editing the job and a prescript file is already specified, you can click the Delete button to remove the file. (The Delete button only appears if a prescript file was already specified.) For information about creating your own prescript files, see  Creating SQL Statements to Run Before and After Jobs.
    15. Postscript — (optional) Specify the full pathname of a file containing SQL statements to be run after the job finishes, or click Browse to specify a file. If you are editing the job and a postscript file is already specified, you can click the Delete button to remove the file. (The Delete button only appears if a postscript file was already specified.) For information about creating your own postscript files, see  Creating SQL Statements to Run Before and After Jobs.
    16. Comments — (optional) Add comments related to this provisioning job.
    17. Email — (optional) Add e-mail address(es) to which to send status messages.
  4. When you are finished, click Save.

Result Snapshot     

Here is a snapshot of the data before and after re-identification to give you an idea of what to expect.

Before Re-Identification


After Re-Identification