Documentation 5.0

Release Information

Introduction

Getting Started

Delphix Engine System Administration

Virtual Database Management with the Delphix Engine

Data Backup and Recovery

Delphix Modernization Engine

Delphix Masking

Virtualizing Unstructured Files in the Delphix Engine

Virtualizing Oracle E-Business Suite in the Delphix Engine

JetStream

Mission Control

Delphix Express User Guide

Reference

Documentation for Previous Versions of the Delphix Engine

Delphix Server 3.0 Documentation
Delphix Server 3.1 Documentation
Delphix Engine 3.2 Documentation
Delphix Engine 4.0 Documentation
Delphix Engine 4.1 Documentation
Delphix Engine 4.2 Documentation
Delphix Engine 4.3 Documentation

Skip to end of metadata
Go to start of metadata

Delphix creates "jobs" to profile, provision, and mask, and certify data.

Jobs on the Environment Overview Screen

The Environment Overview screen provides status of any jobs running or previously run within the environment. When you create jobs for an environment, you must create them from the Environment Overview screen of the environment they mask or the environment they use as a source for data provisioning.

The following columns appear in the Jobs section of the screen:

  • Name
  • Rule Set
  • Status
  • Action
  • Edit
  • Delete

Creating New Jobs

In the Environment Overview screen, select one of the jobs icons to create the corresponding job:

  • Profile
  • Mask
  • Certify
  • Provision

Figure 16 Environment Overview Screen Showing Create Job Buttons

Creating a New Profiling Job

You can create profiling jobs for databases, copybooks, delimited files, fixed-width, and Excel files.

A Profiling job for a mainframe system cannot assign groups because it does not have the heuristics needed to determine sensitive elements per group. The Profiler assigns group-sensitive elements to a single group. Then, in inventory, groups are updated as needed to establish the sensitive element field group sets.

To create a new profiling job:

  1. Click Profile.
    The Create Profiling Job window appears.
    Figure 17 Create Profile Job


    You will be prompted for the following information:
    1. Job Name — A free-form name for the job you are creating. Must be unique.
    2. Multi Tenant — Check box if the job is for a multi-tenant database.
    3. Rule Set — Select a rule set that this job will execute against.
    4. Generator — The default value is Delphix.
    5. No. of Streams — The number of parallel streams to use when running the jobs. For example, you can select two streams to run two tables in the ruleset concurrently in the job instead of one table at a time.
    6. Remote Server — (optional) The remote server that will execute the jobs. This option lets you choose to execute jobs on a remote server, rather than on the local Delphix instance. Note: This is an add-on feature for Delphix Standard Edition.
    7. Min Memory (MB) — (optional) Minimum amount of memory to allocate for the job, in megabytes.
    8. Max Memory (MB) — (optional) Maximum amount of memory to allocate for the job, in megabytes.
    9. Feedback Size — (optional) The number of rows to process before writing a message to the logs. Set this parameter to the appropriate level of detail required for monitoring your job. For example, if you set this number significantly higher than the actual number of rows in a job, the progress for that job will only show 0 or 100%.
    10. Profile Sets — (Optional) The name of a profiler set, which is a subset of expressions (for example, a subset of financial expressions). (See Delphix Administrator's Guide.)
      Note: If you do not select a profile set, Delphix will use all defined expressions instead of just a subset.
    11. Comments — (optional) Add comments related to this job.
    12. Email — (optional) Add e-mail address(es) to which to send status messages. Separate addresses with a comma (,).
  2. When you are finished, click Save.

For information about running jobs, see Running and Stopping Jobs from the Environment Overview Screen. 

Creating a New Masking Job

To create a new masking job:

  1. Click Mask.
    The Create Masking Job window appears.  
    Figure 18 Create Masking Job

    You will be prompted for the following information:
    1. Job Name — A free-form name for the job you are creating. Must be unique across the entire application.
    2. Masking Method — Select either In-Place or On-The-Fly. For more information on masking type, see  Mask Data.
    3. Multi Tenant — Check box if the job is for a multi-tenant database.
    4. Rule Set — Select a rule set that this job will execute against.
      or:
    5. Generator — The default value is Delphix.
    6. Repository Folder name — The folder name in the repository where the objects should be imported.
    7. Parameter File Path — (optional) If checked, this tells Delphix to configure the sessions and workflows to use a parameter file that contains the source and target connection information. If unchecked, the Delphix Engine will generate sessions/workflows that use the connector names as defined within the Delphix Engine, which will require connections with the same names defined within the repository.
    8. Import Mapplet — (optional) if checked, this tells the Delphix Engine to import mapplets that are assigned to columns in the inventory along with the mappings/sessions/workflows. If unchecked, Delphix will not attempt to import any mapplets that are assigned in the inventory.
    9. Mask Method — Choose either of the following:No. of Streams—The number of parallel streams to use when running the jobs. For example, you can select two streams to run two tables in the Rule Set concurrently in the job instead of one table at a time.
      1. Import — When you click the Run icon, creates the mappings but does not execute the workflow. You later run the job.
      2. Import and Run — When you click the Run icon, creates the mappings and executes the workflow.
    10. Remote Server — (optional) The remote server that will execute the jobs. This option lets you choose to execute jobs on a remote server, rather than on the local Delphix instance. Note: This is an optional feature for Delphix.
    11. Min Memory (MB) — (optional) Minimum amount of memory to allocate for the job, in megabytes.
    12. Max Memory (MB) — (optional) Maximum amount of memory to allocate for the job, in megabytes.
    13. Update Threads — The number of update threads to run in parallel to update the target database.
    14. For databases using T-SQL, multiple update/insert threads can cause deadlocks. If you see this type of error, reduce the number of threads that you specify in this box.
    15. Commit Size — (optional) The number of rows to process before issuing a commit to the database.
    16. Feedback Size — (optional) The number of rows to process before writing a message to the logs. Set this parameter to the appropriate level of detail required for monitoring your job. For example, if you set this number significantly higher than the actual number of rows in a job, the progress for that job will only show 0 or 100%.
    17. Bulk Data — (optional) For In-Place masking only. The default is for this check box to be clear. If you are masking very large tables in-place and require performance improvements, check this box. Delphix will mask data to a flat file, and then use inserts instead of updates to bulk load the target table.
    18. Disable Constraint — (optional) Whether to automatically disable database constraints. The default is for this check box to be clear and therefore not perform automatic disabling of constraints. For more information about database constraints, see  Enabling and Disabling Database Constraints.
    19. Batch Update — (optional) Enable or disable use of a batch for updates. A job's statements can either be executed individually, or can be put in a batch file and executed at once, which is faster.
    20. Disable Trigger — (optional) Whether to automatically disable database triggers. The default is for this check box to be clear and therefore not perform automatic disabling of triggers.
    21. Drop Index — (optional) Whether to automatically drop indexes on columns which are being masked and automatically re-create the index when the masking job is completed. The default is for this check box to be clear and therefore not perform automatic dropping of indexes.
    22. Prescript — (optional) Specify the full pathname of a file that contains SQL statements to be run before the job starts, or click Browse to specify a file. If you are editing the job and a prescript file is already specified, you can click the Delete button to remove the file. (The Delete button only appears if a prescript file was already specified.) For information about creating your own prescript files, see  Creating SQL Statements to Run Before and After Jobs.
    23. Postscript — (optional) Specify the full pathname of a file that contains SQL statements to be run after the job finishes, or click Browse to specify a file. If you are editing the job and a postscript file is already specified, you can click the Delete button to remove the file. (The Delete button only appears if a postscript file was already specified.) For information about creating your own postscript files, see  Creating SQL Statements to Run Before and After Jobs.
    24. Comments — (optional) Add comments related to this masking job.
    25. Email — (optional) Add e-mail address(es) to which to send status messages.
  2. When you are finished, click Save.

For information about running jobs, see Running and Stopping Jobs from the Environment Overview Screen.

Creating a New Certify Job

To create a new certify job:

  1. Click Certify.
    The Create Certify Job window appears.  
    Figure 19 Create Certify Job

    You will be prompted for the following information:
    1. Job Name — A free-form name for the job you are creating. Must be unique.
    2. Multi Tenant — Check box if the job is for a multi-tenant database.
    3. Rule Set — Select a rule set that this job will execute against.
    4. Generator — The default value is Delphix.
    5. No. of Streams — The number of parallel streams to use when running the jobs. For example, you can select two streams to run two tables in the ruleset concurrently in the job instead of one table at a time.
    6. Remote Server — (optional) The remote server that will execute the jobs. This option lets you choose to execute jobs on a remote server, rather than on the local Delphix instance. Note: This is an add-on feature for Delphix Standard Edition.
    7. Min Memory (MB) — (optional) Minimum amount of memory to allocate for the job, in megabytes.
    8. Max Memory (MB) — (optional) Maximum amount of memory to allocate for the job, in megabytes.
    9. Feedback Size — (optional) The number of rows to process before writing a message to the logs. Set this parameter to the appropriate level of detail required for monitoring your job. For example, if you set this number significantly higher than the actual number of rows in a job, the progress for that job will only show 0 or 100%.
    10. Prescript — (optional) Specify the full pathname of a file that contains SQL statements to be run before the job starts, or click Browse to specify a file. If you are editing the job and a prescript file is already specified, you can click the Delete button to remove the file. (The Delete button only appears if a prescript file was already specified.) For information about creating your own prescript files, see  Creating SQL Statements to Run Before and After Jobs 
    11. Postscript — (optional) Specify the full pathname of a file that contains SQL statements to be run after the job runs, or click Browse to specify a file. If you are editing the job and a postscript file is already specified, you can click the Delete button to remove the file. (The Delete button only appears if a postscript file was already specified.) For information about creating your own postscript files, see  Creating SQL Statements to Run Before and After Jobs.
    12. Delta Masking — Check this box to mask rows that are not masked and are identified as part of the certification process as polluted.
    13. Batch Update — (optional) Enable or disable use of a batch for updates. A job's statements can either be executed individually, or can be put in a batch file and executed at once, which is faster.
    14. Disable Trigger — (optional) Whether to automatically disable database triggers. The default is for this check box to be clear and therefore not perform automatic disabling of triggers.
    15. Comments — (optional) Add comments related to this certification job.
    16. Email — (optional) Add e-mail address(es) to which to send status messages.
  2. When you are finished, click Save.

For information about running a job, see Running and Stopping Jobs from the Environment Overview Screen.

Creating a New Provisioning Job

When you provision data, you define the provisioning job within the source environment (from source to target). On the other hand, a masking job is defined from the target environment.

To create a new provisioning job:

  1. Click Provision.
    The Create Provisioning Job window appears.  
    Figure 20 Create Provisioning Job

    You will be prompted for the following information:
    1. Job Name — A free-form name for the job you are creating.
    2. Target Environment — The environment into which the data will be loaded.
    3. Multi Tenant — Check box if the job is for a multi-tenant database.
    4. Rule Set — Select a rule set that this job will execute against.
    5. Target Connector — The database connector into which the data will be loaded.
    6. Generator
    7. Repository Folder name — The folder name in the repository where the objects should be imported.
    8. No. of Streams — The number of parallel streams to use when running the jobs. For example, you can select two streams to run two tables in the ruleset concurrently in the job instead of one table at a time.
    9. Remote Server — (optional) The remote server that will execute the jobs. This option lets you choose to execute jobs on a remote server, rather than on the local Delphix instance. Note: This is an add-on feature for Delphix Standard Edition.
    10. Min Memory (MB) — (optional) Minimum amount of memory to allocate for the job, in megabytes.
    11. Max memory (MB) — (optional) Maximum amount of memory to allocate for the job, in megabytes.
    12. Commit Size — (optional) The number of rows to process before issuing a commit to the database.
    13. Feedback Size — (optional) The number of rows to process before writing a message to the logs. Set this parameter to the appropriate level of detail required for monitoring your job. For example, if you set this number significantly higher than the actual number of rows in a job, the progress for that job will only show 0 or 100%.
    14. Disable Constraint — (optional) Whether to automatically disable database constraints. The default is for this check box to be clear and therefore not perform automatic disabling of constraints. For more information about database constraints, see Enabling and Disabling Database Constraints.
    15. Batch Update — (optional) Enable or disable use of a batch for updates. A job's statements can either be executed individually, or can be put in a batch file and executed at once, which is faster.
    16. Truncate — (optional) Whether to truncate target tables before loading them with data. If this box is selected, the tables will be "cleared" before the operation. If this box is clear, data is appended to tables, which potentially can cause primary key violations. This box is clear by default.
    17. Disable Trigger — (optional) Whether to automatically disable database triggers. The default is for this check box to be clear and therefore not perform automatic disabling of triggers.
    18. Prescript — (optional) Specify the full pathname of a file containing SQL statements to be run before the job starts, or click Browse to specify a file. If you are editing the job and a prescript file is already specified, you can click the Delete button to remove the file. (The Delete button only appears if a prescript file was already specified.) For information about creating your own prescript files, see Creating SQL Statements to Run Before and After Jobs.
    19. Postscript — (optional) Specify the full pathname of a file containing SQL statements to be run after the job finishes, or click Browse to specify a file. If you are editing the job and a postscript file is already specified, you can click the Delete button to remove the file. (The Delete button only appears if a postscript file was already specified.) For information about creating your own postscript files, see Creating SQL Statements to Run Before and After Jobs.
    20. Comments — (optional) Add comments related to this provisioning job.
    21. Email — (optional) Add e-mail address(es) to which to send status messages.
  2. When you are finished, click Save.

Running and Stopping Jobs from the Environment Overview Screen

To run or rerun a job from the Environment Overview screen:

  1. Click the Run icon (play icon) in the Action column for the desired job.

The Run icon changes to a Stop icon while the job is running.

When the job is complete, the Status changes.


To stop a running job from the Environment Overview screen:

  1. Click the Stop icon in the Action column for the running job.
    A popup appears asking Are you sure you want to stop job?
  2. Click OK.

When complete, the Status changes.