The Masking Engine creates "jobs" to profile and mask data.

Jobs on the Environment Overview Screen

The Environment Overview screen provides status of any jobs running or previously run within the environment. When you create jobs for an environment, you must create them from the Environment Overview screen of the environment they mask or the environment they use as a source for data provisioning.

The following columns appear in the Jobs section of the screen:

  • Name
  • Rule Set
  • Status
  • Action
  • Edit
  • Delete

Creating New Jobs

In the Environment Overview screen, select one of the jobs icons to create the corresponding job:

  • Profile
  • Mask

Environment Overview Screen showing Create Job buttons

Creating a New Profiling Job

You can create profiling jobs for databases, copybooks, delimited files, fixed-width, and Excel files.

A Profiling job for a mainframe system cannot assign groups because it does not have the heuristics needed to determine sensitive elements per group. The Profiler assigns group-sensitive elements to a single group. Then, in inventory, groups are updated as needed to establish the sensitive element field groupsets.

To create a new profiling job:

  1. Click Profile.
    The Create Profiling Job window appears.

    Create Profile Job

    You will be prompted for the following information:

    1. Job Name — A free-form name for the job you are creating. Must be unique.
    2. Multi Tenant — Check the box if the job is for a multi-tenant database. This option allows existing rulesets to be re-used to mask identical schemas via different connectors. The connector can be selected at job execution time.
    3. Rule Set — Select a rule set that this job will execute against.
    4. Generator — The default value is Delphix.
    5. No. of Streams — The number of parallel streams to use when running the jobs. For example, you can select two streams to run two tables in the ruleset concurrently in the job instead of one table at a time.
    6. Remote Server — (optional) The remote server that will execute the jobs. This option lets you choose to execute jobs on a remote server, rather than on the local Delphix instance. Note: This is an add-on feature for Delphix Standard Edition.
    7. Min Memory (MB) — (optional) Minimum amount of memory to allocate for the job, in megabytes.
    8. Max Memory (MB) — (optional) Maximum amount of memory to allocate for the job, in megabytes.
    9. Feedback Size — (optional) The number of rows to process before writing a message to the logs. Set this parameter to the appropriate level of detail required for monitoring your job. For example, if you set this number significantly higher than the actual number of rows in a job, the progress for that job will only show 0 or 100%.
    10. Multiple PHI  — (optional) When multiple profiler expressions match then

      • If the Multiple PHI checkbox is checked the algorithm assignment is deterministic. The algorithm is specified in an application property file.

      • If the Multiple PHI checkbox is unchecked the profiler expressions are tested in an arbitrary order. The first matching expression wins and its algorithm is assigned. Note: Users do not have a way to order profiler expressions.

    11. Profile Sets — (Optional) The name of a profiler set, which is a subset of expressions (for example, a subset of financial expressions). (See Delphix Administrator's Guide.)
      Note: If you do not select a profile set, Delphix will use all defined expressions instead of just a subset.
    12. Comments — (optional) Add comments related to this job.
    13. Email — (optional) Add e-mail address(es) to which to send status messages. Separate addresses with a comma (,).
  2. When you are finished, click Save.
For information about running jobs, see Running and Stopping Jobs from the Environment Overview Screen. 

Creating a New Masking Job

To create a new masking job:

  1. Click Mask.
    The Create Masking Job window appears.
     

    Create masking job


    You will be prompted for the following information:

    1. Job Name — A free-form name for the job you are creating. Must be unique across the entire application.
    2. Masking Method — Select either In-Place or On-The-Fly. For more information on masking type, see Mask Data.
    3. Multi Tenant — Check box if the job is for a multi-tenant database.

      Provision Masked VDBs

      A job must be Multi Tenant to use it when creating a masked virtual database (VDB).

    4. Rule Set — Select a rule set that this job will execute against.
      or:
    5. Generator — The default value is Delphix.
    6. Repository Folder name — The folder name in the repository where the objects should be imported.
    7. Parameter File Path — (optional) If checked, this tells Delphix to configure the sessions and workflows to use a parameter file that contains the source and target connection information. If unchecked, the Delphix Engine will generate sessions/workflows that use the connector names as defined within the Delphix Engine, which will require connections with the same names defined within the repository.
    8. Import Mapplet — (optional) if checked, this tells the Delphix Engine to import mapplets that are assigned to columns in the inventory along with the mappings/sessions/workflows. If unchecked, Delphix will not attempt to import any mapplets that are assigned in the inventory.
    9. Mask Method — Choose either of the following:
      1. No. of Streams—The number of parallel streams to use when running the jobs. For example, you can select two streams to run two tables in the Rule Set concurrently in the job instead of one table at a time.
      2. Import — When you click the Run icon, creates the mappings but does not execute the workflow. You later run the job.
      3. Import and Run — When you click the Run icon, creates the mappings and executes the workflow.
    10. Remote Server — (optional) The remote server that will execute the jobs. This option lets you choose to execute jobs on a remote server, rather than on the local Delphix instance. Note: This is an optional feature for Delphix.
    11. Min Memory (MB) — (optional) Minimum amount of memory to allocate for the job, in megabytes.
    12. Max Memory (MB) — (optional) Maximum amount of memory to allocate for the job, in megabytes.
    13. Update Threads — The number of update threads to run in parallel to update the target database.

      Multiple threads should not be used if the masking job contains any table without an index. Multi-threaded masking jobs can lead to deadlocks on the database engine.

      Multiple threads can cause database engine deadlocks for databases using T-SQL If masking jobs fail and a deadlock error exists on the database engine, then reduce the number of threads.

    14. Commit Size — (optional) The number of rows to process before issuing a commit to the database.
    15. Feedback Size — (optional) The number of rows to process before writing a message to the logs. Set this parameter to the appropriate level of detail required for monitoring your job. For example, if you set this number significantly higher than the actual number of rows in a job, the progress for that job will only show 0 or 100%.
    16. Bulk Data — (optional) For In-Place masking only. The default is for this check box to be clear. If you are masking very large tables in-place and require performance improvements, check this box. Delphix will mask data to a flat file, and then use inserts instead of updates to bulk load the target table.
    17. Disable Constraint — (optional) Whether to automatically disable database constraints. The default is for this check box to be clear and therefore not perform automatic disabling of constraints. For more information about database constraints, see Enabling and Disabling Database Constraints.
    18. Batch Update — (optional) Enable or disable use of a batch for updates. A job's statements can either be executed individually, or can be put in a batch file and executed at once, which is faster.
    19. Disable Trigger — (optional) Whether to automatically disable database triggers. The default is for this check box to be clear and therefore not perform automatic disabling of triggers.
    20. Drop Index — (optional) Whether to automatically drop indexes on columns which are being masked and automatically re-create the index when the masking job is completed. The default is for this check box to be clear and therefore not perform automatic dropping of indexes.
    21. Prescript — (optional) Specify the full pathname of a file that contains SQL statements to be run before the job starts, or click Browse to specify a file. If you are editing the job and a prescript file is already specified, you can click the Delete button to remove the file. (The Delete button only appears if a prescript file was already specified.) For information about creating your own prescript files, see Creating SQL Statements to Run Before and After Jobs.
    22. Postscript — (optional) Specify the full pathname of a file that contains SQL statements to be run after the job finishes, or click Browse to specify a file. If you are editing the job and a postscript file is already specified, you can click the Delete button to remove the file. (The Delete button only appears if a postscript file was already specified.) For information about creating your own postscript files, see Creating SQL Statements to Run Before and After Jobs.
    23. Comments — (optional) Add comments related to this masking job.
    24. Email — (optional) Add e-mail address(es) to which to send status messages.
  2. When you are finished, click Save.

Masking Jobs list

For information about running jobs, see Running and Stopping Jobs from the Environment Overview Screen.

Running and Stopping Jobs from the Environment Overview Screen

To run or rerun a job from the Environment Overview screen:
  1. Click the Run icon (play icon) in the Action column for the desired job.

The Run icon changes to a Stop icon while the job is running.

When the job is complete, the Status changes.

To stop a running job from the Environment Overview screen:
  1. Locate the job you want to stop.
  2. In the job's Action column, click the Stop icon.
  3. A popup appears asking, "Are you sure you want to stop job?" Click OK.
    When the job has been stopped, its status changes.
  4. After the job completes successfully, return to the Inventory and check that the Domain and Method populated automatically for sensitive data. Sample screenshot below.

Profile job completion

Related Links