Release Information

Introduction

Getting Started

Delphix Engine System Administration

Virtual Database Management with the Delphix Engine

Data Backup and Recovery

Delphix Modernization Engine

Delphix Masking

Virtualizing Unstructured Files in the Delphix Engine

Virtualizing Oracle E-Business Suite in the Delphix Engine

JetStream

Mission Control

Delphix Express User Guide

Reference


Documentation for Previous Versions of the Delphix Engine

Delphix Server 3.0 Documentation
Delphix Server 3.1 Documentation
Delphix Engine 3.2 Documentation
Delphix Engine 4.0 Documentation
Delphix Engine 4.1 Documentation
Delphix Engine 4.2 Documentation
Delphix Engine 4.3 Documentation

Skip to end of metadata
Go to start of metadata

Delphix will mask a number of different file types and formats. These include fixed, delimited, Excel, Mainframe/VSAM, XML, Word, and Powerpoint. The purpose of this document is to provide an overview of general guidelines on how to successfully mask files using Delphix. This document will not replace Delphix training or the Delphix manual set, it is in addition to these items.

Overview

Delphix supports 2 masking methodologies, In-Place and On-The-Fly. In-Place requires a single file connection and Delphix will read from that file, mask data in memory, and update the file with the masked data. On-The-Fly requires 2 file connections. One connection for the source file, and one connection to the target where the masked file will be placed. The target file name must exist. In this scenario, Delphix will read the file from the source connection, mask in memory, and write the masked data to the target file.

File Formats

Unlike databases files for the most part do not have built in metadata to describe the format of the fields in the file. You must provide this to Delphix so it can update the file appropriately. This is done through the settings tab where you will see a menu item on the left for File Format. Select File Format and you will see options to create a file format or input a file format. This will depend on the type of file and how you want to let Delphix know the format of the file.

Mainframe and XML Files

For Mainframe/VSAM files, you can input the file format via input format option which will import the copybook directly into Delphix. You can input this file from SFTP, FTP, Local, or Hadoop-HDFS

For XML files you can also input the file format with the input format option. You can use the file you want to mask as the format. Delphix will input the format of the file directly. You can input this file from SFTP, FTP, or Local.

Delimited, Excel, Fixed Files

For Delimited, Excel, and Fixed files you can either manually create the format of the file yourself, or you can input a text file which describes the structure of the file to Delphix. To input the file format for delimited or Excel files create a text document with the column names each on its own line. For example:

  • Name
  • Address
  • City
  • State

To input the file format for fixed files create a text document with the column names and the length of each column on its own line. For example:

  • Name,25
  • Address,40
  • City,20
  • State,2

Then input this file as the file format. The name of the text file will be the name of the file format. The file will need to be local to the Delphix server.
To create a format manually, you can just click the create format button and give the format a name. We will input the details of the format a little later in this document.

Create a File Connector

In your environment you will need to create a connector to access the files. Each file type will require a separate connector. When you create the connector, scroll down to the particular file type you desire and select it. After you select this you will need to select the connection mode. For Excel, Word, Powerpoint, and XML files there is only one connector mode and that is local. This can be either a directory on the Delphix server or a share that is accessible as a local directory on the Delphix server. For Mainframe/VSAM, Delimited, or Fixed files the connector mode can be:

  • Local
  • SFTP
  • FTP
  • Hadoop – HDFS.

These connection modes (other than local) will require additional information. We provide a test connection button to test the validity of the connection. If you are doing in-place masking the file(s) will be masked and updated in the directory pointed to by the connector. If you plan to do on-the-fly masking then you will need to create a separate environment and connector to be the source for the files to be masked. The masked files will get put into the directory pointed to by the connector you created previously (the target). However, the file name must exist in the target directory. It does not have to be a copy of the file, just an entry in the directory with the same name. It will be replaced by the masked file.

Create a Ruleset

Once you create a connector, you can click on the ruleset tab and create a ruleset. Click on create ruleset, give it a name, and provide the file connector you previously created. Once you do this you will see a list of files that the connector points to. You can select a single file, multiple files, or all the files. Once you save this the ruleset with the file or files will be saved.

Once you create a ruleset with a file or set of files, you will need to assign those files to their appropriate file format. This is accomplished by editing the ruleset. When you click on the edit button for the file a popup screen called edit file will appear with the file name. There will be a dropdown for the format so you can select the proper format for the file. Select the end-of-record to let Delphix know whether the file is in windows/dos format (CR+LF) or Linux format (LF). If the file is a delimited file you will have a space to put in the delimiter. If the file is a Mainframe/VSAM file with a copybook you will see a checkbox to signify if the file is variable length. If there are multiple files in the ruleset you will have to edit each one individually and assign it to the appropriate file format.

File Inventory

For XML or Copybook files, once you select the ruleset and the file you will see the inventory for the file and you can edit this inventory with the appropriate masking settings like any Delphix data source by either using the profiler or setting this manually.
For Excel, Delimited, or Fixed files, if you created the file format by importing it then the format for the file will be set. When you go to the inventory page and select the ruleset and file you will see a line which shows all the records which you can expand to see the inventory. If your file has a header and/or footer you will need to click on Record Type, click on add record type and select Header and/or Footer from the dropdown box. Then enter a name for this and the number of rows/lines. Now you can assign the appropriate masking algorithms either by running the profiler or setting them manually on the all records section.

If you did not import a format and just created a file format with the create format button you will have to enter the actual layout of the file into Delphix. This can be done for Excel, Delimited and Fixed files. This can be accomplished by:

  • Navigating to the inventory screen and selecting the appropriate ruleset and file
  • Click on the Record Type button and add the appropriate record types
  • You will have to add a body record type. Select body from the dropdown and give it a name (i.e. Body).
  • If you have a file with the columns defined you can import this using the import button. If not then just save.
  • If your file has a header or footer you can add those next.
  • If you did not import the format you will have to enter this in manually. To do this click on the Define Fields button, when the screen pops up you enter in the field name, choose the record type (body) and position in the record.
  • If the file is a fixed length file you will also have to enter in the length of the field.
  • You can optionally set the masking here also as you enter this in, or you can do this with the profiler. Enter in all the fields and you will be set.