API Prerequisite Knowledge

JSON

JSON (JavaScript Object Notation) is a minimal, readable format for structuring data. It is a simple format for transmitting data between applications, as an alternative to XML. The Delphix API uses JSON data structure in the format of strings to send and receive data from the API calls, as you will see later in the examples. First, let's look at the JSON fundamentals.

Keys and Values

The two primary parts that makeup JSON are keys and values. Together they make key/value pairs, also called name/value pairs.

• Key – Always a string enclosed in quotation marks.
• Value – Can be a string, number, boolean expression, array, or object. 
• Key/Value Pair – Follows a specific syntax, with the key followed by a colon followed by the value. Key/value pairs are comma separated.

Let's take a JSON sample string and identify each part of the code.

{
    "foo" : "bar",
    "rows" : 100
} 

The curly brackets start and end the string. The key is "foo" and the value is "bar". A colon ( : ) is the delimiter between them. A comma ( , ) is the delimiter for multiple key/value pairs. The second pair is "rows" and the value is a number of 100.

Types of Values

Numbers, Booleans, and Strings.

It is very important to understand the APIs JSON object definitions. Quoted values are treated as strings!

"x" : "1" is treated as a string, while

"x" : 1 is treated as a number

"y" : "true" is treated as a string, while

"y" : true is treated as a boolean true (false)

Null values

{
    "z" : 
  , "b" : "World"
}

Nulls are empty values, but sometimes programmers code "" as a null value.

{
    "z" : ""
  , "b" : "World"
  } 

So always verify how the null values are defined and handled by the application.

Arrays

An array is indicated with the square brackets: [ value1, value2, etc. ]. In this example, we have added a categories key with an array of values.

...
"foo" : {
  "bar" : "Hello",
  "category" : [ "greetings", "morals" ]
}
... 

Objects

An object is indicated by curly brackets: {"key", "value"}. Everything inside of the curly brackets is part of the object. We already learned that a value could be an object. Therefore, "foo" and the corresponding object are a key/value pair.

...
"foo" : { "bar" : "Hello" }
... 

The key/value pair "bar" : "Hello" is nested inside the key/value pair "foo" : { ... }. That is an example of a hierarchy (or nested data) within JSON data.

Arrays and Objects can be nested or contained within the same level.

Summary

JSON arrays are [ , , ]

JSON nested objects are , , "x":{ "a":"1", "b":"2" }, ,

JSON data can be passed within the HTTP URL (file or argument), the header, or other handlers.

From within Shell Scripts or Programming Languages, JSON data is typically processed through a "JSON parser." This topic is covered later.

Delphix CLI

Connecting to the Delphix Engine CLI

Reference: Connecting to the CLI

There are two user roles accessible, the sysadmin and the delphix_admin.

From a shell environment, you can connect using the ssh command. The IP Address (or Hostname) represents the Delphix Engine (case sensitive):

ssh sysadmin@127.16.160.195

ssh delphix_admin@127.16.160.195

From a putty session, open an ssh connection to the Delphix Engine IP Address or Hostname (case sensitive): 

open 127.16.160.195

Login User: sysadmin@SYSTEM

#... or ...

Login User: delphix_admin@DOMAIN

After entering the correct password for the respective user, the menus for that user's role will be different. For example, the sysadmin@SYSTEM user has engine storage, whereas the delphix_admin@DOMAIN user has database provisioning.

You can use the CLI for scripting and configure the connection for ssh passwordless connections.

CLI Cookbook: Configuring Key-Based SSH Authentication for Automation

How to use the CLI to learn the APIs

As stated earlier, a great way to learn how to generate the Delphix RESTFul API calls and the required JSON content is to use the Delphix CLI (Command Line Interface) and turn on the CLI> setopt trace=true option.

Below is an example of how to get the JSON required parameters for a database refresh per the type of refresh performed.

The refresh database example below shows how to use the CLI to identify reference objects for other CLI commands and the respective RESTFul API structure when the setopt trace=true option is set. 

$ ssh delphix_admin@172.16.160.195 
Password: 
Delphix5030HWv8> ls 
Children
about
action
...
connectivity
database
environment
... 
toolkit
user 

Operations
version
Delphix5030HWv8> database 
Delphix5030HWv8 database> ls 
Objects
NAME			PROVISIONCONTAINER	DESCRIPTION
DPXDEV01		- 
Vdelphix_demo	delphix_demo		-
delphix_demo	- 
Scripts 		- 
V_2C1			Scripts				-
Vvfiles			-					- 

Children
template 

Operations
createEmpty
createRestorationDataset
export
fileMapping
link
oracleSupportedCharacterSets
provision
validateXpp
xpp 

First, we need to identify the target Delphix virtualized database object to refresh ...

Delphix5030HWv8 database> select Vdelphix_demo 
Delphix5030HWv8 database 'Vdelphix_demo'> ls 
Properties
	type: MSSqlDatabaseContainer
	name: Vdelphix_demo
	creationTime: 2016-06-16T14:30:03.033Z
	currentTimeflow: 'DB_PROVISION@2016-06-16T10:30:08'
	delphixManaged: true
	description: (unset)
	group: Windows
	masked: false
	os: Windows
	performanceMode: DISABLED
	processor: x86
	provisionContainer: delphix_demo
	reference: MSSQL_DB_CONTAINER-39 
	restoration: false
	runtime:
		type: MSSqlDBContainerRuntime
		logSyncActive: false
	sourcingPolicy:
		type: SourcingPolicy
		loadFromBackup: false
		logsyncEnabled: false
	transformation: false 

Operations
delete
...
purgeLogs
refresh
removeLiveSource
...
Delphix5030HWv8 database 'Vdelphix_demo'> refresh 
Delphix5030HWv8 database 'Vdelphix_demo' refresh *> ls 
Properties
	type: RefreshParameters
	timeflowPointParameters:
		type: TimeflowPointSemantic
		container: (required)
		location: LATEST_POINT
Delphix5030HWv8 database 'Vdelphix_demo' refresh *> set timeflowPointParameters.container=delphix_demo 
Delphix5030HWv8 database 'Vdelphix_demo' refresh *> ls 
Properties
	type: RefreshParameters
	timeflowPointParameters:
		type: TimeflowPointSemantic
		container: delphix_demo (*) 
		location: LATEST_POINT
Delphix5030HWv8 database 'Vdelphix_demo' refresh > *commit 
	Dispatched job JOB-100
	DB_REFRESH job started for "Windows/Vdelphix_demo".
	Validating that this dataset is managed by Delphix.
	Stopping virtual database.
	Unmounting datasets.
	Unexporting storage containers.
	Metadata for dSource "Vdelphix_demo" successfully deleted.
	Starting provisioning of virtual database "Vdelphix_demo".
	Creating new TimeFlow.
	Generating recovery scripts.
	Mounting datasets.
	Mounting read-only source logs dataset.
	Running user-specified pre-provisioning script.
	Recovering virtual database.
	The virtual database recovery was successful.
	Unmounting read-only source logs dataset.
	Running user-specified post-provisioning script.
	The virtual database "Vdelphix_demo" was successfully provisioned.
	DB_REFRESH job for "Windows/Vdelphix_demo" completed successfully. 

Refresh again but this time turn on the setopt trace=true option.

Delphix5030HWv8 database 'Vdelphix_demo'> refresh
Delphix5030HWv8 database 'Vdelphix_demo' refresh *> ls
Properties
	type: RefreshParameters
	timeflowPointParameters:
		type: TimeflowPointSemantic
		container: (required)
		location: LATEST_POINT
Delphix5030HWv8 database 'Vdelphix_demo' refresh *> set timeflowPointParameters.container=delphix_demo
Delphix5030HWv8 database 'Vdelphix_demo' refresh *> ls
Properties
	type: RefreshParameters
	timeflowPointParameters:
		type: TimeflowPointSemantic
		container: delphix_demo
		location: LATEST_POINT
Delphix5030HWv8 database 'Vdelphix_demo' refresh *> setopt trace=true
Delphix5030HWv8 database 'Vdelphix_demo' refresh *> commit
=== POST /resources/json/delphix/database/MSSQL_DB_CONTAINER-39/refresh ===
{
	"type": "RefreshParameters",
	"timeflowPointParameters": {
		"type": "TimeflowPointSemantic",
		"container": "MSSQL_DB_CONTAINER-38"
	}
}
...

Using the CLI, you can identify the RESTFul API POST and GET commands along with the JSON input data requirements.

=== POST /resources/json/delphix/database/MSSQL_DB_CONTAINER-39/refresh ===
{ 
	"type": "RefreshParameters", 
	"timeflowPointParameters": { 
		"type": "TimeflowPointSemantic", 
		"container": "MSSQL_DB_CONTAINER-38" 
	} 
} 

So framing the RESTFul URL for a virtual database refresh, the URL will look like

http://<delphix_engine>/resources/json/delphix/database/ MSSQL_DB_CONTAINER-39 /refresh

where the MSSQL_DB_CONTAINER-39 represents the target virtualized database to refresh. We need to POST the JSON data to the URL for processing.
 

{
	"type": "RefreshParameters", 
	"timeflowPointParameters": {  
		"type": "TimeflowPointSemantic", 
		"container": "MSSQL_DB_CONTAINER-38" 
	}
}

The "timeflowPointParameters" key has 6 "type": "..." options, each of which has its own set of parameters. The type "TimeflowPointSemantic" uses the default LATEST_POINT within the source container, so for simplicity, we will use this type. For more information on timeflowPointParameters 6 types, see the Advanced Section.

If this is a little confusing at this point, do not worry, that's typical. Complete examples will be shown later. The important items to remember are:

• Delphix often uses object reference names within the JSON data.
• Using the setopt trace=true the option provides the construct for the RESTFul API URLs and the JSON data for POST / GET operations.

HTTP

We use the HTTP protocol every day for web browsing and commercial business. From finding a new restaurant to buying a 1986 Ford Thunderbird Turbo Coupe!

Most people see the HTTP within the URL Address field within the Web Browser window – for example, http://www.google.com

But behind the scenes, HTTP is performing a wide range of functionality. For RESTFul APIs, they use HTTP's GET and POST form functionality to process data. In Delphix's case, the data is also represented as JSON structures.

HTTP GET operation is used to return data only, while HTTP POST operation is used to provide data input in the form of a structured JSON data string or file.

cURL

What is cURL?

The cURL client command is based on a library supporting a number of web protocols, including HTTP. The "curl" command can be called from the command line, while the cURL library is commonly integrated with your favorite programming languages, such as Java, JSP, Python, Perl, PHP, .NET, and PowerShell.

Due to its widespread adoption, we will use cURL for making the Delphix RESTFul API calls within this document. Some operating systems or languages support their own HTTP commands / related libraries, and you can use these instead of cURL. One alternative is the "wget" command described later.

Is cURL installed?

Operating System Prompt>	curl --version
curl 7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
Protocols: tftp ftp telnet dict ldap ldaps http file https ftps scp sftp 
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz 

Get the HTTP output from google.com

Operating System Prompt> curl www.google.com

Wget

An alternative to cURL is Wget, which is typically a native command on all Linux environments. See the Appendix for a complete comparison between Wget and cURL.

dxtoolkit2

Delphix has developed a very robust toolkit, dxtoolkit2, which utilizes the Delphix RESTFul APIs. This toolkit is cross-platform. Its commands are built with the Perl programming language.

We recommend that you review the dxtoolkit2 documentation; you may find a utility that already performs your desired function. For example, the utility dx_get_analytics is absolutely great for dumping analytic data from the Delphix Engine into a .csv (comma-separated value) format, which you can then easily integrate into your enterprise monitoring tools. See the sample "Analytics" use case.

Contact Delphix personnel for the latest download.