Requirements for PostgreSQL Hosts and Databases
Datasets / PostgreSQL Environments and Data Sources / PostgreSQL Support and Requirements / Requirements for PostgreSQL Hosts and Databases

In order to begin using PostgreSQL environments with Delphix, you will need to configure the source and target hosts with the requirements described on this page.

PostgreSQL Hosts and Databases 

On each host with Postgres, there must be an operating system user configured to the required specifications for Delphix, as explained in the table below.  These requirements apply to both source and target environments. However, target environments have additional requirements which are detailed in the ‘Target Host Requirements’ section below.

The PostgreSQL plugin requires the use of the default user which gets created during the Postgres DB installation process. Usually, 'postgres'. Please add the environment using this user and create all the datasets with the same. 

Source Database Requirements

Database RequirementExplanation
The source database can be in READ-WRITE or READ only mode. It can be a standby database(streaming site) as well.

The Staging host must have access to a PostgreSQL role on the source side that has replication, and login privileges. This can be the built-in PostgreSQL role or a newly-created role (for example, delphix).

Creating a Role for Use with the Delphix Engine

To create a new role for use with the Delphix Engine, use the following command:

SQL> CREATE USER delphix WITH REPLICATION ENCRYPTED [ PASSWORD 'password'];

You must make the following changes to postgresql.conf:

modifying the parameterlisten_addresses, which specifies the TCP/IP addresses on which the DBMS is to listen for connections from client applications.

Note: TCP/IP connectivity must be configured to allow the role mentioned above to connect to the source database from the Delphix Engine and from the standby DBMS instance set up by the Delphix Engine on the staging environment. 

listen_addresses configuration

The simplest way to configure PostgresSQL is so that it listens on all available IP interfaces:

listen_addresses = '*'    # Default is 'localhost'
(for more information, see the Server Configuration chapter in the PostgreSQL documentation)
The value of max_wal_senders, which specifies the maximum number of concurrent connections from standby servers or streaming base backup clients, must be increased from its desired value by two. That is, in addition to the allowance of connections for consumers other than the Delphix Engine, there must be an allowance for two additional connections from consumers set up by the Delphix Engine.

max_wal_senders Configuration

The default value max_wal_senders is zero, meaning replication is disabled. In this configuration, the value of max_wal_senders must be increased to two for the Delphix Engine:

max_wal_senders = 2       # Default is 0

The value of, wal_level which determines how much information is written to the write-ahead log (WAL), must be set to archive or hot_standby to allow connections from standby servers. The value "logical" for wal_level (introduced in PostgreSQL 9.4) is also supported.

In releases prior to PostgreSQL 9.6.x, parameter 'wal_level' allowed the values 'archive' and 'hot_standby'. These values are still accepted but mapped to 'replica'.
For example,
The plugin will work with ALTER SYSTEM SET wal_level TO 'replica' and ALTER SYSTEM SET wal_level TO 'hot_standby'.

wal_level Configuration

The default value of wal_level is minimal, which writes only the information needed to recover from a crash or immediate shutdown to the WAL archives. In this configuration, you must add the logging required for WAL archiving as follows:

wal_level = archive       # Default is minimal
You must configure PostgreSQL to allow PostgreSQL client connections and replication client connections from the staging target environment.

To configure appropriately, add the following entries to pg_hba.conf:

pg_hba.conf Configuration
host    all       <role>    <ip-address_of_delphix_engine>/32    <auth-method>

host    all       <role>    <ip-address_of_staging_target>/32    <auth-method>

host    replication    <role>    <ip-address_of_staging_target>/32    <auth-method>
<auth-method> can be set to trust, scram-sha-256, or md5 on source. Delphix inherits the same authentication method from the source while dSource creation and VDB provisioning. For more information on how to configure, pg_hba.conf see the Client Authentication section in the PostgreSQL documentation
wal_keep_segments parameter in postgresql.conf file on Source environment should be large enough to support the WAL sync process once the dSource is building up
It is mandatory to have "port" (can be commented or not commented)  in postgresql.conf file.

Target Host Requirements

Host RequirementExplanation
The operating system and architecture of the target environment must match those of the source environment. It is recommended that the source and the target environments should be identical and hardware configurations should match.
Compatible Operating System supported by the Plugin is CentOS 7.3/7.4/7.5/7.6 and RHEL 7.3/7.4/7.5/7.6.The underlying Operating System for both the Source and Staging environment should be amongst these two versions.
There must be an installation of PostgreSQL on the target environment that is compatible with the installation of PostgreSQL on the source environment.

Two installations of PostgreSQL are compatible if and only if:

  1. They share the same vendor (for example, PostgreSQL is incompatible with EnterpriseDB Postgres Plus Advanced Server).
  2. They share the same major version number (for example, 9.5.4 is compatible with 9.5, 9.5.3;  however, it is incompatible with 9.3, 9.3.24, or 9.2).
  3. They are compiled against the same architecture (in other words, 32-bit and 64-bit installations of Postgres are incompatible).
  4. They are compiled with the same WAL segment size. The default WAL segment size of 16 MB is rarely changed in practice, so almost all installations of PostgreSQL are compatible with each other in terms of WAL segment size.

There must be an operating system user (e.g postgres) with the following privileges:

  1. The Delphix Engine must be able to make an SSH connection to the target environment using the operating system user.
  2. The operating system user must have read and execute privileges on the PostgreSQL binaries installed on the target environment.
  3. The operating system user must have permission to run mount and unmount as the superuser via sudo with neither a password nor a TTY.
See Sudo Privilege Requirements for PostgreSQL Environments for further explanation of the commands, and Sudo File Configuration Examples for PostgreSQL Environments for examples of the /etc/sudoers file on different operating systems.

There must be a directory on the target environment where the Delphix Engine Plugin can be installed (for example,  /var/tmp ) with the following properties:

  1. The Plugin directory must be writable by the operating system user mentioned above.
  2. The Plugin directory must have at least 1.5 GB  of available storage.

There must be a mount point directory (for example, /mnt) that will be used as the base for mount points that are created when provisioning a VDB with the following properties:

  1. The mount point directory must be writable by the operating system user mentioned above.
  2. The mount point directory should be empty.

TCP/IP connectivity to and from the source environment must be configured as described in General Network and Connectivity Requirements.

Hostname and IP must be correctly set in /etc/hosts file, for example [postgres@source ~]$ hostname -i

The output of “hostname -i” command should produce the correct result as the IP address of the server.

For example:

[postgres@source postgres]$ hostname -i

10.110.207.113

When using the External Backup method for ingestion the directories being used for keeping Backup file and WAL file should be accessible by OS User.
Zip must be installed on the Staging/Target Host for External Backup Support.

The backup command and File should be in the below format:

PostgreSQL_T<YYYYMMDD>.zip where YYYY

denotes Year, MM denotes month, DD denotes date. For Example PostgreSQL_T20190314.zip.

The above zip file should contain a Database Full Backup from the Source created in tar format.

The below flags can be used with pg_basebackup for taking Source Database backup:

For PostgreSQL versions 9.4.x,9.5.x,9.6.x

<PATH_TO_PG_BIN>/pg_basebackup -p <PORT> -D <EMPTY_BACKUP_DIR> -F t -v -w -P -x

For PostgreSQL versions 10.x and 11.x

<PATH_TO_PG_BIN>/pg_basebackup -p <PORT> -D <EMPTY_BACKUP_DIR> -F t -v -w -P -Xs

If there's no PostgreSQL instance running on the Staging/Target host, however, we have a PostgreSQL installation, even then the plugin relies on the "DELPHIX_PG_PATH" variable to discover the environment.

In the absence of "DELPHIX_PG_PATH" variable or if the value of "DELPHIX_PG_PATH" variable is NULL then Linux "find" command will be used for the Environment discovery which may impact the overall performance. Hence, it is preferred to create this Environment Variable with correct entries.

In order to optimize the performance, it is preferred to create an Environment Variable "DELPHIX_PG_PATH" which should be accessible by OS users.

Below should be the syntax for "DELPHIX_PG_PATH" variable:

DELPHIX_PG_PATH="binary_path:data_path1;binary_path:data_path2;"

For example:

DELPHIX_PG_PATH="/usr/pgsql-9.6/bin:/var/lib/pgsql/9.6/data;/opt/edb/as9.6/bin:/opt/edb/as9.6/data;/usr/pgsql-9.6/bin:/tmp/TESING/data;"

The variable "DELPHIX_PG_PATH"  must be available to the environment user in a non-interactive way. You can test this variable for non-interactive logins using ssh <your_username>@<target_host> "env | grep DELPHIX_PG_PATH". 

Related Topics