Overview

As with many other types of software, toolkit releases can be versioned, and can define logic that handles upgrading between versions.

Included in the Toolkit DevKit (available from our download site) are three example source trees which illustrate this topic. Each represent different releases of the same toolkit:

• versioning-original contains a simple toolkit which defines some of its own datatypes.

• versioning-patch makes a bugfix, but makes no datatype changes.

• versioning-upgrade changes datatypes and provides upgrade scripts.

Identifying Your Toolkit's Version

A toolkit must define its version using a three-part string of the form X.Y.Z Here "X" is the "major number", "Y" is the "minor number", and "Z" is a patch level. This is defined inside the toolkit's main.json file.

• Minor and major numbers are integers. See other restrictions below.
• Patch level can be any sequence of letters, numbers or underscores. 
  
  

Examples of valid version strings:

• 1.0.0
• 4.3.7
• 2.2.fix_sorting_bug

Rules for New Toolkit Releases

There are three different types of toolkit releases, and they have different rules around what you are allowed to change, and what you must change.

Rules for a new major release:

• The major number in the version string must be incremented. That is, it must be 1 greater than the previous release's major number.
• The minor number must be reset to 0.
• You are allowed to change any/all of the schemas defined in the main.json file.
• You must provide upgrade scripts. These scripts will convert data written by an old version of your toolkit to the format expected by your new toolkit version.

Rules for a new minor release:

• The minor number in the version string must be incremented relative to the previous release.
• The major number must be unchanged from the previous release.
• You are allowed to change any/all of the schemas defined in the main.json file.
• You must provide upgrade scripts. These scripts will convert data written by an old version of your toolkit to the format expected by your new toolkit version.

Rules for a new patch releases:

• Neither the minor number nor the major number may change.
• The patch level must change. Any valid patch string is allowed.
• You may not change any of the schemas in the main.json file.
• You do not need to provide any upgrade scripts.

In summary, if you need to change a datatype, you must make a new major or minor release. If you do not need to change a datatype, you may choose to make a new major, minor, or patch release. Apart from these constraints, it is up to the toolkit developer to decide which kinds of changes count as "major", "minor", or "patch".

For examples of different types of toolkit releases, see the "versioning" toolkits in the downloadable Toolkit DevKit.

Rules For Versions and Upgrading

If an engine does not already have a version of your toolkit installed, then there are no special rules about which versions will be accepted. Any valid version string will be allowed.

However, if the engine already does have a version of your toolkit, then there are some constriants about which new versions will be accepted:

• Major/minor numbers must never decrease. It is never allowed to "downgrade" to a lower major or minor number when uploading a toolkit.
• Each new major or minor release can only be installed on top of the previous one. It is not possible to "skip" a release. If a user is on version 1.0 of a toolkit, and versions 1.1, 1.2, and 2.0 have been released, then the user must follow the upgrade chain and sequentially install 1.1, then 1.2, then 2.0 – they cannot install 2.0 directly on top of 1.1.

Note that there is no defined ordering of patch levels. So long as the major and minor numbers are unchanged, any patch level may be uploaded to replace any other patch level.

Defining Upgrade Logic

A toolkit defines a number of data formats by including schemas in its main.json file. As a user interacts with the toolkit, objects are created in these formats, and these objects are stored for later use.

For example, when a user takes a snapshot of a dSource or VDB, the toolkit's postSnapshot Lua script is run. This script returns an object conforming to the snapshotSchema in the toolkit's main.json. This object is stored away. Later, if a user wants to provision a new VDB from this snapshot, this object will be retrieved and passed into the toolkit's configure Lua script.

But, now consider a case where a new version of a toolkit is released, and this new version makes some changes to the snapshotSchema. Obviously, this new toolkit version's code will expect to see snapshots in the "new" format, as defined in its snapshotSchema.

This is where upgrade scripts come in. An upgrade script's job is to modify an "old" object that conforms to the previous version's schema. It must return a "new-format" object that conforms to the new version's schema.

There are four upgrade scripts:

• upgradeLinkedSource
• upgradeVirtualSource
• upgradeSourceConfig
• upgradeSnapshot

Each of these scripts live in a nested subdirectory upgrade/<version>, where "version" is only the major and minor parts of the version that your scripts are upgrading from. So for example, if your new toolkit is version 5.4.x, and your most recently released toolkit was version 5.3.x, then you would put your upgrade scripts in the upgrade/5.3 directory.

Examples

For a simple example, consider a case where a new toolkit version adds a new property to a schema. First, compare the following two virtualSourceSchema entries:

"virtualSourceDefinition": {
        "type": "ToolkitVirtualSource",
        "parameters": {
            "type" : "object",
            "additionalProperties" : false,
            "required": ["dataPath"],
            "properties" : {
                "dataPath" : {
                    "type": "string",
                    "prettyName": "Data Path",
                    "description": "Where the data should be mounted on the target host"
                },
                "comment" : {
                    "type": "string",
                    "prettyName": "Comment",
                    "description": "User comment"
                }
            }
        }
    },

    "virtualSourceDefinition": {
        "type": "ToolkitVirtualSource",
        "parameters": {
            "type" : "object",
            "additionalProperties" : false,
            "required": ["dataPath", "dataDescription"],
            "properties" : {
                "dataPath" : {
                    "type": "string",
                    "prettyName": "Data Path",
                    "description": "Where the data should be mounted on the target host"
                },
                "dataDescription": {
                    "type": "string",
                    "prettyName": "Data Descrption",
                    "description": "Brief description of what data this VDB holds"
                }
            }
        }
    },

Notice that there are two changes. First, there is a new string property called "dataDescription" that has been added in the new toolkit version. This new property is marked as required. But, of course any objects created by the old version of the toolkit will not have this property. Therefore, we must write an upgrade script that knows how to take an old object and add a value for this new property. Similarly, the property "comment" has disappeared, and so our upgrade script must remove this property from the old objects.

-- This script's job is to modify a virtual source's parameters. We need to change it from the format used by the old
-- toolkit version to the format expected by this new version.



-- In this version, we are adding a new required property to virtual sources called "dataDescription".
-- Therefore, we must add a new field called "dataDescription" to our parameter set.

parameters.dataDescription = "Data located at " .. parameters.dataPath



-- In this version, we no longer support a "comment" property on virtual sources.
-- Therefore, we must remove the "comment" field from our parameter set.

parameters.comment = nil




-- Now that all the required modifications have been made, we can return the parameter set.
return parameters

In some cases, a schema might not change at all in a new major/minor version. In that case, the upgrade script can simply return the old-format object as-is, making no changes to the object at all.

-- We are making no changes to the sourceConfigSchema, so we don't need to change anything here.
-- We can simply return the old object unchanged.

return config

For examples of upgrade scripts that handle more complicated changes to schemas, please see the toolkits in the downloadable Toolkit DevKit.