Toolkit Reference

The Olympe toolkit (@olympeio/toolkit npm package) is the tool blended to handle operations with Olympe environments, in particular operations related to the code as data.

The toolkit is generally used as npm devDependency in the Olympe projects to build the appropriate code as data files (also named patches) and install or deploy them to your environment. The toolkit is also available in DockerHub as docker image to be used directly in jobs on your kubernetes cloud (for Saas or on-premise Olympe environments).

Execute toolkit commands

The toolkit is a CLI tool written in JavaScript that can either be installed globally or locally in your project, or be deployed on a docker/kubernetes environment.

Use the help command to know about existing commands:

To install the toolkit locally in your project, execute the following command in a terminal:

npm install --save-dev @olympeio/toolkit

or globally on your computer, available as a global binary:

npm install --global @olympeio/toolkit

To execute a command, use the following syntax:

npx olympe <command> [--<param> <value> ...]

For instance, use this to list the available commands:

npx olympe help

The list of commands and parameters can be found in the reference section.

To use the toolkit with docker, simply pull the `stable` version of the toolkit and run it using the same syntax as for `npm`:

docker run olympeio/toolkit:stable <command> [--<param> <value> ...]

For instance, use this to list the available commands:

docker run olympeio/toolkit:stable help

When using docker images, you can also provide parameters using the environment variables. This is typically used to define jobs to be executed on the instance with the right parameters and secrets injected. The list of commands and parameters can be found in the reference section.

To run a command using the toolkit locally installed as npm dev-dependency use this in a terminal:

npx olympe <command> [--<parameter name> <parameter value>, ...]

For each command, use the -h, --help parameter to see the detailed list of parameter to use in that command.

Code as data

When developing applications on the Olympe platform, developers combine and compose pieces of code, known as coded bricks, using DRAW to create processes and applications. The system generates data in a graph database, referred to as the composition database. While applications composed of brick assemblies offer many advantages over traditional source code files, they require tools to extract these assemblies, create library packages, and deploy them to other environments.

The term Code as Data designates everything that developers assemble in Draw and store in the database. This is code, but stored like data instead of being stored in text files. Hence, that code must be considered as part of the source code of projects built on Olympe.

The files that are used to store the code as data are json files and named patches.

Build Code As Data as Library

When building a library, it is important to only include the source code of that library without bundling the dependencies code. This a standard process when using javascript bundlers like webpack. The same approach must be applied with the code as data.

Use the buildCodeAsData command from the toolkit to build the snapshot folder of your project and configuration files. But make sure to specify the toolkit to only consider patches from the current project and not from dependencies:

npx olympe buildCodeAsData --codeAsData.mode project

This will generate all the patches with the appropriate configuration files for the project to be deployed on an environment in the dist/codeAsData directory.

The dist is typically used as the base folder to store all files intended to be packaged and published on your/the NPM registry. As an Olympe library, the package.json file of your library must contain the specific codeAsData extra entry which indicates the path where to find the patches of your package:

package.json

{
  "codeAsData": "codeAsData"
}

caution

This informs the toolkit about which dependencies are Olympe projects and contain code as data, particularly when building a project for deployment on an environment.

Once the library is ready to be published on NPM registry, it typically looks like this:

and the package.json file should look like this:

Updating code as data

Once the images are built and deployed, it's crucial to update the code as data to reflect the changes in the environment. This is achieved by the deployment pipeline but can be executed manually using the update command from the toolkit. The update process requires the new patches to be deployed to the orchestrator volume, allowing it to compare these patches with the current state of the database.

To optimize the update process, the toolkit can utilize a changelog file generated by rsync when the new patches are deployed to the orchestrator. This file contains a list of patches that have been added, modified, or deleted, enabling the update command to accelerate the update process. If you prefer not to use the changelog file, you can disable it with the --no-update.useChangeLog parameter.

If the update process fails, the toolkit will attempt to rollback the applied patches to restore the database to its previous state. This can happen when patches have some errors or are incompatible with the current database state. The rollback process restores the patches from the last successful update or install, then calculates the differences between the current state and the last successful state to execute the rollback effectively.

The update can be performed in two modes:

Downtime mode (by default)

In Downtime mode, the orchestrator enters maintenance mode during the update, disconnecting all connected clients and temporarily halting application usage. Once the update is complete, the orchestrator returns to normal mode, allowing clients to resume using the application. This mode is recommended for development environments.

No Downtime mode

In No Downtime mode, the orchestrator remains operational during the update, allowing clients to continue using the application. However, during the update, the code as data is in read-only mode, meaning only business data can be modified. After the update, the application prompts users to refresh the page to load the updated code as data. This mode is recommended for production environments as it ensures minimal disruption and provides an opportunity to verify that the deployed code is functioning as expected before it is fully utilized. This is accomplished using the blue-green deployment strategy, which involves maintaining two sets of pods to separate the active and preview versions of the application. The active version remains accessible via https://$APPLICATION.olympe.io, while the preview version can be accessed at https://preview.$APPLICATION.olympe.io. Once the preview version is validated, it can be approved in the pipeline, promoting it to become the new active version.

To activate this mode, please contact your Olympe referent.

Toolkit command parameters

Shell parameter	Environment variable	Type	Default value	Description
--logLevel, -l	LOG_LEVEL	string	info	logging level, one of debug, verbose, info, warn, or error. (Eg. olympe install --logLevel warn)
--help, -h		boolean	false	display help for the current command
--saveLogs	SAVE_LOGS	boolean	false	save logs to a file under directory logs/
--codeAsData.workingDir		string	./dist/codeAsData	relative (to the project's root) path to the temporary directory used to store generated code as data
--codeAsData.mode		string	all	specify if the code as data to process is aimed to be used in a library package (only the code as data of the current project) or in a project to be deployed (including all the dependencies). Accepted values dependencies, project, all
--snapshot.excludeUsers	SNAPSHOTER_EXCLUDE_USERS	boolean	false	whether to exclude users from dump of business data
--snapshot.dumpBinaries		boolean	false	whether to dump binary files while dumping business data
--snapshot.dataRootTag		string	10000000000000000000	what root tag to use to dump business data
--snapshot.userDataDir	SNAPSHOT_USER_DATA_FOLDER	string	users	snapshot directory sub-folder to dump users
--snapshot.businessDataDir	SNAPSHOT_BUSINESS_DATA_FOLDER	string	data	snapshot directory sub-folder to dump business data
--snapshot.repositoryDir		string	/tmp/repositories	define directory where the snapshooter can clone repositories if config contains git configuration
--snapshot.commitMessage	SNAPSHOT_COMMIT_MESSAGE	string	Snapshot at {date} in {folder}	define a commit message for snapshot
--snapshot.dir		string	./snapshot	relative (to the project's root) path to the directory containing the project's snapshots
--versionFileName		string	version.json	Name of the file containing the project's Draw/Framework versions
--runningInDocker		boolean	false	global option to set whether the Olympe Toolkit is running inside a docker image. Primarily used when calling the install command on local environment to determine whether to copy the patches inside the docker container or not .
--jsonConfig		json		set the commands configuration via a JSON config
--downloadFolder		string	/tmp/toolkit	path to the temporary download folder used during update command.
--user.tag	DRAW_USERTAG	string	014831d95fd7d12b8568	tag of the user to update (command updateUser)
--user.name	DRAW_USERNAME	string	admin	new username to set for the specified user.tag (command updateUser)
--user.password	DRAW_PASSWORD	string	admin	new password to set for the specified user.tag (command updateUser)
--import.debug	LOAD_PATCHES_DEBUG	boolean	false	whether the orchestrator will import patches in debug mode
--import.ignoreRelationFail	IGNORE_RELATION_FAIL	boolean	false	when importing patches, ignore relations that cannot be created because of the nodes missing
--import.backupUsers	BACKUP_USERS	boolean	true	whether to backup users before resetting the database when using install command
--import.backupBusinessData	BACKUP_BUSINESS_DATA	boolean	true	whether to backup business data before resetting the database when using install command
--import.timeout	RESET_DB_TIMEOUT	number	3600000	the timeout for importing all patches when using install command, in ms.
--orchestrator.host	ORCHESTRATOR_HOST	string	localhost	orchestrator http host
--orchestrator.port	ORCHESTRATOR_SERVICE_PORT	number	8080	orchestrator http port
--orchestrator.userName	DRAW_USERNAME	string	admin	user for Orchestrator connection
--orchestrator.password	DRAW_PASSWORD	string	admin	user password for Orchestrator connection
--orchestrator.importOrderFile		string	import_order.json	Name of the file that specifies the import order of modules
--orchestrator.containerUser		string	vertx	user which is used to copy patches inside of the orchestrator container
--orchestrator.codeAsDataDir		string	/patches	the path to copy code as data folder in the orchestrator container. Used exclusively when the toolkit is not running in a container.
--bus.protocol		string	amqp	protocol used for bus communication
--bus.hostname	RABBITMQ_HOST	string	localhost	host for bus communication
--bus.port	RABBITMQ_PORT	number	5672	port for bus communication
--bus.username	RABBITMQ_ORCHESTRATOR_USER	string	guest	username used to connect to the bus
--bus.password	RABBITMQ_ORCHESTRATOR_PASSWORD	string	guest	user password used to connect to the bus
--bus.vhost	RABBITMQ_ORCHESTRATOR_VHOST	string	/	virtual host used for any communication sent on the bus
--bus.commandsQueue	RABBITMQ_ORCHESTRATOR_COMMAND_QUEUE	string	COMMAND_QUEUE	rabbitmq queue used for commands sent to the orchestratr
--bus.commandsExchange	RABBITMQ_ORCHESTRATOR_COMMAND_EXCHANGE	string	""	rabbitmq exchange used for commands sent to the orchestrator
--bus.deploymentQueue	RABBITMQ_ORCHESTRATOR_DEPLOYMENT_QUEUE	string	SERVICE_DEPLOYMENT	queue used to send Orchestrator deployment commands
--bus.deploymentExchange	RABBITMQ_ORCHESTRATOR_DEPLOYMENT_EXCHANGE	string	ORCHESTRATOR_BROADCAST_EXCHANGE	exchange for Orchestrator deployment commands
--bus.commandTimeout	RABBITMQ_ORCHESTRATOR_COMMAND_TIMEOUT	number	600000	timeout applied for standard commands sent to the orchestrator (in ms)
--update.useChangeLog	UPDATE_USE_CHANGELOG	boolean	true	whether to use a changelog file to accelerate the database update process (can be negated using --no-update.useChangeLog)
--update.changeLogName		string	change_log.txt	name of the file giving the patches change log
--update.excludeTags	UPDATE_EXCLUDED_TAGS	string	[]	list of tags of projects/folders to exclude from the check of the update command. In a development environment, the appropriate value is to put all the tags of projects that are snapshot in that environment.
--update.includeTags	UPDATE_INCLUDED_TAGS	string	[]	list of tags of projects/folders should be included from the check of the update command, even though one of their ancestor in the Draw hierarchy has been excluded using the --update.excludeTags option.
--update.noDowntime	UPDATE_NO_DOWNTIME	boolean	false	if true, the update will be done without downtime. This should be activated only for production environments.

Toolkit Reference

Command	Typical use	Description
help	N/A	Display the list of available commands with the toolkit, as well as the associated parameters
snapshot	Docker: scheduled CI/CD	Download and save the code as data into patches (json files) from a deployed environment
buildCodeAsData	NPM: within CI/CD	From the project snapshot files and its dependencies, build the dist/codeAsData folder containing all the patches required to build/deploy the project to an environment
update	Docker: Olympe cloud job	Update the graph database containing the code as data of an environment to be aligned with the patches deployed to the orchestrator.
codeAsDataReadMode	Sub-command of update	Set Orchestrator code as data read mode to true/false
broadcastRefreshApp	Sub-command of update	Broadcast a message to refresh the application for all connected users
install	Docker: Olympe cloud job	Clean and Install the content of patches deployed to the orchestrator. /!\ This command starts by clearing the whole composition database, so it must be used with caution.
snapshotUsers	Sub-command of install	Save users and their Draw permissions of an environment to patches (mainly as sub-command in process of install)
restoreUsers	Sub-command of install	Restore patches generated with command snapshotUsers to the database (mainly as sub-command in process of install)
snapshotBusinessData	Sub-command of install	Save business data (business objects visible in Runtime Data Set of projects), stored in the "Embedded graph database" of an environment to patches (mainly as sub-command in process of install)
restoreBusinessData	Sub-command of install	Restore patches generated with command snapshotBusinessData to the database (mainly as sub-command in process of install)
dependencies	NPM: manually	List all npm dependencies from node_modules folder which contain some code as data
updateUser	Docker: Olympe cloud job	Update the username and password of the admin user
maintenance	Docker: Olympe cloud job	Activate/Deactivate the maintenance mode of the orchestrator. When activated, ensures that all connected clients to the orchestrator are disconnected.
getUpgradeConfig	Docker: during deployment	Generate the configuration for the upgrade tool to apply upgrade script on the composition database. Used by the code as data deployment workflow to automatically upgrade the language version from one major version to another.
startGC	Docker: Olympe cloud job	Request the orchestrator to garbage collect useless files in file and backup services
statsDB		Return information about the composition database