Kiuwan On-Premises Advanced Installation: Externalizing Services

Configure a password in Redis

The default configuration of Kiuwan does not set any password in Redis. 

To configure a password in Redis and configure Kiuwan to connect to Redis using a password, follow these steps:

 If you are installing Kiuwan for the first time, make sure that you have completed at least the “Step 6: initialize your volumes” described in Kiuwan On-Premises Distributed Installation Guide before continuing.

Step 1: Stop Kiuwan

If you are installing Kiuwan for the first time in your instance, skip this step.

If you are modifying the configuration of an existing Kiuwan installation, stop executing it:

cd [INSTALLER_DIR]
sudo ./stop-kiuwan.sh
sudo ./start-kiuwan.sh

Step 2: Edit the global configuration file

Using your preferred editor, open the default configuration file located in your config-shared volume. In the following example, the used editor is Vim:

sudo vim [VOLUMES_DIR]/config-shared/globalConfig/globalConfig.properties

Edit these properties and assign a password to be used on connecting to Redis; you should set the same value for both redis.cache and redis.store password:

• redis.cache.password={YOUR_REDIS_PASSWORD}
• redis.store.password={YOUR_REDIS_PASSWORD}

Step 3: Activate authentication in Redis

You should edit the Redis configuration files located in the following path:

sudo vim [VOLUMES_DIR]/config-shared/Redis/conf/redis[1-6].conf

Add the following lines in each of them (6 files in total):

If you have externalized the Redis installation, you should edit the equivalent configuration file to activate the authentication and restart the externalized Redis installation.

Step 4: Propagate the changes in Kiuwan

If you are installing Kiuwan for the first time in your instance, skip this step.

To propagate the changes in globalConfig.properties, execute the following commands:

cd [INSTALLER_DIR]/docker
sudo ./update.sh

Step 5: Start kiuwan

All the required changes are completed. If you are installing Kiuwan for the first time make sure that you complete all the installation steps and start Kiuwan. 

If you are modifying an existing installation start Kiuwan and the setup is completed:

Externalizing services

Kiuwan On-Premises uses three main services under its infrastructure's hood:

• Apache - used as a load balancer when multiple Kiuwan frontal instances are running.
• Redis - an in-memory cache to speed up response times.
• MySQL - Kiuwan's main database.

If you want to use your services for any of the previous ones, Kiuwan On-Premises can connect to them by bypassing their creation at installation time.

Configuring services to externalize

First of all, you will need to edit the main configuration file and mark which services you want to externalize:

• [VOLUMES_DIR]/config-shared/globalConfig/globalConfig.properties

This table shows the properties you should modify when externalizing each service:

When setting to "true" any of the previous properties, the corresponding service will be externalized and the installation tool will not manage any related instance. Note that all the configuration will be up to you, as the Kiuwan On-Premise installer will only be able to configure how Kiuwan On-Premise will connect to your services.

Externalizing Apache

When externalizing this service you should take into account that:

• Each Kiuwan On-Premises frontal instance domain name is wildflykiuwan-f[n], [n] being the frontal instance number. Note that depending on your installation needs you may want to access each instance via IP or its hostname.
• Each Kiuwan On-Premises frontal instance only exposes one port for https/http/ajp traffic. By default, the port 8[n]43, [n] being the frontal instance number will be exposed. If you need to change the exposed ports refer to  Changing exposed ports for more information.
• In case you are accessing through https scheme, you should provide your frontal service the needed certificates to make https connections available (please refer to Managing certificates for more information):
  • domaincert.pem
  • domainkey.pem
  • cacert.pem

Modify globalConfig.properties and set kiuwan.externalize.apacheloadbalancer=true

Refer to Modifying exposed ports in Kiuwan front instances and modify the exposed ports in Kiuwan front instances.

Offload load balancer through container, which is mapped to local port 80 and 443.

Open browser and visit https://localhost/saas/web/dashboard/dashboard 

Externalizing Redis

Kiuwan On-Premises must work with your Redis installation so that it complies with these characteristics:

• Redis version must be equal or higher than 5.0.4.
• Redis must be configured as a cluster.
• Eviction policy must be set to "noeviction" (refer to Redis official documentation, maxmemory-policy configuration property).

In case you set Redis as an external service, Kiuwan On-Premises needs to know where the Redis nodes are deployed and which ports to use when connecting to them.

In case you use a special DNS that can resolve the same host to different hosts and ports (DNS Round-Robin or equivalent), you should configure just a single host in the redis.[cache|store].nodes properties.

All the needed configuration is located in the main configuration file:

[VOLUMES_DIR]/config-shared/globalConfig/globalConfig.properties

The following table shows the properties to configure (note that you should set the same configuration for both "cache" and "store" Redis configurations):

Externalizing MySQL

When externalizing MySQL, note that your MySQL installation should comply with these characteristics:

• MySQL version 5.7
• The maximum number of connections: 130 per Kiuwan On-Premises instance.

Step 1: Create Kiuwan On-Premises schemas

You should create the needed schemas in your MySQL installation. To do so, please execute this script with a user who has schema creation privileges:

create database opt_activity CHARACTER SET utf8 COLLATE utf8_unicode_ci;

create database opt_cinc CHARACTER SET utf8 COLLATE utf8_unicode_ci;

create database opt_metamodel CHARACTER SET utf8 COLLATE utf8_unicode_ci;

create database opt_qmm CHARACTER SET utf8 COLLATE utf8_unicode_ci;

create database opt_transaction CHARACTER SET utf8 COLLATE utf8_unicode_ci;

create database opt_insight CHARACTER SET utf8 COLLATE utf8_unicode_ci;

To generate the required tables, run all the .sql scripts present in the directories:

/installation/resources/install-db and /installation/resources/install-db-data-core

Step 2: Create Kiuwan On-Premises user

You should create the user that will be connecting to Kiuwan On-Premises schemas. Please run this script as an admin user to do so:

create user '[USER]'@'%' identified by '[PASSWORD]';

grant all privileges on `opt_%`.* to '[USER]'@'%' identified by

'[PASSWORD]';

flush privileges;

Note that you should replace [USER] with the desired user name and [PASSWORD] with the desired password.

Step 3: Configure your installation

The following table shows the properties to configure for Kiuwan On-Premises to connect to your MySQL instance:

mysql.host

mysql.port

mysql.username

mysql.password

Step 4: Set kiuwan.externalize.mysql=true in globalConfig.properties

Open browser and visit  https://localhost/saas/web/dashboard/dashboard

Using Amazon S3 as file repository

Kiuwan On-Premises uses these shared file repositories to store analysis-related data:

• kiuwanCentralRepository: stores analysis results files.
• kiuwanSourceCodeRepository: stores source code.
• kiuwanCustomConfigRepository: stores centralized analysis configuration files.

These Kiuwan On-Premises internal file repositories can be replaced with Amazon S3 buckets.

To do so, you should first configure these properties in the main configuration file ([VOLUMES_DIR]/config-shared/globalConfig/globalConfig.properties):

• centralFileRepository.type=s3
• sourceCodeFileRepository.type=s3
• customConfigFileRepository.type=s3

The following table shows the properties you should modify when making Kiuwan On-Premises connect to AWS S3 buckets:

Automatic data purge

To keep used disk space under control, Kiuwan On-Premises is capable of automatically purging unnecessary files and, optionally, old data that you may not need anymore.

By default, Kiuwan On-Premises purges:

• Analyses that are not among the last 50 processed (for each application).
• Analyses older than 5 years (1825 days).
• Support files older than 6 months (180 days).

The previous thresholds can be configured through the following properties:

• kiuwan.purge.analysis.lastAnalysesToKeep
• kiuwan.purge.analysis.antiquity.days
• kiuwan.purge.support.files.antiquity.days

In case you do not want to delete and purge any of your data, simply change the active flag to false in the following properties:

• kiuwan.purge.analysis.data.active
• kiuwan.purge.support.files.active

Also, turn off kiuwan.purge.analysis.files.active flag, but this is only recommended for troubleshooting purposes since you will deactivate a task that only purges temporary files that are not needed when using Kiuwan On-Premises.

All the configuration properties can be found inside the main configuration file:

• [VOLUMES_DIR]/config-shared/globalConfig/globalConfig.properties

Internally, Kiuwan On-Premises uses 5 scheduled tasks to implement file and data purge:

• Soft-delete “old” analyses: performs a logical deletion of analyses that are considered “old”.
• Purge soft-deleted analysis data: purges the data associated with soft-deleted analyses.
• Purge “old” support files: purges “old” files that have been stored for support purposes.
• Purge analysis shared files: purges files that belong to soft-deleted analyses (stored in the data-shared volume).
• Purge analysis local temporary files: purges internal temporary data that belong to “old” analyses and is no longer needed (locally stored on front and analyzer nodes).

The previous tasks are executed daily, following this timetable:

Configuration options reference

All configuration properties you can modify are located in this file, located inside your config-shared volume:

[VOLUMES_DIR]/config-shared/globalConfig/globalConfig.properties

Here is a complete list of the properties you can configure and their meaning (default passwords are omitted):

Installer Behavior

Kiuwan Connection

Kiuwan instances alternative proxy

These properties only apply when kiuwan.proxy.autoconfig is false. Note that the https properties are used to indicate the proxy to use when making https connections, not to configure a proxy that should be accessed through https. Ignore these properties if you want Kiuwan instances to use the proxy configured in your Docker installation.

Kiuwan instances shared configuration

Kiuwan file repositories

Kiuwan file repositories

Kiuwan front instances

Kiuwan analyzer instances

Kiuwan scheduler instances

Apache load balancer

MySQL

Redis cluster cache and store

Supported timezones reference

The following table shows the available time zones that can be set to the Kiuwan On-Premises server when editing the main configuration file located in:

[VOLUMES_DIR]/config-shared/globalConfig/globalConfig.properties

Note that you can set only one of these values to the timezone property found in the previous file: