Post-Installation

License, metadata database access, metadata schema, Apache Spark, smoke tests and Flexter UI setup.

License Setup

Copy the license file to one of these locations:

# For all users
/etc/flexter/conf/flexter.lic

# For the current user
~/.config/flexter/flexter.lic

# Same directory from which the Flexter command is launched
./flexter.lic

Flexter Metadata Access

Define the access to the metadata database in flexter.conf, setting the connection path and the password, in one of these locations:

# For all users
/etc/flexter/conf/flexter.conf

# For the current user
~/.config/flexter/flexter.conf

# Same directory from which the Flexter command is launched
./flexter.conf

For example:

metadata {
  path="jdbc:postgresql:x2er"
  user="flex2er"
  password="flexter"
}

The full list of settings is described in the Configuration section.

Installing/Updating Flexter Metadata Schema

With the license and the metadata access in place, load the schema into the database:

flexchma update

# or with explicit connection parameters
flexchma update -m "jdbc:postgresql:[<host>[:port]/]<db>" -M <user> -w <password>

Installing Apache Spark

When Apache Spark is not already available on the system, it can be downloaded with the download.sh module, described in detail in Flexter Modules / download.sh.

The script checks for an existing local Spark installation and downloads the default version if none is found. A specific Spark version can be requested with -v, and -f forces the switch to it. All options are listed with -h.

# default Spark version
sudo /usr/share/flexter/sbin/download.sh

# specific Spark version
sudo /usr/share/flexter/sbin/download.sh -v 3.3.4 -f

# all options
sudo /usr/share/flexter/sbin/download.sh -h

# default Spark version
~/.local/share/flexter/sbin/download.sh

# specific Spark version
~/.local/share/flexter/sbin/download.sh -v 3.3.4 -f

# all options
~/.local/share/flexter/sbin/download.sh -h

Smoke Tests

Checking Versions

Check that the commands execute successfully and report the expected versions:

flexchma --version
xsd2er --version
xml2er --version
json2er --version
merge2er --version

Checking The Help

Check that the help works and brings the defaults from the right configuration files:

flexchma --help
xsd2er --help
xml2er --help
json2er --help
merge2er --help

Testing

The packages ship sample files to exercise the whole pipeline. The paths differ between root and non-root installations.

Check that the files are parsed:

xsd2er -sRF /usr/share/flexter/samples/donut.xsd.gz
xml2er -s /usr/share/flexter/samples/donut.xml.gz
json2er -s /usr/share/flexter/samples/donut.json.gz

Check that Flexter writes metadata. The command prints the origin ID used in the following steps:

$ xsd2er -RF -g1 /usr/share/flexter/samples/donut.xsd.gz
   origin: <origin_id>

Check that xml2er loads the metadata:

xml2er -s -x<origin_id> /usr/share/flexter/samples/donut.xml.gz

Check that xml2er writes the parsed data:

xml2er -x<origin_id> /usr/share/flexter/samples/donut.xml.gz -o /tmp/flexter_test

Check that merge2er processes the files, in place and then to a different path:

merge2er /tmp/flexter_test
merge2er /tmp/flexter_test -o /tmp/flexter_test2

Check that the files are parsed:

xsd2er -sRF ~/.local/share/flexter/samples/donut.xsd.gz
xml2er -s ~/.local/share/flexter/samples/donut.xml.gz
json2er -s ~/.local/share/flexter/samples/donut.json.gz

Check that Flexter writes metadata. The command prints the origin ID used in the following steps:

$ xsd2er -RF -g1 ~/.local/share/flexter/samples/donut.xsd.gz
   origin: <origin_id>

Check that xml2er loads the metadata:

xml2er -s -x<origin_id> ~/.local/share/flexter/samples/donut.xml.gz

Check that xml2er writes the parsed data:

xml2er -x<origin_id> ~/.local/share/flexter/samples/donut.xml.gz -o /tmp/flexter_test

Check that merge2er processes the files, in place and then to a different path:

merge2er /tmp/flexter_test
merge2er /tmp/flexter_test -o /tmp/flexter_test2

Flexter UI Setup

Services

The flexter-ui package installs two systemd services.

flexterui-backend serves the API, and also the frontend when the standalone mode is active:

sudo systemctl <start|stop|restart|status> flexterui-backend

flexterui-frontend serves the frontend in production mode only:

sudo systemctl <start|stop|restart|status> flexterui-frontend

Settings

The flexter-ui settings are found in /usr/share/flexter/flexter-ui/conf by default.

env.json

Switches between standalone and production mode. Production requires both the backend and the frontend services; standalone only needs the backend.

{
  "env": "standalone"
}

backend.properties

For the installation, the datasource configuration matters most: it must point to the Flexter metadata database.

# Server configuration
server.host = 127.0.0.1
server.port = 8081

# Datasource configuration
spring.datasource.url = jdbc:postgresql:x2er?stringtype=unspecified
spring.datasource.username = flex2er
spring.datasource.password = flexter

# Storage for files and logs configuration
storage.path=/tmp
logs.path=/tmp

# Frontend configuration
cors.origins=*
pagination.pageSize = 20
#link.resource-manager=http://localhost:8088
#link.spark-history-server=http://localhost:18080

# security section
storage.enabled = false
cmd.enabled = false

debug=false
logging.path=/usr/share/flexter/flexter-ui/logs/

# Disable the default mappings
spring.resources.add-mappings=false
spring.mvc.throw-exception-if-no-handler-found=true

config.production.json

For the installation, apiUrl matters most: it must point to the server and port where the backend service is running.

{
  "apiUrl" : "http://localhost:9091/v2",
  "disableParameters": true,
  "disableNewWorkflow": true
}

Restart the services

After changing the settings, restart the services for them to take effect:

sudo systemctl restart flexterui-backend

# only if env.json is set to production mode
sudo systemctl restart flexterui-frontend