Post Installation

The flexter post installation steps

License Setup

Copy the license file in one of these paths:

# For all users
/etc/flexter/conf/flexter.lic

# For the current user
~/.config/flexter/flexter.lic

# same directory that you are launching a flexter command
./flexter.lic

Flexter Metadada Access

Define the access to the database server at flexter.conf replacing the metadata’s connection path and password, at one of these locations:

# For all users
/etc/flexter/conf/flexter.conf

# For the current user
~/.config/flexter/flexter.conf

# same directory that you are launching a flexter command
./flexter.conf

As the example below:

metadata {
  path="jdbc:postgresql:x2er"
  user="flex2er"
  password="flexter"
}

Installing/Updating Flexter Metadata Schema

With the license and the flexter metadata access, now it’s possible to load the schema into the database, as the example below:

# For all users
$ flexchma update

# or with parameters
$ flexchma update -m "jdbc:postgresql:[<host>[:port]/]<db>" -M <user> -w <password>

Installing the Apache Spark

In cases where the apache spark isn’t available in the system, it can be downloaded using the script as below:

With root access

$ /usr/share/flexter/sbin/download.sh spark

The above script will then check any existing local Spark installation, downloading the default one if no installation is found.

Optionally, you can specify the Spark version. For instance:

$ /usr/share/flexter/sbin/download.sh spark -v 3.1.2 -f

More information about the script’s parameters can be listed with the --help option.

$ /usr/share/flexter/sbin/download.sh spark --help

Without root access

$ ~/.local/share/flexter/sbin/download.sh spark

The above script will then check any existing local Spark installation, downloading the default one if no installation is found.

Optionally, you can specify the Spark version. For instance:

$ ~/.local/share/flexter/sbin/download.sh spark -v 3.1.2 -f

More information about the script’s parameters can be listed with the --help option.

$ ~/.local/share/flexter/sbin/download.sh spark --help

Smoke Tests

Checking Versions

Checking if the commands execute successfully and have the expected versions.

$ flexchma --version
$ xsd2er --version
$ xml2er --version
$ json2er --version
$ merge2er --version

Checking The Help

Checking if the help works correctly, and bring the defaults based on the right configuration files.

$ flexchma --help
$ xsd2er --help
$ xml2er --help
$ json2er --help
$ merge2er --help

Testing

With root access

Examples based on the flexter installation with root access

Check if the files are being parsed

$ xsd2er -sRF /usr/share/flexter/samples/donut.xsd.gz
$ xml2er -s /usr/share/flexter/samples/donut.xml.gz
$ json2er -s /usr/share/flexter/samples/donut.json.gz

Check if Flexter is successfully writing metadata

$ xsd2er -RF -g1 /usr/share/flexter/samples/donut.xsd.gz
   origin: <origin_id>

Check if xml2er is loading metadata

$ xml2er -s -x<origin_id> /usr/share/flexter/samples/donut.xml.gz

Check if xml2er is writing the parsed data

$ xml2er -x<origin_id> /usr/share/flexter/samples/donut.xml.gz -o /tmp/flexter_test

Check if merge2er is successfully processing files

$ merge2er /tmp/flexter_test

Check if merge2er is successfully processing files and writing files to a different path

$ merge2er /tmp/flexter_test -o /tmp/flexter_test2

Without root access

Examples based on the flexter installation without root access

$ xsd2er -sRF ~/.local/share/flexter/samples/donut.xsd.gz
$ xml2er -s ~/.local/share/flexter/samples/donut.xml.gz
$ json2er -s ~/.local/share/flexter/samples/donut.json.gz

Check if Flexter is successfully writing metadata

$ xsd2er -RF -g1 ~/.local/share/flexter/samples/donut.xsd.gz
   origin: <origin_id>

Check if xml2er is loading metadata

$ xml2er -s -x<origin_id> ~/.local/share/flexter/samples/donut.xml.gz

Check if xml2er is writing the parsed data

$ xml2er -x<origin_id> ~/.local/share/flexter/samples/donut.xml.gz -o /tmp/flexter_test

Check if merge2er is successfully processing files

$ merge2er /tmp/flexter_test

Check if merge2er is successfully processing files and writing files to a different path

$ merge2er /tmp/flexter_test -o /tmp/flexter_test2

Flexter UI Setup

Services

To get all the supported flexter-ui services the following command can be used:

flexterui-backend

It works both for backend and frontend if the standalone mode is active.

$ systemctl <start|stop|restart|status> flexterui-backend

flexterui-frontend

$ systemctl <start|stop|restart|status> flexterui-frontend

Settings

The flexter-ui settings can be found at /usr/share/flexter/flexter-ui/conf by default

env.json

This settings can be found at /usr/share/flexter/flexter-ui/conf/env.json

It offer options to switch between standalone and production, where the production requires both backend and frontservices to work, and standalone only the backend.

{
  "env": "standalone"
}

backend.properties

This settings can be found at /usr/share/flexter/flexter-ui/conf/backend.properties

For the installation propouses, it’s important to pay attention at the datasource configuration, which should access the flexter metadata database.

# Server configuration
server.host = 127.0.0.1
server.port = 8081

# Datasource configuration
spring.datasource.url = jdbc:postgresql:x2er?stringtype=unspecified
spring.datasource.username = flex2er
spring.datasource.password = flexter

# Storage for files and logs configuration
storage.path=/tmp
logs.path=/tmp

# Frontend configuration
cors.origins=*
pagination.pageSize = 20
#link.resource-manager=http://localhost:8088
#link.spark-history-server=http://localhost:18080

# security section
storage.enabled = false
cmd.enabled = false

debug=false
logging.path=/usr/share/flexter/flexter-ui/logs/

# Disable the default mappings
spring.resources.add-mappings=false
spring.mvc.throw-exception-if-no-handler-found=true

config.production.json

This settings can be found at /usr/share/flexter/flexter-ui/conf/config.production.json

For the installation propouses, it’s important to pay attention at apiURL which should point the correct server and port where the backend service is running.

{
  "apiUrl" : "http://localhost:9091/v2",
  "disableParameters": true,
  "disableNewWorkflow": true
}

Restart the services

After the changes, the services must be restarted to take affect.

flexterui-backend

$ systemctl restart flexterui-backend

flexterui-frontend

Optional, if the the env.json is set in the production mode

$ systemctl restart flexterui-frontend