Post-Installation

The flexter post-installation steps

License Setup

Copy the license file in one of these paths:

# For all users
/etc/flexter/conf/flexter.lic

# For the current user
~/.config/flexter/flexter.lic

# same directory that you are launching a flexter command
./flexter.lic

Flexter Metadata Access

Define the access to the database server at flexter.conf replacing the metadata’s connection path and password, at one of these locations:

# For all users
/etc/flexter/conf/flexter.conf

# For the current user
~/.config/flexter/flexter.conf

# same directory that you are launching a flexter command
./flexter.conf

As the example below:

metadata {
  path="jdbc:postgresql:x2er"
  user="flex2er"
  password="flexter"
}

Installing/Updating Flexter Metadata Schema

With the license and the flexter metadata access, now it’s possible to load the schema into the database, as the example below:

# For all users
flexchma update

# or with parameters
flexchma update -m "jdbc:postgresql:[<host>[:port]/]<db>" -M <user> -w <password>

Installing the Apache Spark

In cases where the Apache Spark isn’t available in the system, it can be downloaded using the download.sh module.

It can be explored in detail in the section Flexter Modules / download.sh.

With root access

sudo /usr/share/flexter/sbin/download.sh spark

The above script will then check any existing local Spark installation, downloading the default one if no installation is found.

Optionally, you can specify the Spark version. For instance:

sudo /usr/share/flexter/sbin/download.sh spark -v 2.4.6 -f

More information about the script’s parameters can be listed with the -h option.

sudo /usr/share/flexter/sbin/download.sh spark -h

Without root access

~/.local/share/flexter/sbin/download.sh spark

The above script will then check any existing local Spark installation, downloading the default one if no installation is found.

Optionally, you can specify the Spark version. For instance:

~/.local/share/flexter/sbin/download.sh spark -v 2.4.6 -f

More information about the script’s parameters can be listed with the -h option.

~/.local/share/flexter/sbin/download.sh spark -h

Smoke Tests

Checking Versions

Checking if the commands execute successfully and have the expected versions.

flexchma --version
xsd2er --version
xml2er --version
json2er --version
merge2er --version

Checking The Help

Checking if the help works correctly and brings the defaults based on the right configuration files.

flexchma --help
xsd2er --help
xml2er --help
json2er --help
merge2er --help

Testing

With root access

Examples based on the flexter installation with root access

Check if the files are being parsed

xsd2er -sRF /usr/share/flexter/samples/donut.xsd.gz
xml2er -s /usr/share/flexter/samples/donut.xml.gz
json2er -s /usr/share/flexter/samples/donut.json.gz

Check if Flexter is successfully writing metadata

$ xsd2er -RF -g1 /usr/share/flexter/samples/donut.xsd.gz
       origin: <origin_id>

Check if xml2er is loading metadata

xml2er -s -x<origin_id> /usr/share/flexter/samples/donut.xml.gz

Check if xml2er is writing the parsed data

xml2er -x<origin_id> /usr/share/flexter/samples/donut.xml.gz -o /tmp/flexter_test

Check if merge2er is successfully processing files

merge2er /tmp/flexter_test

Check if merge2er is successfully processing files and writing files to a different path

merge2er /tmp/flexter_test -o /tmp/flexter_test2