Post-Installation

The flexter post-installation steps

License Setup

Copy the license file in one of these paths:

# For all users
/etc/flexter/conf/flexter.lic

# For the current user
~/.config/flexter/flexter.lic

# same directory that you are launching a flexter command
./flexter.lic

Flexter Metadata Access

Define the access to the database server at flexter.conf replacing the metadata’s connection path and password, at one of these locations:

# For all users
/etc/flexter/conf/flexter.conf

# For the current user
~/.config/flexter/flexter.conf

# same directory that you are launching a flexter command
./flexter.conf

As the example below:

metadata {
  path="jdbc:postgresql:x2er"
  user="flex2er"
  password="flexter"
}

Installing/Updating Flexter Metadata Schema

With the license and the flexter metadata access, now it’s possible to load the schema into the database, as the example below:

# For all users
$ flexchma update

# or with parameters
$ flexchma update -m "jdbc:postgresql:[<host>[:port]/]<db>" -M <user> -w <password>

Installing the Apache Spark

In cases where the Apache Spark isn’t available in the system, it can be downloaded using the download.sh module.

It can be explored in detail in the section Flexter Modules / download.sh.

With root access

$ sudo /usr/share/flexter/sbin/download.sh spark

The above script will then check any existing local Spark installation, downloading the default one if no installation is found.

Optionally, you can specify the Spark version. For instance:

$ sudo /usr/share/flexter/sbin/download.sh spark -v 2.4.6 -f

More information about the script’s parameters can be listed with the -h option.

$ sudo /usr/share/flexter/sbin/download.sh spark -h

Without root access

$ ~/.local/share/flexter/sbin/download.sh spark

The above script will then check any existing local Spark installation, downloading the default one if no installation is found.

Optionally, you can specify the Spark version. For instance:

$ ~/.local/share/flexter/sbin/download.sh spark -v 2.4.6 -f

More information about the script’s parameters can be listed with the -h option.

$ ~/.local/share/flexter/sbin/download.sh spark -h

Smoke Tests

Checking Versions

Checking if the commands execute successfully and have the expected versions.

$ flexchma --version
$ xsd2er --version
$ xml2er --version
$ json2er --version
$ merge2er --version

Checking The Help

Checking if the help works correctly and bring the defaults based on the right configuration files.

$ flexchma --help
$ xsd2er --help
$ xml2er --help
$ json2er --help
$ merge2er --help

Testing

With root access

Examples based on the flexter installation with root access

Check if the files are being parsed

$ xsd2er -sRF /usr/share/flexter/samples/donut.xsd.gz
$ xml2er -s /usr/share/flexter/samples/donut.xml.gz
$ json2er -s /usr/share/flexter/samples/donut.json.gz

Check if Flexter is successfully writing metadata

$ xsd2er -RF -g1 /usr/share/flexter/samples/donut.xsd.gz
   origin: <origin_id>

Check if xml2er is loading metadata

$ xml2er -s -x<origin_id> /usr/share/flexter/samples/donut.xml.gz

Check if xml2er is writing the parsed data

$ xml2er -x<origin_id> /usr/share/flexter/samples/donut.xml.gz -o /tmp/flexter_test

Check if merge2er is successfully processing files

$ merge2er /tmp/flexter_test

Check if merge2er is successfully processing files and writing files to a different path

$ merge2er /tmp/flexter_test -o /tmp/flexter_test2