Features

This section details the features of Flexter and how powerful it can be for your use case.

Features of Flexter:

  • Convert XML to a relational Target Schema (XML2ER module).
  • Convert JSON to a relational Target Schema (JSON2ER module).
  • Convert XSD, optionally combined with XML Statistics, to a relational Target Schema (XSD2ER module).
  • Optimisation algorithms to simplify Target Schema.
  • Load text files to a database (MERGE2ER module from Flexter 1.7).
  • Import SQL query results for data conversion (MERGE2ER module from Flexter 1.7).
  • Convert data from one database to another database (MERGE2ER module from Flexter 1.7).
  • Merge small files (MERGE2ER module).
  • Visualise diagrams and schema lineage with the FLEXTER-UI module.

Input

  1. XML/JSON on FTP/SFTP, HTTP/HTTPS, Network folder, Hadoop-FS, Amazon S3, Google Storage
  2. Inside tabular file columns - CSV/TSV/PSV, Parquet, ORC, Avro
  3. Inside database columns - BLOB/VARCHAR/CLOB/XMLTYPE/JSON, Oracle, Postgres, MySQL, MSSQL, Derby, Hive, BigQuery, Snowflake, Teradata, Yellowbrick

Output

  1. Network folder, Hadoop-FS, Amazon S3, Google Storage
  2. CSV/TSV/PSV, Parquet, JSON, ORC, Avro
  3. Oracle, Postgres, MySQL, MSSQL, Derby, Hive, BigQuery, Snowflake, Teradata, Yellowbrick

Automation

  1. Auto-generates the target output schema and mappings to it based on XSD (normalised relational format).
  2. Auto-generates the target output schema and mappings to it based on statistics taken over XML/JSON sample (normalised relational format).
  3. Auto-generates the table relationships of output and primary and foreign keys. Execution engine automatically populates the output schema.

Optimisations

By using various schema optimisation algorithms, Flexter makes the target schema more compact and easier to work with for downstream analytics, e.g. it eliminates duplicate and redundant data points or intelligently re-assigns data points to entities.

Loops or multiple passes over XML/JSON files are eliminated. In most ETL tools developers have to iterate many times over the same XML/JSON file to free the data. With Flexter, each XML/JSON file is loaded once and shredded into its data components.

Performance

Flexter is built on top of Spark, a distributed compute framework. It also scales linearly and can handle any data volume.

Processing can happen in-memory for faster throughput.

Schema and Lineage Browser

Data analysts can use Flexter to visually browse the input and optimised output schemas. They can also browse the lineage and mappings between source and target schemas.