Direct Installation

The direct installation refers to downloading and installing packages directly in your own server(s)

Requirements

  • OS: RedHat or compatible (CentOS, Oracle Linux, Fedora), Debian or compatible (Ubuntu)
  • Java: JDK 8/11 (OpenJDK, Oracle, Temurin or other JDK compatible distributions)
  • Scala: 2.12
  • Memory: 16GB+
  • Metadata DB: PostgreSQL 12+
  • Spark: 2.1.x to 2.4.x (for local mode it can be installed with the download.sh module)
  • YARN+HDFS: Hadoop 2.7+
  • Hadoop Tools: Hive: 2.1+, Impala
  • Ports open:
    • Flexter metadata - ports to a PostgreSQL instance storing Flexter’s metastore. Not required if installed on the same machine as the Flexter processor. Usually 5432 (default).
    • Flexter UI
      • backend (default: 8081)
      • frontend (default: 9091)
    • Spark
      • Master - Web UI and job submissions (defaults: 8080, 7077)
      • Worker - WebUI (defaults: 7071)
      • Application WebUI/driver (default: 4040 and subsequent if multiple)
      • History Server (default: 18080)
    • HDFS (as per existing cluster configuration)
      • master node(s) (default: 8020/9000)
      • worker node(s) (default: 50010)
    • Hive (if used)
      • metastore (as per existing metastore configuration)
Download/Install

The Flexter download process.

Database Setup

The flexter database setup based on Official PostgreSQL documentation

Post-Installation

License, metadata database access, metadata schema, Apache Spark, smoke tests and Flexter UI setup.

Upgrade

The Flexter upgrade process.