Alation Agent

Alation Cloud Service Applies to Alation Cloud Service instances of Alation

The Alation Agent (or simply the Agent) is optional software you can install on your network to securely connect Alation Cloud Service to your on-premise data sources. After connecting the Agent to data sources that are behind your firewall, you can securely catalog metadata from those data sources to your Alation Cloud instance.

When considering use of the Agent, keep in mind the following:

  • Each Alation Cloud Service instance can support multiple Agents in different geographical locations, network segments, or security zones.

  • Each Agent can support multiple connectors and data sources.

  • The Agent only works with connectors based on the Open Connector Framework (OCF). It doesn’t support native or custom DB connectors.

  • The Agent supports RDBMS, BI, and file system connectors.

  • Newer versions of the Alation Agent now support Compose. See Compose Compatibility below for more details.

This page includes information about:

Agent System Requirements

Alation recommends running the Agent on a dedicated physical or virtual Linux machine with no other software installed. A virtual machine can be set up in a shared server environment as long as the required CPU, RAM, and HDD are allocated for the Agent.

You can install multiple Agents, each on its own machine, and connect them all with Alation Cloud Service. This may be needed if you have data sources in different geographical locations, network segments, or security zones.

Operating System

Alation Agent versions 1.5.0.2541 and later support the following operating systems:

  • Debian based:
    • Debian 9, 10, and 11

    • Ubuntu 16, 18, 20, and 22

  • Red Hat based:
    • AWS Linux 2

    • CentOS 7.x (x86 64-bit)

    • Fedora 33 and 34

    • Oracle Linux 7, 8, and 8.5 (on Red Hat Compatible Kernel)

    • Red Hat 7.x, 8.x, and 9.x (x86 64-bit)

Alation Agents before version 1.5.0.2541 support the following operating systems:

  • Debian based:
    • Debian 9 and 10

    • Ubuntu 16, 18, and 20

  • Red Hat based:
    • AWS Linux 2

    • CentOS 7.x (x86 64-bit)

    • Fedora 33 and 34

    • Oracle Linux 7, 8, and 8.5 (on Red Hat Compatible Kernel)

    • Red Hat 7.x and 8.x (x86 64-bit)

Hardware

The hardware requirements for the Agent depend on how many objects per data source you will be cataloging. Larger data sources require more hardware resources.

The Agent has been certified on the following hardware at the specified scale. For cases with more objects, connectors, or Agents, contact Alation.

Small

Large

Scale

# of objects per data source

5 Million

15 million

# of Agents per Alation instance

5

5

# of connectors per Agent

5

10

System Component Requirements

CPU

2 or more cores

2.5-3.1 GHz

4 or more cores

2.5-3.1 GHz

RAM

8 GB

16 GB

HDD

20 GB

40 GB

The number of Agents per Alation instance may apply if you have data sources in different geographical locations, network segments, or security zones and need to install and connect multiple Agents to your Alation Cloud Service.

Alation Cloud Service Compatibility

In the table below, find the version of Alation you’re currently using. To get the latest Alation Agent features and fixes, we recommend upgrading to the latest version of the Alation Agent that’s compatible with your version of Alation Cloud Service.

Alation provides downloads for the latest two versions of the Alation Agent on the Customer Portal. Older versions of the Agent will become unavailable as newer versions are released.

Alation Agent Version

Compatible Versions of Alation Cloud Service

1.6.1.3465

2023.3.4, 2023.3.5

1.6.1.3288

2023.3.4, 2023.3.5

See the Alation Agent Version History page for a full listing of historical Agent releases and compatible Alation Cloud Service versions.

Checking the Agent Version

On the Agent host machine, check the installed Agent’s version by running:

hydra version

The version number will be in the first line of the output.

Compose Compatibility

Compose Compatibility with the Agent

In order to use Compose with the Alation Agent, you must:

  • Be on Alation Cloud Service version 2022.4 or later.

  • Have Alation Agent version 1.2.1.868 or newer installed. Agent version 1.2.0.815 does not support Compose.

  • Have a supporting version of the relevant connector installed on the Agent. See the documentation for individual OCF connectors to find out if a particular connector can connect to Compose through the Alation Agent.

Architecture

The Agent has three parts:

  • Alation Connector Manager: This Docker container manages all data source connectors installed on the Agent. Management requests include actions like starting, stopping, updating, and deleting connectors.

  • Reverse proxy: This handles communication between your Alation Cloud instance, the Alation Connector Manager, and connectors. The reverse proxy initiates an outbound connection to your Alation Cloud instance using mutual authentication (mTLS). Subsequent two-way communication occurs via this encrypted, persistent tunnel.

  • Connectors: Each data source connector installed on the Agent lives in its own Docker container. Data requests from your Alation Cloud instance are forwarded through the reverse proxy to the relevant connector. From there, the connector communicates with the individual data sources.

../../_images/AgentArchitectureDiagram.png

The Alation Connector Manager, the reverse proxy, and the OCF connectors each reside in their own Docker container. The Docker containers are installed by the Alation Container Service, which is part of the Agent installer.

Security

Alation designed the agent to comply with security policies that only allow outbound connections. It uses mutual TLS and end-to-end encryption to secure communications between the Agent and Alation Cloud Service.

Establishing a Secure Connection

Alation uses digital certificates to provide end-to-end encryption between the Agent and Alation Cloud Service. After installing the Agent software in your network, you’ll generate a certificate signing request (CSR) on the Agent machine. You then upload the certificate signing request to the Alation Cloud Service. Alation Cloud uses the CSR to create a TLS certificate that is used to establish a trusted relationship between the Agent and Alation Cloud Service. You’ll install this TLS certificate on the Agent to finalize the trusted connection.

Alation uses the AWS Certificate Manager (ACM) Private Certificate Authority for generating all Agent certificates. ACM is a highly-available private certificate authority service. Using ACM as the root certificate ensures that only certificates generated from that certificate authority can establish trusted communication with the Alation Cloud Service.

You can renew or revoke the certificate at any time. See Work with the Agent’s Certificates.

Once the required certificate is in place, the Agent will initiate an outbound TLS v1.3 connection to Alation Cloud Service. The Agent and Alation Cloud Service will mutually authenticate.

  • Alation Cloud Service validates that the Agent’s certificate was signed by the ACM Private Certificate Authority.

  • The Agent validates Alation Cloud Service’s certificate authority trust chain, the certificate’s expiration and revocation status, and the ID of your Alation Cloud instance.

Continuing Communication

This TLS connection ensures that all subsequent communication is fully encrypted and allows Alation Cloud Service and the Agent to transfer metadata during metadata extraction and query log ingestion. The connection is persistent, so future queries or extraction requests can be executed immediately.

If network interruptions ever break the connection between the Agent and your Alation Cloud instance, the Agent will attempt to reconnect. It keeps trying to connect using an exponential backoff algorithm. Once the Agent can connect to your Alation Cloud instance again, it will reauthenticate and reestablish a secure connection.

Any jobs, such as metadata extraction, that were underway will automatically restart as long as the connection is reestablished within 30 seconds. If it takes longer than that, you’ll have to restart the job manually.

Further Reading

Explore the following topics for help with: