Alation Agent

The Alation Agent (or simply the Agent) is optional software you can install on your network to securely connect Alation Cloud Service to your on-premise data sources. After connecting the Agent to data sources that are behind your firewall, you can securely catalog metadata from those data sources to your Alation Cloud instance.

When considering use of the Agent, keep in mind the following:

  • The Agent only works with connectors based on the Open Connector Framework (OCF).

  • The Agent only supports RDBMS and BI connectors.

  • The Agent does not currently support Compose.

  • Each Alation Cloud Service instance can support multiple Agents.

  • Each Agent can support multiple connectors and data sources.

See below for information about:

Explore the following topics for help with:

Agent System Requirements

Alation recommends running the Agent on a dedicated physical or virtual Linux machine with no other software installed. A virtual machine can be set up in a shared server environment as long as the required CPU, RAM, and HDD are allocated for the Agent.

Operating System

The Agent supports the following operating systems.

  • Debian based:
    • Debian 9 or 10

    • Ubuntu 16, 18, 20

  • Red Hat based:
    • AWS Linux 2

    • CentOS 7.x (x86 64-bit)

    • Fedora 33, 34

    • Oracle Linux 7, 8, 8.5 (on Red Hat Compatible Kernel)

    • Red Hat 7.x or 8.x (x86 64-bit)

Hardware

The hardware requirements for the Agent depend on how many objects per data source you will be cataloging. Larger data sources require more hardware resources. The Agent has been certified on the following hardware at the specified scale.

Small

Large

Scale

# of objects per data source

5 Million

15 million

# of Agents

5

5

# of connectors per Agent

5

10

System Component Requirements

CPU

2 or more cores

4 or more cores

RAM

8 GB

16 GB

HDD

20 GB

40 GB

For cases with more objects, connectors, or Agents, contact Alation.

Alation Cloud Service Compatibility

Alation Cloud Version

Compatible Alation Agent Versions

2022.3

1.2.1.868, 1.2.0.815

2022.2

1.2.0.815

On the Agent host machine, check the installed Agent’s version by running:

hydra version

The version number will be in the first line of the output.

Architecture

The Agent has three parts:

  • Alation Connector Manager: This Docker container manages all data source connectors installed on the Agent. Management requests include actions like starting, stopping, updating, and deleting connectors.

  • Reverse proxy: This handles communication between your Alation Cloud instance, the Alation Connector Manager, and connectors. The reverse proxy initiates an outbound connection to your Alation Cloud instance using mutual authentication (mTLS). Subsequent two-way communication occurs via this encrypted, persistent tunnel.

  • Connectors: Each data source connector installed on the Agent lives in its own Docker container. Data requests from your Alation Cloud instance are forwarded through the reverse proxy to the relevant connector. From there, the connector communicates with the individual data sources.

../../_images/AgentArchitectureDiagram.png

The Alation Connector Manager, the reverse proxy, and the OCF connectors each reside in their own Docker container. The Docker containers are installed by the Alation Container Service, which is part of the Agent installer.

Security

Alation designed the agent to comply with security policies that only allow outbound connections. It uses mutual TLS and end-to-end encryption to secure communications between the Agent and Alation Cloud Service.

Establishing a Secure Connection

Alation uses digital certificates to provide end-to-end encryption between the Agent and Alation Cloud Service. After installing the Agent software in your network, you’ll generate a certificate signing request (CSR) on the Agent machine. You then upload the certificate signing request to the Alation Cloud Service. Alation Cloud uses the CSR to create a TLS certificate that is used to establish a trusted relationship between the Agent and Alation Cloud Service. You’ll install this TLS certificate on the Agent to finalize the trusted connection.

Alation uses the AWS Certificate Manager (ACM) Private Certificate Authority for generating all Agent certificates. ACM is a highly-available private certificate authority service. Using ACM as the root certificate ensures that only certificates generated from that certificate authority can establish trusted communication with the Alation Cloud Service.

You can renew or revoke the certificate at any time. See Work with the Agent’s Certificates.

Once the required certificate is in place, the Agent will initiate an outbound TLS v1.3 connection to Alation Cloud Service. The Agent and Alation Cloud Service will mutually authenticate.

  • Alation Cloud Service validates that the Agent’s certificate was signed by the ACM Private Certificate Authority.

  • The Agent validates Alation Cloud Service’s certificate authority trust chain, the certificate’s expiration and revocation status, and the ID of your Alation Cloud instance.

Continuing Communication

This TLS connection ensures that all subsequent communication is fully encrypted and allows Alation Cloud Service and the Agent to transfer metadata during metadata extraction and query log ingestion. The connection is persistent, so future queries or extraction requests can be executed immediately.

If network interruptions ever break the connection between the Agent and your Alation Cloud instance, the Agent will attempt to reconnect. It keeps trying to connect using an exponential backoff algorithm. Once the Agent can connect to your Alation Cloud instance again, it will reauthenticate and reestablish a secure connection.

Any jobs, such as metadata extraction, that were underway will automatically restart as long as the connection is reestablished within 30 seconds. If it takes longer than that, you’ll have to restart the job manually.