DBT Add-On OCF Connector: Overview

The DBT OCF connector was developed by Alation and is available as a Zip file that can be uploaded and installed in the Alation application.

Create a ticket with Alation Support about receiving the DBT OCF connector package from Alation.

The DBT connector can be used as an add-on to a primary OCF connector to extract and catalog descriptions from DBT target tables, views and columns. Currently, the DBT add-on is available for:

  • Snowflake OCF Connector

  • Redshift OCF Connector

  • PostgreSQL OCF Connector

The DBT add-on is compatible with Alation release 2022.4 and later.

The DBT add-on cannot be used as a stand-alone connector as it works only with primary connectors listed above and shares the user interface of primary connectors.When the DBT add-on is configured for a primary connector, MDE will extract descriptions from DBT along with the metadata from the primary data source.

Team

You may need the assistance of your DBT administrator to configure the DBT add-on

  • Alation Administrator:

    • Installs the connector.

    • Configures the DBT add-on in the primary data source settings.

  • DBT Administrator:

    • Provides the JDBC URI to access metadata in DBT.

    • Provides access to projects to extract metadata.

Scope

The table below shows which metadata objects are extracted by this connector and which operations are supported.

Feature

Scope

Availability

Authentication

GitHub personal access token

Authenticate on the GitHub storage using a personal access token

Yes

Basic (AWS access key and secret key)

Authentication with AWS access key and secret key

Yes

AWS STS authentication

Authentication with an AWS STS access key and secret key

No

Metadata Extraction (MDE)

Default MDE

Extraction of metadata based on default extraction queries in the connector code

Yes

Custom query-based MDE

Extraction of metadata based on extraction queries provided by user

No

Extracted metadata objects

Table descriptions

Description of the DBT target table descriptions

Yes

Views descriptions

Descriptions of the views

Yes

Column descriptions

Description of the columns

Yes

Catalog Features

Sampling and profiling

Not applicable

QLI

Not applicable

Compose

Not applicable

Lineage

Jinja code extraction

Extraction of Jinja code from DBT into dataflow content

Yes

Automatic lineage generation

Auto-calculation of lineage based on query history ingested from QLI and MDE

Yes*

Column-level lineage

Calculation of lineage data at the column level

N/A

* Lineage is supported for Snowflake (Tables, Views, Ephemeral and Incremental) and Redshift (Views) datasources. For incremental models, the user must perform the QLI for the first run of lineage generation. The lineage between the source table and the target table will not be available.