Azure Synapse Analytics OCF Connector: Overview

The OCF Connector for Azure Synapse Analytics was developed by Alation and is available as a Zip file that can be uploaded and installed in the Alation application. The connector is compiled together with the required database driver, so no additional effort is needed to procure and install the driver.

Contact your Customer Success Manager about receiving the Azure Synapse OCF connector package from Alation.

This connector should be used to catalog data resources under Azure Synapse Analytics as data sources in Alation. The connector catalogs such Azure Synapse Analytics objects as SQL pools, databases, schemas, tables, columns, views, and functions. After the metadata is extracted, it is represented in the data catalog as a hierarchy of catalog pages under the parent data source. Alation users can leverage the full catalog functionality to search for and find the extracted metadata, curate the corresponding catalog pages, create documentation about the data source, and exchange information about it.

Team

You may need the assistance of your DBA to configure this data source.

  • Azure Synapse Administrator

    • Creates a service account for Alation

    • Provides a JDBC URI to access metadata

    • Provides access to data resources to extract metadata

    • Enables auditing for QLI

  • Alation Administrator

    • Installs the connector

    • Creates and configures an Azure Synapse Analytics data source in the catalog

Scope

The table below shows which metadata objects are extracted by the connector and which operations are supported.

Feature

Scope

Availability

Authentication

Basic authentication

Authentication with a service account created on the database that uses a username and password

Yes

Azure AD

Authentication with an Azure AD user account (Azure username and password)

Yes

Metadata extraction (MDE)

Default MDE

Extraction of metadata based on default extraction queries in the connector code

Yes

Custom query-based MDE

Extraction of metadata based on extraction queries provided by a user

Yes

Extracted metadata objects

Data Source

Data source object in Alation that is parent to extracted metadata

Yes

Schemas

List of schemas

Yes

Tables

List of tables

Yes

Columns

List of columns

Yes

Column data types

Column data types

Yes

Views

List of views

Yes

Source comments

Source comments

No

Primary keys

Primary key information for extracted tables

No

Foreign keys

Foreign key information for extracted tables

No

Functions

Function metadata

Yes

Function definitions

Function definition metadata

Yes

Sampling and Profiling

Table sampling

Retrieval of data samples from extracted tables

Yes

Column sampling

Retrieval of data samples from extracted columns

Yes

Deep column profiling

Profiling of columns with the calculation of value distribution stats

Yes

Dynamic profiling

Ability for individual users to connect with their own database accounts to retrieve table and column samples and profiles

Yes

Custom query-based table sampling

Ability to use custom queries for sampling specific tables

Yes

Custom query-based column sampling

Ability to use custom queries for profiling specific columns

Yes

Query log ingestion (QLI)

Table-based QLI

Ingestion of query history based on a table or view that contains query history data

Yes

Query-based QLI

Ingestion of query history based on a custom query history extraction query

Yes

JOINs and filters

Calculation of JOIN and filter information based on ingested query history

Yes

Predicates

Ability to parse predicates in ingested queries

Yes

Lineage

Automatic lineage generation

Auto-calculation of lineage based on query history ingested from QLI, MDE, and Compose queries

For lineage to be generated for views, Alation expects fully qualified table names in view definitions: <database_name>.<schema_name>.<table_name>

Yes

Compose

On-prem Alation instances

Compose on on-prem Alation instances

Yes

Alation Cloud Service instances

Connection to the data source does not go through Alation Agent

Yes

Basic authentication in Compose

Authentication in Compose with a username and password

Yes

SSO authentication in Compose

Authentication in Compose with SSO credentials

No