Overview

Alation Cloud Service Applies to Alation Cloud Service instances of Alation

Customer Managed Applies to customer-managed instances of Alation

The Databricks on Google Cloud OCF connector is developed by Alation.

To download the Databricks on Google Cloud OCF connector package, go to the Alation Connector Hub available from the Customer Portal. Go to Customer Portal > Connectors > Alation Connector Hub. Only Alation users with access to the Customer Portal can access the Alation Connector Hub. If you don’t have access to the Customer Portal, contact Alation Support.

This connector should be used to catalog Databricks on Google Cloud as a data source in Alation. The connector catalogs Databricks on Google Cloud OCF objects such as schemas, tables, columns, and views. It enables end users to discover, search, browse, and curate Databricks on Google Cloud objects from the Alation user interface.

Team

The following administrators are required to install this connector:

  • Alation administrator

    • Installs the connector

    • Creates and configures the Databricks on Google Cloud data source in the catalog.

  • Databricks on Google Cloud administrator

    • Creates a service account with the Enterprise authentication type for Alation

    • Provides the JDBC URI to access metadata

    • Provides access to public schemas to extract metadata

    • Creates a compute resource for metadata extraction
      • Creates a compute (Cluster or SQL Warehouse)

Scope

The table below lists the supported features of a Databricks on Google Cloud OCF connector.

Feature

Scope

Availability

Authentication

Token-based authentication

Authentication using Databricks personal access tokens

Yes

SSL authentication

SSL Authentication

No

Kerberos

Authentication with Kerberos

No

Keytab

Authentication with Keytab

No

LDAP

Authentication with the LDAP protocol

No

SSO

Authentication with SSO

No

Metadata Extraction (MDE)

Default MDE

Extracts supported metadata objects based on Databricks JDBC driver methods in the connector code

Yes

Custom query-based MDE

Extracts supported metadata objects based on extraction queries provided by user

No

Popularity

Indicator of the popularity (intensity of use) of a data object, such as a table or a column

Yes

Extracted metadata objects

Data Source

Data source object in Alation that is parent to the extracted metadata

Yes

Schemas

List of schemas

Yes

Catalog

List of catalogs

No

Tables

List of tables

Yes

Columns

List of columns

Yes

Column data types

Column data types

Yes

Views

List of views

Yes

Schema source comments

Schema source comments

Yes

Column source comments

Column source comments

Yes

Table source comments

Table source comments

No

Primary keys

Primary key information for extracted tables

No

Foreign keys

Foreign key information for extracted tables

No

Functions

Extract function metadata

No

Function definitions

Extract function definition metadata

No

Sampling and Profiling

Table sampling

Extracts data samples from all extracted tables

Yes

Column sampling

Extracts data samples from all extracted columns

Yes

Deep column profiling

On-demand profiling of specific columns with the calculation of value distribution stats

Yes

Dynamic profiling

On-demand table and column profiling by individual users who use their own database accounts to retrieve the profiles

Yes

Custom query-based table sampling

Ability to use custom queries for sampling specific tables

Yes

Custom query-based column sampling

Ability to use custom queries for profiling specific columns

Yes

Query Log Ingestion (QLI) - Not applicable

Lineage

Automatic lineage generation

Auto-calculation of lineage based on query history ingested from QLI, MDE, and Compose queries

Yes

Compose

Customer-managed (on-prem) Alation instances

Compose on on-prem Alation instances

Yes

Alation Cloud Service instances

Depending on your network configuration, you may be using Alation Agent to connect to your data source.

Compose via Agent is supported from connector version 2.1.0.4607.

Yes

Personal Access Token (PAT) authentication in Compose

Authentication in Compose with username and password

Yes

SSO authentication in Compose

Authentication in compose with SSO credentials

No