Databricks Unity Catalog OCF Connector Release Notes¶
Alation Cloud Service Applies to Alation Cloud Service instances of Alation
Customer Managed Applies to customer-managed instances of Alation
April 24, 2024¶
Databricks Unity Catalog OCF Connector: Version 2.2.2¶
March 27, 2024¶
Databricks Unity Catalog OCF Connector: Version 2.2.1¶
Fixed Issues¶
Bulk Sampling Fails for Column Data Types timestamp_NTZ, Void, and Interval¶
For example, for Databricks Unity Catalog data source, sampling fails with the following error for column data type timestamp_NTZ:
The connector was provided with a list of columns to profile from a table, however the schema of the returned result set did not match those columns.
Now, connector allows bulk sampling to proceed by skipping tables with timestamp_ntz, void, and interval issues. (AL-150864)
January 17, 2024¶
Databricks Unity Catalog OCF Connector: Version 2.2.0¶
Compatible Alation Version - 2023.1.7 or higher
Enhancements¶
With Alation version 2023.3.4 and Databricks Unity Catalog OCF connector version 2.2.0, Alation has introduced an enhanced user interface for configuring the Databricks Unity Catalog data source settings.
Note
The connector version is compatible with Alation versions prior to 2023.3.4. However, you will view the old user interface with Alation versions prior to 2023.3.4.
The documentation for the new Databricks Unity Catalog experience is available at: Databricks Unity Catalog OCF Connector
The enhanced user interface provides the following features:
Better User Experience¶
The enhanced Databricks Unity Catalog user interface provides better navigation and multiple visual cues, making it easy to configure and manage Databricks Unity Catalog data source.
Easy to Connect¶
The Databricks Unity Catalog user interface now provides a step-by-step process to connect to your Databricks Unity Catalog data source easily and quickly.
Additional ErrorPrevention Checks¶
The Databricks Unity Catalog user interface now includes added checks to avoid obvious configuration mistakes. The user interface also includes text to assist and provide cues while configuring.
Detailed Error Reports and Troubleshooting Support¶
The improved Databricks Unity Catalog user interface makes troubleshooting easier for metadata extraction and query log ingestion, as you can now have a detailed error report for both processes. The report contains a detailed error description and steps to resolve the issue.
December 20, 2023¶
Databricks Unity Catalog OCF Connector: Version 2.1.1¶
Fixed Issues¶
QLI Execution Fails for Databricks Unity Catalog OCF Connector¶
The issue related to QLI execution for Databricks Unity Catalog data source with ANSI Mode enabled that resulted in the CAST_INVALID_INPUT error is now fixed.
November 29, 2023¶
Databricks Unity Catalog OCF Connector: Version 2.1.0¶
Compatible Alation Version - 2023.1.7.1 or higher
Enhancements¶
Support for OAuth Authentication with Azure Active Directory¶
The Databricks Unity Catalog OCF connector now supports OAuth authentication for Compose, enabling you to securely access and utilize data assets within the Databricks environment.
Previous Versions¶
Connector version |
Compatible with Alation from version |
New features in this version |
|---|---|---|
2.0.3.6564 |
2023.1.7.1 |
Disabled automatic lineage generation from queries ingested during query log ingestion (QLI) for this connector. The Data Source admins no longer need to manually disable automatic lineage on the data source settings page. Lineage is calculated based on direct lineage extraction from system tables. |
2.0.2.6259 |
2023.1.7.1 |
Implemented query log ingestion (beta). |
1.2.1.5335 |
2023.1.4 |
Implemented Query Service. |
1.1.0.4393 |
2023.1.4 |
Added support for incremental extraction of direct lineage (beta). |
1.0.3.4144 |
2023.1.2 |
Implemented direct lineage extraction (beta). |
Note
Direct lineage extraction and query log ingestion are considered beta as they rely on the Unity Catalog system tables that are currently in Public Preview in Databricks and require separate access enablement. You may need assistance from your Databricks account admin to enable access to these features in Databricks.