Release Notes 2023.1 (General Availability)¶

RELEASE 2023.1.2¶

build 15.3.2.32690

Alation Catalog and Compose Improvements¶

SQL literals can now be obfuscated for all view objects in the catalog. To enable this feature, set the alation_conf flag alation.feature_flags.enable_obfuscating_view_sql to True. This will mask the view SQL query on the catalog pages where it is displayed for all views. See Obfuscate Literals in View SQL for more details.
Added a new Compose setting Always show column names which is turned on by default. With Always show column names enabled, the column headers of query results tables will show physical column names even when columns also have curated titles in Alation. With Always show column names disabled, the column headers of query results tables will show titles or physical column names depending on whether a column has a title. Titles will be shown for columns that have them and physical column names will be shown for columns without titles.

Alation API¶

Added support for Catalog Set (dynamic_set_property) and Catalog Set Shared Fields (propagated_catalog_set) in Supported Object Types in the Public Custom Field Values API.

Azure Blob Storage OCF Connector¶

New connector version available: 2.2.0.4373

Fixed an issue where extraction failed with an “unexpected error” when information for some fields that were optional for extraction was missing in the extracted files. Now, if object properties that are optional for extraction, for example, size or modified time, are missing, MDE will successfully complete without ingesting these object properties.
The rule name parameter is now a mandatory field for extraction. Extraction will fail in case of an empty rule name or empty destination container, or both, with the corresponding message in the connector log.
Added an option to exclude individual files during schema extraction (applicable if Schema Path Pattern is enabled).

Azure Synapse OCF Connector¶

New connector version available: 1.1.0.4029

Added ability to extract metadata from serverless SQL pools.

Databricks Unity Catalog OCF Connector¶

New connector version available: 1.0.3.4144

Added support for lineage generation.

dbt OCF Connector¶

New connector version available: 2.1.0.4367

Added support for the Google BigQuery data source.
Added support for the Databricks on AWS, Databricks on GCP, and Azure Databricks data sources.

EMR Presto OCF Connector¶

New connector version available: 1.0.4.4060

Added support for extracting Hive Delta tables.

Impala on CDP OCF Connector¶

New connector version available: 1.0.3.4132

Fixed an issue with Kerberos with a keytab connection.
Fixed an issue with SSL-enabled connection.
Fixed an issue with extraction where it failed if external tables pointed to the HDFS file path with spaces in the file name.
Added support for extracting Kudu tables.

Power BI OCF Connector (Non-Scanner)¶

New connector version available: 3.0.0.4371

Fixed a number of metadata extraction issues.

Redshift OCF Connector¶

New connector version available: 1.2.0.4216

Added support for EC2 instance profile-based authentication.

S3 OCF Connector¶

New connector version available: 3.5.0.4216

Added support for extraction from the PSV and TSV files.
Fixed an issue where extraction failed with an unexpected error when information for some fields that were optional to extraction was missing in the extracted files. Now, if object properties that are optional to extraction, for example, size or modified time, are missing, metadata extraction will successfully complete without ingesting these object properties.
Added an option to exclude individual files during schema extraction (applicable if Schema Path Pattern is enabled).

SAP BW OCF Connector¶

New connector version available: 1.0.9.4297

Fixed a number of extraction issues.
Added support for schema prefix extraction. When we extract from JSON files with a schema prefix, metadata extraction now completes successfully and all columns are extracted. Previously, column names were not extracted from JSON files with a schema prefix.

Snowflake OCF Connector¶

New connector version available: 1.1.1.4303

Added ability to reuse the QLI view created for the native Snowflake data source.
Added support for Snowflake key pair authentication with the public and private key generated using the aes-256-cbc encryption.

Tableau OCF Connector¶

New connector version available: 1.5.0.4134

Fixed a number of metadata extraction issues, including extraction failing due to an out of memory exception and other errors.
Added ability to successfully extract metadata from a Tableau BI source when embedded data sources have a very large number of upstream columns.
Updated the connector’s JRE image.
Fixed issues with metadata upload when the data sources are Google BigQuery and MySQL.
Fixed an exception-handling bug where the connector froze without erroring out when an API request failed. The connector will now log a clear error message and skip the object that caused the error.
Updated the Tableau BI source settings user interface. The fields required for Personal Access Token authentication now appear on top of the settings page under Connector Settings > Server Connection.
Added ability to extract the owner information for workbooks, dashboards, reports, embedded data sources, and published data sources.

Connected Sheets¶

Added ability to edit the dataset configuration including columns and filter criteria at any time. Available in both Google and Excel add-ins.
Added the trust flags information to data objects downloaded from an add-in. You will see the relevant trust flag descriptions in the add-in with a link to view more details on the catalog page.
Connected Sheets can now work with an on-premise PostgreSQL data source connected with Alation Agent.

Bug Fixes¶

Resolved an issue where the HA pair rebuilding failed after the upgrade to version 2023.1 with cluster splitting due to a bug in the replication service. Now, the admin performing the upgrade of an HA pair with cluster splitting should be able to successfully complete it.
Fixed a bug in Compose where large query executions on a Redshift data source were getting stuck. The issue was due to a race condition on the server created by one of the internal API calls. Now, users should be able to successfully execute large queries on Redshift.
Fixed a bug that caused an error when accessing the Policy Center page on some Alation instances. The issue was due to the API response for the “/all_policies” GET request containing unexpected null values for the Stewards field on some of the policy objects. Fixed by adding logic to the frontend to properly handle unexpected values.
Fixed a bug where the QLI archiving job was set to run once a year in the internal database even though users had previously added a different schedule in the settings of a data source. The QLI archiving job will now respect schedules set by users.
Fixed an issue with the Compound Layout lineage view where the More nodes indicator disappeared from the graph after adding a node from the Search for more objects dialog. Now, when there are collection nodes present for the graph, the down arrow indicator will also be present.
Fixed an issue where the Show Parent and Show Children actions did not have an effect after a user attempted to expand a lineage node on a large Lineage diagram.
Fixed an issue where the Show Parent and Show Children actions only displayed a part of the available lineage nodes on large Lineage diagrams.
Fixed a number of user interface issues to stabilize and improve the Lineage Asset Grouping feature.
Added a warning to the Delete BI server confirmation dialog that deleting large BI sources may cause issues in the Alation user interface. Deleting a large BI source may cause a 500 error on the Alation server. We recommend contacting Alation Support for help with deleting such BI sources until this issue is resolved. Note that BI sources with a small number of objects or with no objects can be safely deleted from the user interface.
Fixed an issue where the lineage generation job was stuck in the Running status even though the corresponding metadata extraction job was completed and assigned the status Succeeded. The job history will now reflect the correct status of all jobs subordinate to metadata extraction.
Fixed the time zone discrepancy in the Next Run column in Admin Settings > Monitor > Scheduled Query Dashboard. Now the Next Run time will be shown in an admin’s local time zone.
Fixed an issue where Alation failed to start after remounting the data volumes from the old to the new Alation instance. The problem was due to a permission issue when trying to restart a process. Fixed by updating the directory permission.

Known Issues¶

The Alation update of an HA pair without splitting the cluster will fail. We recommend upgrading HA instances using the cluster splitting upgrade method until this issue is resolved.
Deleting a large BI source may cause a 500 error on the Alation server. We recommend contacting Alation Support for help with deleting such BI sources until this issue is resolved. Note that BI sources with a small number of objects or with no objects can be safely deleted from the user interface.

RELEASE 2023.1.1¶

build 15.3.1.30768

Fixed an issue where tag values on multiple Snowflake data sources cataloged using the Snowflake OCF connector were reset to Not set after metadata extraction on one of them. The fix ensures that after metadata extraction on one of the existing Snowflake OCF data sources, the tag field values will not change on other Snowflake OCF data sources.
Fixed an issue for an Oracle data source where the Compose connection kept timing out with the error Could not start query. Check your connection and try again in the UI and the Connection closed error in the log. Added a configuration option to enable a check that the Oracle database connection is open before running a query. To enable, append -DcheckOracleConnIsOpen=true to the alation_conf parameter connector.extra_flags.
Note

Do not override existing values. If the parameter connector.extra_flags already has some values set, add your new value:
alation_conf connector.extra_flags -s ' -Darg1=value1 -Darg2=value2 -DcheckOracleConnIsOpen=true'
On how to use alation_conf, see Using alation_conf.
Fixed an issue where users were unable to see the table and column object lists on the Curate > Assign Stewards page. The issue was due to the internal APIs used for curation dashboards performing multiple queries to the database in an inefficient manner, which resulted in the 504 error. Added a fix where for every given batch of 1000 curation object records, all related records will get pre-fetched using a single bulk query.
Queries ingested with QLI with the only change in the comments will no longer create duplicate dataflow objects.
Fixed a performance issue with Multi-Select Picker custom fields. With multiple populated Multi-Select Picker custom fields on a catalog page, multiple duplicate calls to the internal API were happening to refresh the Multi-Select Picker field data. These extra calls were redundant and caused performance slowness. Fixed by updating the conditional check on Multi-Select Pickers to only fetch data when the selected values have changed.
The health check API http(s)://<your_alation.com>/monitor/i_am_alive now has a configurable timeout threshold. The the health status check of the lineage service will be terminated if the threshold is reached. The threshold is controlled by the alation_conf parameter lineage-service.health_check.timeout, set in seconds. Previously, this threshold could not be configured. When it was reached, the i_am_alive API response or the corresponding email alert indicated that the lineage service health check failed but did not contain any specific exception. Now, the API response and the email alert will include a message that the request timed out based on the health check timeout threshold.
Previously, Alation users could run into a bug where the column profile histogram on the Sample tab of the column catalog page only added new counts and values but never removed values that had been removed from the underlying database. This issue has now been addressed and column samples should correctly reflect the removal of a value during any corresponding profiling request (scheduled table profiling or manual column profiling).
Fixed a SAML authentication issue on Alation Cloud Service instances where the login failed for new users with the error RelatedObjectDoesNotExist: User has no profile. After the fix, Alation will process SAML responses with empty attributes lists correctly. For example, if an empty list is received from the IdP for an attribute like Title, then an empty string will be saved instead of breaking the login flow.
Fixed a SAML authentication issue on Alation Cloud Service instances where active users were unable to log in or suspended users could not be activated if the case in their username had changed in the IdP. Now, a username can be patched in IAM using the PATCH endpoints. If there is any change in the username in the IdP, the username will be sync’ed to IAM as well.

RELEASE 2023.1.0 - General Availability¶

build 15.3.0.29706

Governance App¶

Add Object Request Workflow for Glossary Terms¶

This new workflow aims at governing the creation of terms in Alation. It allows you to set up a review process for any newly created glossary term before it is added to a glossary. For any new term under a workflow, there are two modes:

DRAFT—A private mode for terms in a workflow. It allows the creator to work on the term in a private space and invite other users to collaborate. The term is not accessible to users unless they are granted access by the creator or are Catalog or Server Admins.

UNDER REVIEW—Review mode. Once the creator has finished drafting the term, they can submit it for review. This initiates the review process, and the term enters the Under Review mode. The term is available for review by a defined set of reviewers, which includes the creator and collaborators. The term remains in the Under Review mode until it is reviewed and approved. Once it is approved, the term goes live and can be viewed by all users.

All workflow participants get real-time email notifications about the changes and reviewers get a rich experience enabling them to act on terms pending review.

Compose¶

Granular Access Permissions for Queries¶

Previously, Alation didn’t have the capability to set granular access permissions for query objects that would allow only certain users to view, edit, or run them. Starting with this release, you can use the new Sharing & Access feature to configure access to queries in Compose and the catalog, thus controlling who can view, edit, and run them. Levels of access supported for queries are:

No access

View Access

View & Run Access

View, Run, & Edit Access

Owner.

The access rules are set on individual query objects and apply to both unpublished and published queries. For new queries, the default is View & Run Access for everybody.

If you are using the alation_conf parameter alation.catalog.unpublished_query_visibility_level to control visibility of unpublished queries, note that this setting will take precedence over individual access permissions on unpublished query objects. If the parameter alation.catalog.unpublished_query_visibility_level is currently set to not_visible (unpublished queries can only be accessed by Owners and users with whom they were explicitly shared), then after enabling the Sharing & Access feature, unpublished queries will remain invisible to users even if they are given View Access in the user interface. We do not recommend changing the parameter alation.catalog.unpublished_query_visibility_level to visible before all protected unpublished queries have been assigned the correct access privileges.

Scheduled Query Dashboard for Admins¶

The Scheduled Query Dashboard in Admin Settings > Monitor empowers Server Admins to manage scheduled queries across all users and data sources in an Alation instance:

All scheduled queries at-a-glance

Tabular view of all scheduled queries with the following details: Query Title, Data Source, Query Owner, Owner Status, Schedule, Next Run Time, Status, and Actions.

Actions on specific scheduled queries

Change Schedule Settings

Modify schedule settings to spread query loads over time, improving overall system performance.

Change Permission

Grant or revoke View, Run, and Edit permissions to or from individuals or groups in case the query owner is not available.

Change the owner of a query to avoid “ghost” scheduled queries.

Cancel Next Run

Cancel the current execution or the next pending execution of a scheduled query to save resources.

Disable Schedule

Disable a query from running on a schedule, so that scheduled queries with owners who are no longer employed at the company or queries that are no longer needed don’t keep running forever.

Ability to Use Hive Variables in Compose¶

Previously, since Hive variables were in conflict with Query Form variables due to their syntax, they could not be used in Compose. With this enhancement, Compose users can use an escape character (${\hivevar:tablename}) to resolve syntax conflicts between Query Form variables and Hive variables, enabling the use of Hive variables in Compose queries.

Lineage¶

Compound Layout View¶

Alation now provides the ability to represent lineage objects based on their logical alignment with the data source and dataflow objects: the Compound Layout View of the lineage data. The view supports the Show Parent/Show Children functionality which will display additional upstream and downstream lineage nodes that are not directly linked to the main object.

This feature is not enabled by default. To make it available on your Alation instance, set the following alation_conf flags to True:

alation.feature_flags.DEV_enable_lineage_asset_grouping—Enables the Compound Layout View toggle on the Lineage tab.

alation.feature_flags.DEV_enable_lineage_asset_grouping—Enables the three-dot menu on lineage nodes on the Compound Layout View.

alation.feature_flags.DEV_enable_lineage_collection_nodes_modal—Adds the Show Parent/Show Children functionality.

If a data source supports DBT connections, users will see the DBT dataflow objects grouped together.

Compound Layout View currently doesn’t support column-level lineage.

Lineage Default View Configuration—No Restart Required¶

Previously, after the lineage settings were modified in Admin Settings > Customize Catalog > Lineage, the lineage service had to be restarted. Starting in 2023.1, the restart is no longer required.

Impact Analysis Performance Improvements¶

We implemented a number of changes to improve the general performance of the Impact Analysis reports:

Reports now use lazy loading to load faster.

Temp objects are excluded by default. Previously, the configuration parameter lineage-service.filters.exclude_temp was applied only to the graph view. Now, it’s applied to both the graph and the Impact Analysis reports.

Catalog¶

Domain and Term Objects Now Respect Object-Level Permissions¶

On the catalog page for a domain, if a user doesn’t have View permission for a specific object, they will not see this object in the Browse this Domain list.
Users can no longer apply changes to objects they don’t have Edit permission for when adding an object to a domain.
The Domains field on the catalog pages of objects will now respect object-level permissions. Only users with permissions to edit an object will be able to modify its domain membership.
If a user does not have Edit permission for a parent glossary, they will not be able to apply membership changes to child terms. A user will need View or Edit access to a term object and Edit access to a glossary to add or remove a term to or from a glossary.
If a term object is added to more than one glossary and is configured to inherit permissions from all of them, a user will need access to all glossaries to have access to this term.
A user will be able to see and/or update the template of a term object based on their object-level permissions for this term.

Data Quality Indicators Added in Search Results¶

Alation Search users will be shown an icon indicating a Data Quality warning that affects objects in the search results. This icon represents quality issues affecting the object or upstream objects.

Quality Propagation of Downstream Data Objects—General Availability¶

Alation’s Data Quality service now includes quality values generated from ingestion from OCF data sources. On detection of deleted data objects, Data Quality creates a deleted quality value on that object which then propagates to the downstream nodes on the lineage graph. This information will be displayed for downstream tables, BI reports, and BI datasources and is surfaced on the new Data Health > Upstream Issues tab for each respective catalog page.

This functionality, introduced in the previous release as Beta, is now General Availability.

Search Results Span Title and Rich Text Fields¶

The Search function now enables search results that span the title as well as rich text fields. For example, an article with “new york city” in the title and “local park” in the body can be found by searching for “new york park”.

Search Indexing Improvements¶

Implemented the force merge API for Elasticsearch for better search index storage management. Force merge executes every Sunday.
Completed a number of modifications to reduce the search index size, including setting the current default stored values for fields to False so that Elasticsearch saves source data only.

Alation Analytics App¶

Supportability Improvements¶

Added functionality to improve the ETL process and failure tracing for Alation Analytics. Implemented an error notification framework in order to communicate ETL errors. Server Admins will now receive email notifications if the ETL process ends in an error, reducing the time required for them to become aware of an issue.

The health status of Alation Analytics Postgres and RabbitMQ containers can now be monitored from the user interface in Admin Settings > Monitor > Health Check. Note that on the instances on Cloud Native Architecture, only the RabbitMQ health is displayed, as Postgres is a managed RDS service. Failure notifications are sent as an email to Server Admins.

Data Model Additions¶

Added fields file_url, directory _url, and filesystem_url to tables files, directories, and filesystems respectively in Alation Analytics to improve consistency with columns available for other data objects like datasources, schemas, tables, and columns.

Added a new boolean column private to the rdbms_tables table in Alation Analytics to indicate whether the table is private or public.

Alation APIs¶

New Conversations API¶

Added a new API for working with Conversations, introducing endpoints for getting, creating, updating, and deleting Conversations and Conversation replies (posts).

Updates to OCF Data Sources API¶

Added new endpoints to the OCF Data Sources API:

API now allows users to update OCF data source information using the PUT endpoint /integration/v2/datasource/{datasource_id}/.

Added datasource config checks group of endpoints that allow performing connection tests that are equivalent to the Test Connection functionality in data source settings in the user interface:

/integration/v2/datasource/{datasource_id}/configuration_check/—Check all configurations of the data source.

/integration/v2/datasource/{datasource_id}/configuration_check/service_account_authentication/—Check the service account authentication status of the data source.

/integration/v2/datasource/{datasource_id}/configuration_check/service_account_privileges/—Check the service account privileges of the data source.

/integration/v2/datasource/{id}/configuration_check/network_connection/—Check if the OCF connector is responsive. Returns response 200 if responsive and response 400 when a gRPC timeout occurs.

Updates to User V2 API¶

Enhanced the GET methods to return the date the user was created and their last login date. We also added new query parameters to enable filtering on the user’s creation date and last login date. This will enable Alation program owners to drive adoption of the catalog by programmatically identifying users who haven’t logged in for a given time. You can then send users a message encouraging them to take advantage of Alation.

Updates to Virtual File Systems API¶

Fixed a bug where POST calls were failing if the ts_last_accessed parameter was missing from the request. This has been fixed so that ts_last_accessed is no longer required for uploading virtual file systems.

Alation Integrations¶

Connected Sheets¶

Connected Sheets for Microsoft Excel¶

The Connected Sheets for Microsoft Excel is a new add-on that helps users discover data sources in the Alation catalog, download data using filter & sort criteria, and refresh on demand. The Connected Sheets page, accessible from the Apps menu in Alation, will have the MS Excel add-on installation link.

This feature is available on the Alation Cloud Service with cloud-native architecture. Users must have an Explorer or Creator license to use the add-on.

Connected Sheets for Google Sheets¶

Users can now edit the configuration of the downloaded sheet.

Alation Anywhere for Tableau¶

To provide more visibility into automated data upload jobs for Server Admins, we added the Upload Job Status table to the Metadata Upload From Alation to BI Server section of the Tableau BI source settings showing the status of automated data uploads. The details for each upload job are available on clicking the View Details link in the Details column of the table. The table will be available if Alation Anywhere for Tableau is enabled on the Alation instance and if automated uploads are enabled for the Tableau data source.
The @-mentions used in the Description field and in the trust flag comments of data source and table objects are now transferred to Tableau as part of metadata upload.

Alation Anywhere for Slack¶

User Experience Improvements¶

Added a success indicator that an item was sent to a chat to the Alation Search dialog in Slack.
Undefined items have been removed from preview.

Cloud Platform¶

New Explorer Role¶

Alation Cloud Native architecture customers will now be able to use a new user role Explorer with the corresponding new license type. This role offers all the capabilities of a viewer and also provides access to Connected Sheets, query forms, and the Queries API, except the CREATE endpoint.

Bring Your Own Key (BYOK)¶

Bring Your Own Key (BYOK) is an encryption key management system that allows customers to encrypt their data and retain control and management of their encryption keys. Available for Alation Cloud Native architecture customers.

Connect Platform¶

Column Level Lineage Add-Ons¶

Column-level lineage for PostgreSQL will be shown on the Lineage tab along with table-level lineage after running MDE or QLI jobs.
Column-level lineage (CLL) for Presto is GA (previously Beta).
Column-level lineage (CLL) for Databricks is GA (previously Beta).
Column-level lineage (CLL) for Hive is Beta.
Column-level lineage (CLL) for Sybase is Beta.

User Interface for Migrating Native BI Sources to OCF¶

Admins can now migrate native BI sources to OCF in the Alation user interface. Previously, this action required access to the backend of the Alation server.

View Jinja Code on Dataflow Catalog Pages for DBT Dataflows¶

Added ability to view Jinja code on the dataflow catalog pages for DBT dataflows.

New OCF Connectors¶

Athena OCF Connector¶

The Athena OCF connector is available in this release.

Databricks GCP OCF Connector¶

The Databricks GCP OCF connector is available in this release.

DBT Add-on OCF Connector¶

The DBT add-on OCF connector is available in this release. The connector supports DBT Core and DBT Cloud, and extracts DBT descriptions for tables and columns in the catalog and DBT lineage for supported databases and data warehouses (Snowflake, Redshift, and PostgreSQL). The configuration settings for add-on connectors are available on the General Settings tab of OCF RDBMS connectors when the add-on is enabled on the Alation instance.

Also see View Jinja Code on Dataflow Catalog Pages for DBT Dataflows.

If the Compound Layout View is enabled on Lineage diagrams, the dataflow objects from DBT will be grouped together.

SSRS OCF Connector GA¶

The SQL Server Reporting Services (SSRS) OCF BI connector is now General Availability.

Improvements and Updates to OCF Connectors¶

Power BI OCF Connector Enhancements¶

Added support for commas in queries available in dataset expressions returned in Power BI REST API responses for the SQL Server data source.
The connector can now parse queries with multiple quotes and spaces to get the connection information for the SQL Server data source.
The connector now validates port numbers in dataset expressions returned by Power BI REST API responses for all data sources.
The connector now gets connection information from a non-quoted connection string in the dataset expression returned by Power BI REST API responses for the Snowflake datasource.
Enhanced the connector to work with HTTP proxies (for example, Squid).
Added the ability to restrict access to only those workspaces to which the service principal has access.
Enhanced the connector to support Google BigQuery data source for table-level lineage.

Tableau OCF Connector Enhancements¶

Added support for column-level lineage.
Added an error log summary report at the end of metadata extraction.
Added extract or live indicators and extract date for Tableau datasources (both published and unpublished).
Removed pagination fields from the Tableau OCF settings page as pagination is now handled within the system.
Implemented a fix to remove un-extracted sites and (or) projects when selective MDE is run.
Enhanced the connector to support BigQuery data source for table-level lineage.

Looker OCF Connector Enhancements¶

From version 1.4.1, the Looker OCF connector has support for Looker 4.0 API.
Fixed an issue where the curation data for BI reports and BI datasources was lost after extraction if a column had been added or removed to a report in Looker. Alation provides a migration script to update Looker BI objects in Alation to restore missing curation data. The script should be run before the connector upgrade.
Enhanced the connector to support BigQuery data sources for table-level lineage.

Azure Blob Storage OCF Connector Enhancements¶

Added support for schema extraction for CSV, TSV, PSV, and Parquet files.
Added support for on-demand end-user driven Sampling for a CSV, TSV, PSV, and Parquet file. Access keys and Shared Access Signatures are supported authentication mechanisms. Azure Active Directory authentication is also supported via AuthService. Available with Azure Blob Storage OCF Connector version 2.0.0 and Alation version 2023.1.

AWS Databricks OCF Connector Enhancements¶

AWS Databricks OCF connector version 2.0.0 now uses Databricks driver version 2.6.29.
AWS Databricks OCF connector version 2.0.0 is certified to be used with Databricks runtime 11.x.

Databricks Unity Catalog OCF Connector Enhancements¶

Databricks Unity Catalog OCF connector version 1.0.2 supports Sampling, profiling and Compose. Azure Active Directory auth is supported for Compose and Dynamic Sampling for Databricks on Azure.

Query Service Support for More OCF Connectors¶

Query Service enables the use of Compose for OCF data sources that are added through Alation Agent. In this release, Query Service support was added for DB2, Teradata, Greenplum, SAS Base, SAP HANA, Vertica, SAP IQ, and EMR Presto.

QLI Support for RDS-Based Data Sources¶

Added QLI support for MySQL, Oracle, and SQL Server on RDS. Previously, Alation did not support QLI for these RDS-based data sources.

Direct Lineage Extraction Is General Availability¶

The direct lineage extraction feature is now available to all users as the feature flag alation.ocf.mde.direct_lineage.enable_extraction has been removed. OCF connectors that support direct lineage will extract direct lineage metadata from data sources as part of metadata extraction.

Alation Agent¶

For Alation Cloud Service instances, we’ve certified the AWS S3 OCF file system source with Alation Agent running in the customer VPC.

Enterprise Readiness¶

Postgres Upgrade to 13.8¶

The Alation internal PostgreSQL database is upgraded to version 13.8 for customer-managed (on-prem) Alation instances. PostgreSQL version 13.8 includes a security vulnerability fix and provides performance gains over previous versions.

Default Backup Tool Is pgBackRest¶

The default backup tool is now pgBackRest, which has improved backup reliability and performance capabilities over the previous tool, pg_probackup. The pg_probackup tool remains as an option to restore older backups.

Bug Fixes¶

Fixed an issue where if term objects were added as Object Set field values, their descriptions did not appear on the table view of the Object Set field in the main panel of the catalog template. Terms will now have their descriptions displayed in the Object Set field tabular view, as expected.
Fixed a bug where the Details column of the Job History table on the Query Log Ingestion tab of data source settings did not display the success message for the Succeeded status. Now, the Details information will be available for every QLI status.
Fixed an issue where back references from term objects did not appear in the See more dialog. The See more link is displayed when there are more than five back references in the Related Articles field. Now, the dialog will show all available back references.
Fixed an issue where permission to share a query from a catalog page was not available to users with the Viewer role. Now, Viewers can share queries they have access to. The Share button will be available to them under the More menu on catalog pages of queries and query forms.
Fixed an issue where Server Admins could not see the Access dialog for queries if they were not the owner or editor. Now, Server Admins can change query permissions on the query catalog page or in Compose, if necessary.
Fixed an issue where the workflow lookup did not include the workflow after removing an excluded object from the workflow configuration. Former known issue in LA.
For Alation Anywhere for Slack, fixed an issue where the search results were incorrect when the search string contained spaces. Users can now use search strings with spaces.
Fixed an issue where the Impact Analysis or Upstream Audit reports were sending duplicate API requests to the database. For large lineage graphs, this could noticeably affect the lineage data loading time. After the fix, the loading time for large graphs has been significantly improved.
Optimized the migrations of the database which were slowing down the upgrade of the Alation Analytics app. After the installation or upgrade of 23.1, the flags will need to be reconciled between Alation Analytics and the internal database. To do that, execute the one-off script reconcile_flag that is located in the one-off-script folder under Alation Analytics after updating Alation to 23.1. (Included into the 23.1 upgrade steps).
Fixed an issue with tags, which were not updated in Alation Analytics after being removed from the catalog. After the installation or upgrade, the tags will need to be reconciled between Alation Analytics and the internal database. To do that, execute the reconcile_tags script that is located in the one-off script folder of Alation Analytics. (Included into the 23.1 upgrade steps).
Alation Cloud Service instances will no longer send a redundant email notification that was sent to Server Admins after the Alation update that extraction from Alation Analytics failed due to the outdated Alation Analytics version.
Alation now respects custom field permissions during field editing set through People Sets when users use Bulk Actions. Previously, bulk updates of custom fields of picker type by users given edit permissions through a People Set did not work from Stewardship Workbench.
Fixed an issue where users observed a discrepancy between lineage graphs on Lineage V2 and V3 when temp tables were excluded and when the main node was connected to all temp nodes on the first lineage level. Lineage V3 rendered an empty graph, while Lineage V2 displayed the main node. Now, the main node will be displayed even if temp tables are excluded.
Fixed a bug where the column width of the results table in Compose did not scale correctly and clipped the last column, adding horizontal scrolling. The fix also addresses other display issues where content was not visible when the browser window (Chrome) was set to 90% Zoom.
Fixed an issue where a user with the Catalog Admin role, assigned as a Data Source Admin to a data source, could not use the Test Connection feature on the data source settings page. The connection test resulted in an unexpected error. The fix allows Catalog Admins to use the test connection feature.
Fixed a bug where for a Redshift data source, query snippets for queries containing a regex expression produced result sets different to query results for a query without the snippet. The issue occurred due to a discrepancy in the expansion of the regex in the snippet and the query. The issue has been resolved.
Fixed an issue in the Account Settings section where users were unable to navigate to other tabs from the DB Connections tab. The fix ensures that users can click between tabs of the Account Settings page.
Fixed an issue where the Compose auto-complete feature suggested column name in lowercase for columns where the column name had uppercase letters. The attribute model has two fields for storing the table column name. The name field is case-insensitive, while the original_name field maintains the case of the original column name. In order to maintain the case of the column during auto-suggestion, Alation now returns the value of the original_name field.
Fixed an issue where the GBM V2 GET folders bulk API threw an internal server error for a non-existing parent folder. Now, if the parent folder does not exist, the API will return null for the parent_folder parameter.

Known Issues¶

2023.1 Update Known Issue¶

If you have a password set on the internal PostgreSQL database (Rosemeta), the update to 2023.1 will result in an error similar to the following:

WARN: unable to check pg-1: [DbConnectError] unable to connect to 'dbname='postgres' port=5432':
FATAL: password authentication failed for user "postgres"
password retrieved from file "/home/postgres/.pgpass"
ERROR: [056]: unable to find primary cluster - cannot proceed

Update to 2023.1 requires that the password on the internal PostgreSQL database should be cleared for the time of the update. The password can be set again after the update is completed. On how to set the Postgres password, see Set Password for Internal PostgreSQL Instances.

Lineage Compound Layout View Known Issues¶

Searching for a node that is not part of the currently opened pagination page does not return it. Searching for any available node in the cluster should return it even if it’s not part of the current page.
The Show Parent and Show Children actions do not have an effect after a user attempts to expand a lineage node on a large Lineage diagram. (FIXED IN 2023.1.2)
The Show Parent and Show Children actions only display a part of the available lineage nodes on large Lineage diagrams. (FIXED IN 2023.1.2)
After a manual update, the Lineage graph displays the incomplete lineage data with some disconnected lineage nodes.
After a manual update to the Lineage graph and adding new nodes to existing nodes, the updated part of the graph is not displayed and the node that was updated is removed.
Cyclic Lineage graph between multiple nodes results in duplicated nodes and incorrect lineage links.
On the Compound Layout view, the More nodes indicator disappears from the graph after adding a node from the search for more objects dialog, although the nodes are listed in the dialog itself and can be added to the graph. (FIXED IN 2023.1.2)

Scheduled Query Dashboard Known Issue¶

On the Scheduled Query Dashboard, the Next Run column is displayed in UTC while the Schedule column is displayed in the user’s time zone.

Term and Glossary Known Issues¶

Sometimes a term in Draft or Under Review mode shows up in search. Terms should show up in search only after they were reviewed and approved.
Selection of reviewers from People Sets is not available for the Add Object Workflow, as glossaries do not support People Sets yet.
The first click within the Add / Remove Glossaries dialog seems to get “swallowed” and has no effect, but further clicks succeed. The same behavior exists on the Add / Remove Policy Group dialog.

Workflows Known Issue¶

Accessing the Workflows page for the first time after enabling it in Admin Settings > Feature Configurations shows the permission denied error.

Catalog Known Issue¶

After switching a data source from Public to Private, all data source objects are still showing in catalog searches.

Known Issues in 2023.1 Limited Availability Release (build 15.2.0.27498)¶

Server Admin cannot see the permission dialog if they are not the owner/collaborator: In 2023.1 LA, a Server Admin who is neither the owner nor the author cannot see the permission dialog for a query and make permission changes to it. This is a capability that used to be available in 2022.4 and earlier releases. FIXED IN GA
Viewers do not have permission to share a query: In 2023.1 LA build, Viewers do not have the permission to share a query with another Alation user. This is a capability that used to be available in 2022.4 and earlier releases. FIXED IN GA
Workflow lookup not including workflow after removing excluded object in workflow configuration: In certain scenarios, workflow running on a glossary does not show up on the respective Glossary page. FIXED IN GA

Release Notes 2023.1 (General Availability)¶

RELEASE 2023.1.2¶

Alation Catalog and Compose Improvements¶

Alation API¶

OCF Connector Updates¶

Azure Blob Storage OCF Connector¶

Azure Synapse OCF Connector¶

Databricks Unity Catalog OCF Connector¶

dbt OCF Connector¶

EMR Presto OCF Connector¶

Impala on CDP OCF Connector¶

Power BI OCF Connector (Non-Scanner)¶

Redshift OCF Connector¶

S3 OCF Connector¶

SAP BW OCF Connector¶

Snowflake OCF Connector¶

Tableau OCF Connector¶

Connected Sheets¶

Bug Fixes¶

Known Issues¶

RELEASE 2023.1.1¶

RELEASE 2023.1.0 - General Availability¶

Governance App¶

Add Object Request Workflow for Glossary Terms¶

Compose¶

Granular Access Permissions for Queries¶

Scheduled Query Dashboard for Admins¶

Ability to Use Hive Variables in Compose¶

Lineage¶

Compound Layout View¶

Lineage Default View Configuration—No Restart Required¶

Impact Analysis Performance Improvements¶

Catalog¶

Domain and Term Objects Now Respect Object-Level Permissions¶

Term and Glossary Navigation and Search Improvements¶

Data Quality Indicators Added in Search Results¶

Quality Propagation of Downstream Data Objects—General Availability¶

Search Results Span Title and Rich Text Fields¶

Search Indexing Improvements¶

Alation Analytics App¶

Supportability Improvements¶

Data Model Additions¶

Alation APIs¶

New Conversations API¶

Updates to OCF Data Sources API¶

Updates to User V2 API¶

Updates to Virtual File Systems API¶

Alation Integrations¶

Connected Sheets¶

Connected Sheets for Microsoft Excel¶

Connected Sheets for Google Sheets¶

Alation Anywhere for Tableau¶

Alation Anywhere for Slack¶

User Experience Improvements¶

Cloud Platform¶

New Explorer Role¶

Bring Your Own Key (BYOK)¶

Connect Platform¶

Column Level Lineage Add-Ons¶

User Interface for Migrating Native BI Sources to OCF¶

View Jinja Code on Dataflow Catalog Pages for DBT Dataflows¶

New OCF Connectors¶

Athena OCF Connector¶

Databricks GCP OCF Connector¶

DBT Add-on OCF Connector¶

SSRS OCF Connector GA¶

Improvements and Updates to OCF Connectors¶

Power BI OCF Connector Enhancements¶

Tableau OCF Connector Enhancements¶

Looker OCF Connector Enhancements¶

Azure Blob Storage OCF Connector Enhancements¶

AWS Databricks OCF Connector Enhancements¶

Databricks Unity Catalog OCF Connector Enhancements¶

Query Service Support for More OCF Connectors¶

QLI Support for RDS-Based Data Sources¶

Direct Lineage Extraction Is General Availability¶

Alation Agent¶

Enterprise Readiness¶

Postgres Upgrade to 13.8¶

Default Backup Tool Is pgBackRest¶