Release Notes 2023.3 (General Availability)¶
build 17.0.0.49181
Released September 5, 2023
Alation Agent¶
Alation Agent Notifications¶
Once a user subscribes to Alation Agent notifications, the user will receive email alerts 60, 45, 30, 20, 10, 5 and 1 day before the certificate expires. If the certificate expires due inaction, daily reminders will be sent to ensure action is taken.
Alation Agent Diagnostic Tool¶
The Agent diagnostic tool allows users to troubleshoot connectivity issues between the Alation Agent and Alation. It runs a series of diagnostic steps, and is installed on the Agent node. The tool provides an easy way to gather all the connector logs with a simple command.
Alation Agent Docker Version Update¶
Upgraded the version of Docker bundled with the alation_container_service to 24.0.5.
Alation Analytics¶
Stewardship Data Is Available in Alation Analytics¶
Stewardship Workbench (Bulk Actions) data is now available in Alation Analytics. With this change, customers can query Alation Analytics to get metrics around bulk actions.
Third-Party BI Access to Alation Analytics¶
Cloud native architecture only
Alation Cloud Service customers on the cloud native architecture can now connect third-party BI tools to Alation Analytics. This is done by replicating your Alation Analytics database to Snowflake, where you can connect your BI tools to access Alation Analytics data.
Alation Anywhere¶
Slack Conversations¶
Added support for previewing Conversation object types in Slack, and added the conversation option for all supported object types. A user is able to quickly start new conversation from Slack per specific object, reply to conversation, or mark one as resolved.
Preview Dashboard Object Types in Slack and Microsoft Teams¶
Support of enhanced Dashboard preview in Alation Anywhere for Slack and Microsoft Teams integration.
Alation Database Schema Enhancements¶
Modified the Alation database schema to support larger volumes of metadata. Users can now ingest more metadata on a frequent basis without running into issues with integer overflow. Note that migrations during an upgrade can add an additional 30 minutes depending on the volume of the metadata.
Compose¶
Flexible Query Result Protection¶
To ensure Alation instances comply with customer security policies, this feature provides more control and flexibility to datasource admins on how query results can be shared. Datasource admins have three levels of control on whether the sharing the query result is allowed or not: No sharing is permitted, only private sharing is allowed, or allowing private and public sharing. Datasource admins can also control if users have the ability to preserve query results without automatic deletion
Compose Compatibility with Query Service-enabled OCF Connectors¶
Compose is now compatible with Query Service enabled Starburst Trino and SybaseIQ OCF Connectors, so that clients using these data sources can migrate to these OCF Connectors.
Compose Export Notifications¶
When Exporting a query execution, Compose will show an alert message containing the title of the query and the number of result rows exported upon each completed query statement, so that users are informed if the Export is successful.
Connected Sheets Improvements¶
Users can now see Column Titles were available in search left navigation bar
Search results auto expand in left navigation
Conversations¶
Conversations are now Reflected in Domains¶
Conversations about catalog objects, which are associated with data domains, are now transitively associated to the same domains.
Multiple Users and Groups Assigned to Conversations¶
Handling multiple assignees for a single conversation (before the change only one user could be assigned to a conversation).
Added possibility to assign group profiles to conversations.
Authentication¶
Metadata Extraction based on Cross-Account IAM Role-based Authentication¶
Support for cross-account authentication is needed for Metadata Extraction (MDE) between the Alation instance and AWS resources that might reside in different AWS accounts. With IAM role-based authentication, customers do not have to enter IAM user credentials in the Alation Catalog. Authentication is performed based on trust set up between an IAM role in the Alation account and an IAM role set up in the customer’s AWS account where the AWS resource is running. Metadata extraction and query log ingestion from the following OCF AWS connectors: S3, Glue, Athena, and Redshift, are supported by this feature.
API Authentication using JWT - Cloud native architecture only¶
APIs can now be secured with JWT token-based authentication in addition to using opaque tokens. With this new authentication, customers can register an external application with Alation and use the JWT tokens assigned to the application for authentication and authorization.
Cloud Configuration - Cloud native architecture only¶
New Cloud Native Architecture Service: Entitlement and License Management Service¶
Cloud Entitlement and License service displays customer licenses in the license user interface page based on entitlement grants/revocations and track usage of features. The license screen can be viewed only by Server Admins.
Connectors¶
Azure Blob Storage OCF Connector, version 2.3.4¶
Improved the Azure Blob Storage OCF Connector performance by streaming directories on the fly instead of storing it in memory.
Azure Databricks OCF Connector¶
Azure Databricks OCF Connector Driver has been upgraded to 2.6.33. (AL-97592)
Databricks Unity Catalog OCF Connector, version 2.0.3¶
Added QLI support and upgraded the Simba Spark Driver 2.6.21 to the Databricks JDBC driver 2.6.33. (AL-118979)
Disabled QLI Lineage by default. The datasource admins no longer need to disable the automatic lineage generation flag on the datasource settings page manually to turn off QLI Lineage. (AL-130658)
Google BigQuery OCF Connector, version 1.1.8¶
Improved the memory management and performance of the Google BigQuery OCF Connector. (AL-125140)
Power BI OCF Connector¶
Power BI Dataflow features introduced for Power BI Scanner Connector. (AL-111227)
PowerBI Dataflow objects are now cataloged within Alation for the PowerBI OCF Scanner Connector.
Dataflow objects provide reusable transformation logic that can be shared by Datasets and Reports within PowerBI and the Power Platform.
Dataflow objects strengthen security around underlying datasources by exposing required data within it for reports instead of from the datasource.
Dataflows are cataloged alongside Datasets in Alation.
Users can now view the datasource referred by the Dataflow along with Datasets and Reports that are created from it and the fields that comprise the dataflow object.
Hybrid sources that power a Dataset, for example, created from Dataflow(s) and another datasource are now visible.
Dataflow objects can be nested i.e. created from other Dataflows.
Snowflake OCF Connector¶
Added check for supported Alation versions during installation. (AL-129242)
Installation will fail on unsupported Alation versions. (AL-129242)
SQL OCF Connector¶
Added check for supported Alation versions during installation. (AL-129243)
Installation will fail on unsupported Alation versions. (AL-129243)
Starburst Trino OCF Connector, version 1.2.8¶
Fixed the issue when a customer adds catalog name in JDBC URI, GET list of schemas picks up schemas for all catalogs. (AL-127362)
Changed Parser Grammar type from ANSI to PRESTO to align with native connector implementation. (AL-127352)
Tableau OCF Connector, version 1.6.1¶
Resolved out of memory issues and added support for report fields value sampling. (AL-127096) (AL-120383)
Fixed report columns value sampling feature. (AL-114383)
Added owner property for projects and sub projects. (AL-127028)
Added lineage support for tableau dashboards with hidden sheets. (AL-125127)
Enhanced User Experience for the OCF Connector Authentication Process (Private Preview)¶
OCF Connector Authentication improvements include: (AL-125067)
Enlarged the URI field to improve user readability and editing.
Clear help text for URI and QLI fields.
Streamlined the interface to display relevant fields based on the selected authentication method.
Rearranged the authentication process to follow a logical order: Connection, Service Account, Permissions, and QLI.
Governance – Machine Learning¶
Allie Steward Suggestions (Public Preview – Cloud native architecture only)¶
Allie Suggested Stewards helps users automatically find the right people as stewards for each data object. This feature leverages a machine learning model that suggests stewards based on interactions, and also improves the suggestions over time by learning from user actions. This is applicable for cloud native architecture customers only.
Lineage¶
Compound Layout Feature Improvements (E2E Column-Level Tracing/Beta Launch)¶
The compound layout feature, which has been available behind feature flags since 2022.4, is now available as a Beta offering to all customers in all environments. The compound layout feature groups data sources, e.g. Snowflake, separately from other data sources, e.g. Tableau, to provide focus on data movement for certain objects, e.g. tables, across these data sources. This feature has been enhanced to drill down from data sources to the column level with the ability to trace columns upstream and downstream across data source groups to capture a holistic E2E lineage view. (AL-126630)
Support for Rename and Deletion of RDBMS data sources in Lineage Source Groups¶
Deleting an RDBMS datasource deletes the corresponding lineage source group.
Renaming a RDBMS source renames the corresponding lineage source group.
Impoved Lineage Performance¶
Created scripts to clean up duplicate temp objects and dataflows from lineage to address performance issues with large graphs. (AL-107764)
Native to OCF Migration¶
Improvements made related to Bulk API for Native to OCF Migration. (AL-12868)
PostgreSQL Upgrade¶
PostgreSQL is upgraded to 13.11 from 13.6.
Search¶
Save Column Selections in Saved Search¶
Saved Search will automatically save the column selections and their sequence in the Search Results Table view, so that users will see their prior column selections when they retrieve their Saved Searches.
Terms and Glossaries Improvements¶
Customers will now see all valid custom fields, fields present on one or more term type templates, as selectable options on Term Tables by default. (AL-127525).
Verbiage on Bulk Utility article body has been updated to indicate that other usages are “not supported” (previously “not recommended”). (AL-127569).
Term detail pages now show the “Description” field name. (AL-128745).
APIs¶
Public APIs for Table User Permission / Privacy Updates¶
Added Public APIs for updating the table permissions and privacy settings. More information is available in the swagger docs under ‘/openapi/members_permission/’ and ‘/openapi/privacy_settings/’. Server admins by default have access to both APIs. Members with catalog or source admin roles can only access these APIs if they are tagged as data source admins for the table’s parent data source.
Upload Data Dictionary Public API (Public Preview)¶
Public Preview of Upload Dictionary Public API that brings an improved, performant and asynchronous approach to update records supporting a csv as an input file.
Conversation Public API Version 2¶
The new API is available at the following endpoint: ‘/integration/v2/conversations/’. This API includes the ability to handle multiple assignees for a single conversation and added the possibility to assign group profiles to conversations.
Conversations Public API Improvements¶
Conversations Public API is updated with advanced handling of deleted conversations, including new search options and possibility to restore deleted conversation.
Updated the Conversations Public API V2 and Conversations Internal API to support new searching and filtering options. Users can search for conversations by creation date, assignee, or last response date. (AL-125306).
Workflows Public APIs¶
Public APIs for workflows are now available. These will allow users to list all the workflow requests, create a change request, and show workflow request details.
Bug Fixes¶
Alation Analytics: - Fixed issue causing customer to not be able to query rdbms_columns. (AL-126878)
Fixed issue causing deprecation flag description that was updated using the API not being updated in Alation Analytics V2. (AL-123876)
BI Migration:
Migration script validates if source and target are the same before starting migration using the script. (AL-127623)
Added validation for source and target when migrating servers from native to OCF in Alation. (AL-126483)
Bulk Import Utility: Fixed an issue causing keys inconsistency and incomplete, missing data source ID (ds_id) and causing an error. Download template now has all object sets of type table, data source, column, schema prepended with ds_id. (AL-119516)
Bulk Utility: Fixed issue to avoid Bulk Utility Download process issue on encountering a Term linked to a Glossary without title. (AL-125179)
Compose:
Comment syntax inside query variables are now treated as plain text. (AL-128065)
The “Open New Query In Compose” button from AAV2 dashboards now creates queries without titles and prevents them from being automatically published to avoid duplicate queries. (AL-126656)
Fixed issue causing Schedule Result Sharing Modal to not follow the admin setting ‘Allow Public Sharing Of Compose Results’. After the fix, public sharing is disabled if it is disabled from admin settings. (AL-124132)
Conversations: Fixed an issue causing the conversation resolution status to be empty or null. (AL-106466)
Google BigQuery OCF Connector: Fixed issue causing Lineage to not display for Temporary Tables and other connected upstream and downstream tables, although the queries are parsed successfully. Added support to show the lineage via a temporary node. (AL-117243)
Governance/Workflow: Fixed issues for Review Policy/Term object set and Suggest column object set. (AL-128809)
Lineage:
Added support to create lineage for duplicate queries with different comments across data sources. (AL-125865)
Fixed issue causing incorrect list of data sources to display for DataFlow sources. Deleting or renaming a RDBMS source now deletes/renames its corresponding lineage source groups. (AL-120403)
Fixed issue to support column-level lineage (CLL) for Select * queries with SET operators. (AL-77284)
Power BI Scanner OCF Connector: Fixed connector links for apps and app related objects. (AL-120876)
RDBMS Migration: Migration script validates if source and target are the same before starting migration using the script. (AL-127532)
Search:
Fixed an issue causing multi-picker custom fields in catalog search to not display an accurate not_set count for custom fields. (AL-116323) (AL-95802)
Fixed an issue when using Edit Columns on the Advanced Search page to not display custom fields. (AL-100937)
Fixed appearance of backend value for “not set” filter. (AL-130483)
Snowflake OCF Connector:
Fixes KeyError that occurs while Snowflake tag ingestion when a schema/table/column associated with a Snowflake tag is not found in Alation. (AL-128018)
Fixed failure in policy links ingestion due to missing objects caused by inconsistent custom queries for tables and columns. (AL-125463)
Fix failure in tag links ingestion due to missing objects caused by inconsistent custom queries for schemas, tables and columns. (AL-125462)
Fixed issue causing the error when trying to establish database connection in Compose when Snowflake datasource is configured to use OAuth. (AL-124305)
Fixed issue when detecting role switching for Snowflake configured using External Authentication. (AL-122112)
Lineage is not getting created for Snowflake Create View DDL queries having “With Tag” clause. (AL-115186)
Tableau OCF connector: Fixed connector links in Tableau OCF connector. (AL-118648)