Release Notes 2024.3 (General Availability)

Note

The release notes on this page reflect changes to the Alation application. If you’re looking for OCF connector release notes, jump to OCF Connector Release Notes.

build 19.0.0.85267

Released September 3, 2024

Note

This release is available for all Alation instance types: customer-managed (on-premise) and Alation Cloud Service.

In this release:

New User Experience Updates

Alation Cloud Service instances on the cloud-native architecture

Ability to Remove the Switch to New Experience/Switch to Classic Experience Button

We’ve introduced a new feature flag, Remove the Back to Classic button in the new UI, which allows Server Admins to hide the Switch to New Experience/Switch to Classic Experience button from the user interface. Enabling this feature establishes the New User Experience as the user interface without the ability to go back to the Classic Experience. Pages that haven’t yet been converted to the New Experience will still be presented in the Classic Experience. (AL-164824)

Alation Anywhere

Alation Cloud Service instances on the cloud-native architecture

Public Preview of Alation Chrome Extension (Beta)

The Alation Chrome Extension enhances browser functionality by allowing Alation users to search and preview data catalog objects directly in Chrome. After installing the extension from the Chrome Web Store, users can log into their Alation cloud instance, conduct quick searches, and view metadata previews. Snowflake users get additional benefits from this integration. After signing into Snowflake Snowsight, users can access detailed metadata about schemas and tables directly through the extension, enriching their data context with metadata from Alation, without leaving Snowflake. (AL-159629, AL-159628)

Homepage

Dynamically Rearrange Custom Blocks on the Homepage

Alation Cloud Service instances on the cloud-native architecture

We’ve introduced a dynamic user interface enhancement, enabling the flexible positioning of custom blocks on the homepage. Server Admins and Catalog Admins can now rearrange homepage blocks by dragging-and-dropping them. Save the changes for the updates to be applied to the homepage. (AL-153691)

Alation Analytics

Group Access Control for Alation Analytics

Data Source Admins now have the capability to manage access to the Alation Analytics data source through groups. On the Access tab of the Alation Analytics data source settings, you can add or remove entire groups and adjust their access levels. Previously, access had to be granted to users individually, which could become inefficient with the increasing number of Alation Analytics users. (AL-82666)

Compose

Scheduled Query Execution Notifications for Allowed Users and Groups

Alation now sends notifications on scheduled query execution to the users and groups with whom the results have been previously shared. To opt out, users can disable all scheduled query alerts via their profile preferences. The feature is controlled with two alation_conf flags:

  • notify_allowed_users_and_groups: Enables or disables sending notifications to users and/or groups listed in the Scheduled Query Setting > Sharing Settings dialog, with whom the results have been shared previously. Enabled by default.

  • notify_users_in_groups: Enables notifications directly to individual users in a group specified in the Scheduled Query Settings> Sharing Settings dialog. Disabled by default, and the specified group is notified through the group email. (AL-168820)

The Sapphire Compose Theme

The Sapphire Compose theme is designed to complement the New User Experience. It will be automatically applied for all users after an admin enables the New User Experience feature flag. This ensures a consistent aesthetic across all Alation interfaces. If users opt to switch from the Sapphire theme to another, the auto-application of the theme will be disabled, and subsequent themes will require manual selection. (AL-159192)

Note

The Sapphire theme is also available on customer-managed (on-premise) Alation instances.

Differentiate Between Launching and Running a Query

We’ve introduced a new status indicator for running queries to differentiate between launching and running a query. Previously, the status message Launching query was displayed for the duration of the query execution. Now, it’ll change to Executing query once the database connection is established. This enhancement is intended to help discern the stage of query processing and quickly spot any connection delays. (AL-21736)

Change to Default Setting for Automatic Connections

The default value for the alation_conf parameter allow_automatic_connection_on_tab_open has been changed to false. This update disables the automatic creation of connections to the datasource when users open a new tab or switch tabs in Compose, causing database account lockout problems when passwords have expired. (AL-165067)

Change to Default Setting for DDL Query Processing

We’ve updated the default behavior for processing of Data Definition Language (DDL) queries from Compose. Previously, the incremental extraction and lineage generation for DDL queries ran synchronously. Now, the alation_conf parameter alation.query_exec.ddl.run_callbacks_async setting is enabled by default, and these processes will run asynchronously. This update aims to enhance performance and reduce the time it takes for queries to complete. (AL-136855)

Deprecation of Desktop Compose

The Desktop Compose installer is no longer available for download from Alation, and the option to select Desktop or Web as the default Compose user experience has been removed from admin and user settings. Existing Desktop Compose installations will continue to function, but users are encouraged to transition to the Compose Web interface that is accessible via a regular browser. All queries and settings like theme and font size are preserved in the Web interface, but open tabs and connections will need to be re-established. Critical support for Desktop Compose will cease by August 2025. (AL-154706)

Lineage

Lineage Support for SQL Server Stored Procedures

Alation Cloud Service instances on the cloud-native architecture

We’ve enhanced lineage support for cataloged stored procedures and functions for SQL Server OCF connectors. It is available for systems on the cloud-native architecture and the mt_lineage service enabled. To activate this feature, the alation_conf parameter alation.ocf.mde.rdbms.parse_function_ddl.enabled should be set to true. This enables the detailed parsing and lineage tracing of function and procedure definitions. (AL-156708)

Known Issue with SQL Server Lineage from Stored Procedures

The lineage chart may not be accurate when queries include columns with unqualified names. (AL-170059, AL-170264)

Note

Alation Cloud Service customers can request server configuration changes through Alation Support.

Lineage Minimap Available by Default

Alation Cloud Service instances on the cloud-native architecture

The lineage minimap feature in lineage charts no longer requires enablement through a feature flag and is available by default. (AL-168144)

Bulk-Import Lineage Information (Beta)

Users with the Data Stewards, Data Source Admins, Catalog Admins, and Server Admins roles can now bulk-import lineage information via an Excel template. This feature allows users to easily identify the success or failure of each submission through a detailed upload report. The feature is currently in beta and can be enabled with the alation_conf parameter alation.feature_flags.dev_enable_lineage_bulk_upload. (AL-159481)

Note

Alation Cloud Service customers can request server configuration changes through Alation Support.

Lineage Settings User Interface for RDBMS OCF Connectors (Beta)

Data Source Admins can now use a separate user interface for configuring cross-system lineage. The feature is currently in beta and can be enabled with the alation_conf parameter alation.feature_flags.DEV_enable_etl_elt_lineage_settings. (AL-152599)

Note

Alation Cloud Service customers can request server configuration changes through Alation Support.

Authentication and User Management

User Profile Upload Limited to 25 Updates

We’ve introduced new limits for uploading user profiles. Each CSV can now contain no more than 25 user creations or updates. If a CSV contains more than 25 rows but the total number of user creations and updates does not exceed 25, the upload will proceed. Details about this restriction are available in the info panel in Admin Settings > User Profiles. Uploads exceeding this limit will show a preview but disable the Confirm button, with an error message displayed below the preview area. (AL-166371)

Bypassing SAML Authentication Is Disabled

The mechanism to bypass SAML authentication has been disabled for administrators of both Alation Cloud Service on the cloud-native architecture and on-premise deployments. In emergency situations, instance admins will need to contact Alation Support to request a temporary bypass. (AL-127742)

Governance

Improvements to Domain Membership Rules and Performance

Catalog Pages will now always accurately reflect the object’s Domain membership, without the lag time that was previously seen. The rules for domain propagation have changed. Previously, exclusions of child objects from Domains were very persistent:

  • An exclusion rule was permanent unless an object was added to the Domain directly.

  • When a parent object was included recursively, then child objects were only added to the Domain if no parent objects at any level were excluded.

Now, the most recent action takes precedence. For example, if a user adds a Schema recursively this week, it will override a previous Domain exclusion on a child Table from last month. This change is designed to align more closely with user expectations, ensuring that catalog pages immediately and accurately reflect an object’s Domain membership. (AL-166059, AL-166060)

Handling of People Set Reviewer Deletion in Workflows

We’ve updated how existing workflows handle the removal of People Sets designated as reviewers. If a People Set set as a Reviewer is removed from the system, it will now be marked as [Gone] in the Reviewer field to indicate its absence. (AL-164891)

De-duplication of Snowflake Tags Enabled by Default

The alation_conf parameter alation.feature_flags.enable_centralized_tags_deduplication is now enabled by default. Snowflake tags from a common database or schema that are used across other databases or schemas will now be de-duplicated by default during extraction. (AL-166048)

Deprecation of the Suggested Stewards Feature

The Steward Suggestions feature is being deprecated. While customer instances with this feature currently enabled can still view, approve, or reject steward suggestions, the option to enable it through the feature flag in Admin Settings > Feature Configuration has been removed. However, the corresponding alation_conf parameter (alation.feature_flags.enable_allie_steward_suggestions) remains available, allowing for the feature to be disabled if necessary. (AL-168008)

Open Connector Framework

OCF Connectors

Updates to the Open Connector Framework (OCF) connectors are released every week. See OCF Connector Release Notes for information on the updates in the month of August 2024. A few August highlights are:

Databricks Unity Catalog OCF Connector Improvements

We have released version 3.0.0 of the Databricks Unity Catalog OCF connector that enhances its metadata extraction capabilities. This update includes switching from a driver-based extraction method to a more robust query-based extraction from the information_schema. (AL-157004)

Enable Token Refreshing for Google BigQuery Compose OAuth

We’ve added the capability to auto-refresh OAuth tokens via the Compose authentication flow for Google BigQuery data sources. Now, when Data Source Admins enable the Request Refresh Tokens checkbox on the Compose tab of the settings page, Alation will auto-refresh tokens for connections established in Compose. (AL-162370)

Improved Workspace Extraction from Azure Power BI

Version 2.4.0 of the Power BI Scanner connector introduces an option to Extract all workspaces from Azure Power BI within the Customize additional extraction scope section on the Metadata Extraction tab.

  • Default Behavior: With the Extract all workspaces from Azure Power BI toggle turned off, only workspaces accessible to the configured service principal will be extracted.

  • Enhanced Extraction: When the toggle is turned on, all workspaces from Azure Power BI will be extracted, regardless of the service principal’s access permissions.

Additionally, this version supports lineage from Redshift native query-based datasets and Snowflake-based datasets. (AL-168903)

Bug Fixes

New User Experience

Alation Cloud Service instances on the cloud-native architecture

  • Previously, if a policy title consisted only of multiple spaces, it would display as blank and would be non-editable. Now, titles composed of multiple spaces will be displayed as Untitled <object type>. (AL-163248)

  • In the Policy Center, the policy group membership dialog previously showed the Save button enabled by default, even if no groups were selected. We’ve updated this behavior so that the Save button will only become enabled when a user selects or deselects any policy group. (AL-162835)

  • Previously, when the Enable New UI - Default and UI Promotion access flag was enabled, links on some pages under Govern would still redirect to pages on the Classic Experience. Now, all pages will correctly redirect to New User Experience when it is set as the default behavior. (AL-163421)

  • Fixed an issue with the Title field validation when creating new workflows. (AL-163548)

  • Addressed an issue that occurred when attempting to edit workflows associated with custom fields that had been deleted from the system. Previously, trying to edit such workflows would result in an error in both Classic and Neo Experiences. Now, instead of blocking the user, the system will display the label [Gone] for the deleted custom field within the workflow. (AL-166130)

  • Previously, when users were assigning Reviewers to workflows, the Reviewer list failed to display all existing People Sets. This fix ensures that all People Sets are now visible when adding a Reviewer to workflows. Additionally, this issue was fixed in the bulk actions and catalog activity features, ensuring consistent functionality. (AL-166710)

  • Addressed an issue where changes to the order of columns in the Document table were not persisting after users attempted to swap them and then save. (AL-163769)

  • Fixed an issue where a homepage name disappeared from the list of homepages in Admin Settings > Homepage Customization when the homepage was disabled and then enabled and saved or enabled and then disabled enabled and saved. Now, the homepage name remains visible in the list after enabling or disabling. (AL-167711)

  • Fixed an issue with adding terms to custom blocks on the homepage where adding a term, folder, or document to a custom block on any homepage resulted in an unexpected error. These objects can now be successfully added to custom blocks without errors. (AL-167993)

  • Fixed an issue where logo updates done on the Admin Settings > Homepage Customization page weren’t reflected on the homepage despite a message indicating a successful save. The issue has been resolved, and the logo now updates correctly in the top left corner after saving. (AL-168163)

  • Fixed an issue where the Column Organization widget was missing from the Policy Center in New User Experience. The Column Organization feature is now available for policy pages. (AL-162838)

OCF Platform and Connectors

  • Fixed a bug in the OCF downstream job for tags that previously led to errors when handling missing tags. The issue caused deleted tag links in the source to remain uncleared in Alation. With this fix, the system now correctly cleans up missing tag links. (AL-166880)

  • Enhanced the ingestion process for Snowflake tags to address an issue where tag values containing newline characters were not being handled properly. This fix ensures that all options for the Snowflake tag picker field are populated correctly. (AL-164875)

  • Updated the SQL Parser grammar type for the Athena OCF connector to address issues with query parsing. This enhancement improves the accuracy of parsing and ingesting queries from Athena into Alation. (AL-167009)

  • Resolved an issue where the GET_DDL() function was called indiscriminately for procedures and functions in Snowflake, leading to extraction query failures and multiple errors. This update enhances the stability and efficiency of metadata extraction from Snowflake data sources. (AL-163439)

  • Fixed an issue in the Snowflake OCF connector data sampling process. Previously, when the maximum rows to store was set to 0, the table row count was incorrectly displayed as 0, rather than reflecting the actual row count. This has been corrected in connector version 2.4.4 to ensure that the row count accurately represents the data in the table, regardless of the max rows setting. (AL-162276)

  • Resolved a limitation in the Tableau OCF connector where Excel sheets embedded from OneDrive weren’t appearing on the Connections tab in Alation. We have now added support for the creation of connections for cloud file types with connector version 1.9.5. (AL-161089)

  • Fixed an issue with storing STRUCT columns in the Alation internal database as NoSQL attribute objects where some of them were duplicated as could be discovered through Search. Duplicating is the result of the internal logical model for NoSQL data types. After this fix, duplicate objects will no longer be discoverable by users, as they only service the internal model. (AL-147379)

  • Fixed an issue where the impact analysis report for a Snowflake data source resulted in empty Path values for BI objects. With this fix, the correct path information will be displayed for BI objects in both Impact Analysis and Upstream Audit reports, ensuring lineage traceability. (AL-164388)

  • Resolved an issue after upgrading to version 2023.3.5.1, where the connector.log file was missing and older connector log files were not being compressed using gzip. This was due to an internal process incorrectly removing the connector from the log path. A conditional check has now been implemented to prevent this from happening. (AL-161279)

Governance

  • Resolved an issue in the Stewardship Workbench Bulk Actions feature, where the Select All option incorrectly applied actions to more objects than displayed in the search results. Now, when using the Select All option, only the objects shown in the search results will be updated. (AL-163762)

  • Optimized the internal APIs behind the workflow step execution after observing slow response times in logs. This enhancement reduces the overall execution time of the APIs. (AL-161158)

  • Corrected a typo in the spelling of Catalog Activity on the Curate and Govern landing page in the Classic Experience. (AL-164324)

  • Resolved an issue that prevented users from editing existing workflows, which was caused by the system attempting to fetch all users and groups. The solution now bypasses the user or group fetch API if no unique users or group profiles are added as reviewers. This change eliminates unnecessary data retrieval, ensuring that workflow edits can be made efficiently. (AL-164472)

  • Fixed an issue causing data dictionary uploads to fail with a 500 internal server error. The problem was traced back to extra double quote characters that were introduced by a third-party CSV editor tool. The fix handles these characters properly, ensuring successful uploads. (AL-165445)

  • Resolved an issue in the Bulk Utility where uploading XLSX files with terms resulted in a 504 time-out error. (AL-161711)

  • We’ve introduced the capability to schedule a backend job that recomputes the members of all conditional catalog sets, which is useful for addressing an issue where previously deleted objects that were restored via an MDE job, did not appear in conditional sets with matching rules. This job is disabled by default and can be enabled with the feature flag stewardship.DEV_recompute_conditional_catalog_sets_members.enabled. You can customize the schedule by modifying the following feature flags according to Celery crontab standards:

    • Day of week: stewardship.DEV_recompute_conditional_catalog_sets_members.schedule.day

    • Hour: stewardship.DEV_recompute_conditional_catalog_sets_members.schedule.hour

    • Minute: stewardship.DEV_recompute_conditional_catalog_sets_members.schedule.minute

    Changes to these scheduling flags also require a restart of the celery-beat service to take effect. We recommend contacting Alation Support if you observe this issue and need to adjust the flags. (AL-168175)

Alation Analytics and Analytics Experience

  • Resolved a synchronization issue between the catalog and the Alation Analytics database regarding deleted fields. Although fields marked as deleted in Rosemeta were accurately updated, this change was not reflected in Alation Analytics. This was corrected by updating the ETL query to ensure it now properly includes records marked as deleted. (Alation Cloud Service instances on the cloud-native architecture, AL-155423)

  • Fixed an issue where Steward updates in term objects made through Bulk Actions were not appearing in Alation Analytics. The solution includes a migration script in the upgrade package that resets the ETL checkpoint for terms. Consequently, the ETL process for terms will restart from the beginning after the update. Given Alation Analytics’ ETL capacity to process only six months of data per ETL run and depending on the volume of terms data, multiple ETL cycles may be necessary to fully update the terms data. This could result in a delay in reflecting the latest terms changes. (AL-158292)

  • Fixed an issue in Alation Analytics that caused errors during data insertion into Snowflake when non-ASCII characters were present in custom fields. (Alation Cloud Service instances on the cloud-native architecture, Analytics Experience, AL-147037)

Other Fixes

  • Resolved an issue where Search results filters failed to populate additional filter data when any documentation object type was used as the object filter. Additionally, deleted folders will no longer appear in full search results. (AL-160582)

  • Addressed an issue in the LMSTitleDescriptionSerializer, where each call to a specific method triggered a separate query execution. This problem was present when the Annotation Value Service was enabled. To optimize performance, unnecessary calls within the LMSTitleDescriptionSerializer have now been removed. (Alation Cloud Service instances on the cloud-native architecture, AL-164402)

  • To enhance application security we have now restricted the uploading of SVG files for articles. (AL-135045)

  • Fixed an issue with sending alerts to admin users. The system now performs status checks on Celery jobs every 30 minutes. If a job missed its scheduled run or if any Celery queues become excessively backed up, an email notification will be sent out. (Alation Cloud Service instances on the cloud-native architecture, AL-156446)

  • Fixed an issue where emails were not being sent to group emails when posts were made in conversations. This update ensures that all conversation assignees, including groups, now receive email notifications. (AL-159349)

  • Previously, users had no option to return to the original object page from the data dictionary upload page except by using the browser’s Back button. Now, users can easily navigate back to the data source from the upload dictionary page using a breadcrumb trail. (AL-152150)

Known Issues

  • Conversations that are deleted and subsequently restored are currently not appearing in the Inbox. (AL-169183)