Databricks Azure OAuth for Compose¶
Applies from version 2020.4
Alation supports OAuth 2.0 for connections from Compose to Azure Databricks data sources with authorization in Azure Active Directory (AAD).
Note
OAuth 2.0 provides a secure mechanism for authorizing an application to access certain resources. This process is overseen by an Authorization Server. Authorization is managed with access tokens which are used by an application to access resources to which it is authorized according to the token until that token expires. When an access token has expired, a refresh token can be used to retrieve a new access token without re-authorization if the Authorization Server is configured to issue refresh tokens and a refresh token has been requested.
When OAuth is enabled for a Databricks data source in Alation, Compose users’ login credentials for Databricks are not stored on the Alation server. When Compose users establish a connection to the data source, they are redirected to the Databricks OAuth login screen in a new browser tab. They authenticate with the authorization server. After that the login screen tab closes, the tokens are stored, and Alation establishes the connection to the Databricks resources.
The following functionality is supported with OAuth:
Query execution in Compose
Scheduled query execution
Query forms
Excel Live Reports
Data upload
Dynamic Profiling
Databricks Service Account¶
The service account that is set up in Alation during the Databricks data source configuration and that is used for MDE, Profiling, and QLI does not use OAuth to connect to Databricks and is not affected.
No Enforcement of OAuth¶
Enabling OAuth for a Databricks data source is an option and is not enforced. Connection with OAuth can be enabled by including the Authmech=11;Auth_flow=0
parameters into the Compose connection URI after OAuth configuration is provided in the data source Settings > General Settings.
Compose users still can connect by directly providing their Azure Databricks credentials. In such a case, their credentials are stored within Alation for reuse.
Required Information¶
The following information is required from the Microsoft Azure portal to configure OAuth for Databricks in Alation:
Client ID
Client Secret
Authorization Endpoint
Token Endpoint
JDBC URI
Username/Claim
Configuration in Azure Portal¶
Note
Refer to Azure Databricks Documentation for the latest version of Service Principal configuration and User Creation information.
Create a Service Principal in Azure Portal¶
Perform the following steps to provision a Service Principal in Azure Portal:
Sign in to Azure Portal using your Azure account.
Select Azure Active Directory > App registrations > New registration.
Provide a name for the app:
Select Single Tenant under Supported account types:
Under Redirect URI, select Web as the app type and provide the Redirect URI as http://localhost:8000/api/datasource_auth/oauth/callback. Make sure there is no forward slash / at the end of the URI. If the Redirect URI is not
localhost
then it must beHTTPS
:Click Register to complete the app registration. The Overview page is displayed.
Select API permissions > Add a permission > APIs my organization uses:
Search with the keyword
AzureDatabricks
and click the app AzureDatabricks. Make sure there is no space in the search keyword:Select the User Impersonation permission and click Add permissions:
Select Grant admin consent for <Account> (Default Directory) and click Yes:
Select Certificates & secrets > New client secret to add a new client secret. Provide the description, select never for the expiry of the client secret and click Add:
Copy the client secret and save it at any other location. You will get only one chance to copy the client secret.
Go to Overview to get the Client ID.
Go to the Endpoints tab. Copy the Authorization Endpoint, the Token Endpoint, and save them to any other location. Make sure that you copy version 1 of the Authorization Endpoint and the Token Endpoint:
User Creation¶
To execute the queries in Compose, the user details must be added to the Admin Console in the Azure portal. Perform the following steps to add/modify the user details:
In Azure Databricks Cluster page, go to Admin Console.
Click Add User to add a new user to the console and provide the user email in the dialog box.
Select the Admin checkbox against the new user to enable the user to write queries in compose.
Setup Databricks Azure Custom DB¶
Refer to Databricks Azure to set up Databricks Azure as Custom DB in Alation.
Enable OAuth in Alation¶
Perform the following configuration on the Settings > General Settings page of a Databricks Azure Custom DB source.
To enable OAuth for Databricks Azure:
Open the data source Settings > General Settings tab, scroll down to the Compose Connections section, and locate the OAuth Connection section.
Modify the JDBC URI. To enable OAuth, add Authmech=11;Auth_flow=0 at the end of JDBC URI.
Select the checkbox Enable OAuth for all Compose Connections. This reveals several parameters for the OAuth setup. Provide the values for the fields based on the description in the table below:
Field
Value
Client ID
Provide the Client ID. Refer to step 12 section Create a Service Principal in Azure Portal.
Client Secret
Provide the Client Secret. refer to step 11 in section Create a Service Principal in Azure Portal.
Enable Refresh Token
Select the Enable Refresh Token checkbox.
Enable PKCE
Does not apply to this data source type.
Authorization Endpoint
Provide the Authorization Endpoint, refer to step 14 in section Create a Service Principal in Azure Portal.
Token Endpoints
Provide the Token Endpoint. Refer to section Create a Service Principal in Azure Portal.
Default Scope
Leave this field blank as the value for this field will be enabled by default.
Username/Claim
Provide email and select the JWT checkbox. You can configure a different claim in JSON Web Token.
Access Token Parameter name
Provide Auth_AccessToken. You can leave this field. blank as the value for this field will be enabled by default.
OAuth Enablers
Provide Authmech=11&Auth_flow=0. Make sure there is an ampersand symbol in between the parameters.
Click Save to enable OAuth for your Azure Databricks source.
Connect in Compose¶
Compose users can connect to the OAuth enabled connection and run the query. Once you run the query, the enabled OAuth connection will redirect to provide the authenticated user credentials to finish the query run.
In Compose:
Select a connection which OAuth is enabled:
Write the query and click Run.
Click Click here to authorize access before connecting.
Provide the authenticated user name and password.
The following query execution features are supported in Compose using OAuth connections:
Run Full Query
Run Current Statement
Run Full Query as Script
Run Full Query & Ignore Errors
Schedule Queries
Live Excel
Data Upload
Query Forms
Dynamic Profiling - Both table profiling and column profiling
Troubleshooting¶
In case the OAuth configuration is incorrect and does not match the Authorization Server configurations, you may get authorization errors in Alation. The error usually contains details about possible causes and may include troubleshooting tips.
Please refer to the table below for message examples.
Error |
Description |
---|---|
Authorization terminated unexpectedly |
This message is shown:
|
The authorization server reported a failed authorization attempt: |
If authorization fails, the authorization server may respond respond with error details which will be listed after such an error message. Examples:
|
Token request failed following successful authorization. |
In some cases after successful authorization, the request for tokens can fail. This message will be followed by further details. |
There was a problem extracting username information following successful authorization and token retrieval. Please check the OAuth settings for the data source. |
The causes of this error may be:
|
No default role has been assigned to the user, contact a local system administrator to assign a default role and retry. |
Displayed if there is no default role assigned to the user in Snowflake and none is specified in the connection URI. |
Role ‘<role_name>’ specified in the connect string is not granted to this user. Contact your local system administrator, or attempt to login with another role, e.g. PUBLIC. |
Displayed if the role is not accessible to the user and this role is specified in the connection URI and is authorized either explicitly in the scope or via SESSION:ROLE-ANY scope. |
The role requested in the connection or the default role if none was requested in the connection (‘SYSADMIN’) is not listed in the Access Token or was filtered. Please specify another role, or contact your OAuth Authorization server administrator. |
Displayed if either:
|
User’s configured default role ‘SYSADMIN’ is not granted to this user. Contact your local system administrator, or attempt to login using a CLI client with a connect string selecting another role, e.g. PUBLIC. |
Displayed if the role is not accessible to the user in Snowflake but this role is authorized either via Default Scope by Alation or a role-related scope on the authorization server. |
Where to Find Logs¶
The log entries for OAuth authorization can be found in /opt/alation/site/logs/uwsgi.log (path inside the Alation shell).