Migration from Native to OCF Connectors¶

You can migrate an RDBMS data source or a BI source cataloged using a built-in (native) connector to Open Connector Framework (OCF) if an OCF connector is available for your database type or BI server. For on-premise installations of Alation, RDBMS and BI data sources can be migrated using the command line. Starting with 2022.4, RDBMS data sources can be migrated using the Alation user interface. Alation instances on Alation Cloud Services can use this new UI option for RDBMS data sources or contact Alation Support for BI data sources.

We recommend migrating to OCF to take advantage of the following benefits:

The OCF connector releases are not tied to major Alation releases and follow their own schedules. Alation frequently delivers OCF connector bug fixes and improvements.
Unlike built-in connectors or Custom DB, OCF connectors can be upgraded separately from the Alation application when a new connector version becomes available.
OCF connectors are more scalable, flexible, and performant compared to built-in connectors.
In the coming releases, built-in connectors will be gradually deprecated.

Important

To interact with your data source via API after migrating to OCF connectors, you will need to start using v2 of the Data Sources API. The older v1 of the Data Sources API doesn’t work with OCF connectors.

Migration Scope¶

After the migration to OCF, your data source type will change from the specific database name or Custom DB to OCF. The connector type information is available on the Sources page.

The migrated RDBMS data source or BI source will be added to the Datasources table on the Admin Settings > Manage Connectors > Connector page.

The following information will be migrated from the built-in RDBMS data source or BI source to the OCF-based source:

Source ID — The data or BI source ID will remain the same as before the migration.

Physical metadata of the source such as schemas, tables, attributes, projects, and other previously extracted metadata. If needed, a Data Source Admin can re-run extraction after migrating to the OCF connector.

Logical metadata such as values of the custom fields, data quality flags, @-mentions in the custom fields, @-mentions in articles, the title and description, stewards, tags, top users, and fields shared through catalog sets.

Saved searches.

Sampling and profiling data.

Catalog information generated from query log ingestion (top users, popularity, and join and filter information).

Lineage data.

Query log data ingested from Compose.

Important

For RDBMS data sources, the connection and extraction settings of the data source will be migrated too starting with release 2022.2. In earlier versions, the data source settings were not migrated.

For BI sources, the information on the Settings page is currently not migrated.

We recommend saving the settings of your source or taking screenshots of the Settings page before performing the migration. For a BI source, you will need to repopulate the values in the fields on the Settings page. For an RDBMS data source, you’ll be able to validate that the previous settings were successfully migrated.

Prerequisites¶

You must first install the relevant OCF connector on your Alation instance. See Manage Connectors for directions.

Migrate an RDBMS Data Source to OCF Using the UI¶

Starting in 2022.4, you can migrate RDBMS data sources to OCF using the UI for both on-premise and Alation Cloud Service instances of Alation. To migrate BI or file system data sources, you can use the command line option below for on-premise instances of Alation or contact Alation Support for Alation Cloud Service instances.

To use this method, a Server Admin must first enable it.

As a Server Admin, click the gear icon in the top right corner to open the Admin Settings page.
Click Feature Configuration.
Find Enable Native Connector Migration to OCF Connector and click the toggle to enable it.

To perform the migration, you must be a Data Source Admin for the data source you are migrating. For more information about data source privileges, see the Access Tab topic.

To migrate your RDBMS data source to OCF using the Alation interface:

Click the Apps menu in the top right corner, then select Sources.

Select the Manage Settings tab, then click the wrench icon for the data source you want to migrate.

Important

This is a good time to take screenshots or otherwise take note of the current settings for your data source. Some settings will not be migrated. You will need to manually reconfigure any settings that aren’t migrated.

Settings	Google Big Query, Hive, and AWS Glue	All Other RDBMS Data Sources
Metadata extraction filters	Migrated	Migrated
Connection settings (username, password, and JDBC URI)	Not migrated	Migrated
All other settings (certificates and advanced authentication, for example)	Not migrated	Not migrated

Select the General Settings tab.
Scroll down to Migrate Data Source at the bottom. If it isn’t there, see the instructions above for enabling this feature.
Click the Migrate link. The migration dialog appears.
Using the Choose a Connector drop-down menu, select the OCF connector you’re migrating to. The connector name indicates what type of database it supports.

Important

Data source migration to an OCF connector is irreversible. If you select the wrong OCF connector, your data source could become permanently broken.

Note

You can only migrate RDBMS data sources using this method.

If the needed OCF connector is unavailable in the drop-down list, you must first install the connector. See Manage Connectors for directions.
To verify that you’ve chosen the right connector, type in the entire connector name and version in capital letters. Type the exact name as shown in the Choose a Connector drop-down menu. This will enable the Migrate button.
Click the Migrate button.
The data source will be migrated to the chosen OCF connector, and the OCF data source settings page will open. Check the notes you made of your previous settings and reconfigure any settings that were not migrated.

Migrate a Data Source to OCF Using the Command Line¶

This option is provided for on-premise installations of Alation. This method can be used to migrate both RDBMS and BI data sources to OCF. For Alation Cloud Service instances of Alation, use the option above for RDBMS data sources or contact Alation Support for BI data sources.

To migrate your data source to OCF using the command line:

Make sure that the required OCF connector has been installed and is available on the Alation server by going to Admin Settings > Server Admin > Manage Connectors page. The Manage Connectors page displays all OCF connectors currently available on the Alation server. On this page, locate the OCF connector you are migrating to.
Find out the ID of the source you want to migrate. See How to Find Data Source ID.
Make sure you have saved the connection, extraction, profiling, and QLI information for your source or made screenshots of its settings.
Use SSH to connect to the Alation host.
Enter the Alation shell:
sudo /etc/init.d/alation shell
Change the user to alation:
sudo su alation
Run the command given below to find out the ID of the OCF connector that will be used for this migration:
alation_ypireti list --fields id name
This command returns a list of IDs and names of the OCF connectors that are currently running on the Alation host. Locate the ID of the connector that you need.
Navigate to the tmp directory.
cd /tmp
Use the command below to migrate your source to OCF, substituting the placeholder values with real values.
<Source_ID> — The source ID of the source that you want to migrate to OCF.

<Connector_ID> — The ID of the OCF connector.
alation_ypireti migrate --source_id <Source_ID> --id <Connector_ID>
Note

The migration tool will function correctly even if you have RDBMS and BI data sources with the same source ID. It will automatically detect whether the target connector is RDBMS or BI and treat the data source accordingly.
Once the migration is complete, the following success message should appear:
Exit from the user alation.
```
exit
```
Exit the shell.
```
exit
```

Migration Log Location¶

For on-premise installations of Alation, the migration logs are available in the ocf.log file.

Validate Connection after Migration¶

After the migration, validate the connection between Alation and the database or the BI system. To test the connection:

In the Alation user interface, go to the Settings page of the source that was migrated.
Scroll down to the Test Connection section.
Under Test Connection, click Test. The test should return the message Network connection successful.
If the connection fails, check and update the connection settings and try again. You can also check the connector logs for any specific connection errors. For RDBMS data sources, to view the connector logs, click the link on top of the Settings page to go to the corresponding connector page in Admin Settings > Manage Connectors. For BI sources, go to Admin Settings > Manage Connectors and open the corresponding BI connector page to view the log output.