Amazon DynamoDB OCF Connector (Public Preview): Overview

Alation Cloud Service Applies to Alation Cloud Service instances of Alation

Customer Managed Applies to customer-managed instances of Alation

Available from Alation version 2023.3

The Amazon DynamoDB OCF Connector is currently released as Public Preview or Limited Availability.

Overview

The latest OCF connector for Amazon DynamoDB is available for download from the Connector Hub on Alation Customer Portal. Ask an Alation admin with access to Customer Portal to download the connector from the Connectors section (Customer Portal > Connectors). You can upload and install the connector file in the Alation application. The connector is compiled together with the required database driver, so no additional effort is needed to procure and install the driver. Use this connector to catalog Amazon DynamoDB as a data source on Alation on-premises and Cloud Service instances. It extracts and catalogs such database objects as schemas, tables, views and columns. After the metadata is extracted, it is represented in the data catalog as a hierarchy of catalog pages under the parent data source. Alation users can leverage the full catalog functionality to search for and find the extracted metadata, curate the corresponding catalog pages, create documentation about the data source, and exchange information about it.

Team

You may need the assistance of your database administrator to configure this data source.

  • DynamoDB administrator:

    • Creates a service account for Alation.

    • Provides the connection information and the JDBC URI.

    • Provides the authentication information and assists in configuring the authentication.

  • Alation Server administrator:

    • Ensures that Alation Connector Manager is installed and running or installs it.

    • Install the connector.

    • Creates and configures the DynamoDB data source in the catalog.

    • Performs initial extraction and prepares the data source for Alation users.

Scope

The table below describes which metadata objects are extracted by the connector and which catalog functionality is supported.

Feature

Scope

Availability

Authentication

Basic

Authentication with username, password, and security token

Yes

AWS Root Keys

Authentication with Access Key and Secret Key.

Yes

SSL

Connection over the TLS protocol

No

LDAP

Authentication with a database service account that is an LDAP account in an organization’s network

No

OAuth

Authentication with the OAuth 2.0 protocol.

No

SSO

Authentication using an SSO flow through an IdP application

No

Metadata Extraction (MDE)

Default MDE

Extraction of metadata based on the JDBC driver methods in the connector code

Yes

Custom query-based MDE

Extraction of metadata based on extraction queries provided by a user

No

Popularity

Indicator of the popularity (intensity of use) of a data object, such as a table or a column

No

Extracted metadata objects

Data Source

Data source object in Alation that is parent to extracted metadata

Yes

Schemas

List of databases

Yes

Tables

List of tables

Yes

Columns

List of columns

Yes

Column data types

Column data types

Yes

Views

List of views

N/A

Source comments

Source comments

N/A

Primary keys

Primary key information for extracted tables

Yes

Foreign keys

Foreign key information for extracted tables

N/A

Functions

Extraction of function metadata

N/A

Function definitions

Extraction of function definition metadata

N/A

Sampling and Profiling

Table sampling

Retrieval of data samples from extracted tables

Yes

Column sampling

Retrieval of data samples from extracted columns

Yes

Deep column profiling

On-demand profiling of specific columns with the calculation of value distribution stats

Yes

Dynamic profiling

On-demand table and column profiling by individual users who use their own database accounts to retrieve the profiles

Yes

Custom query-based table sampling

Ability to use custom queries for sampling specific tables

Yes

Custom query-based column profiling

Ability to use custom queries for profiling specific columns

Yes

Query Log Ingestion (QLI)

Table-based QLI

Ingestion of query history based on a table that contains query history data

No

Query-based QLI

Ingestion of query history based on a custom query history extraction query

No

JOINs and filters

Calculation of JOIN and filter information based on ingested query history

No

Predicates

Ability to parse predicates in ingested queries

No

Lineage

Automatic lineage generation

Auto-calculation of lineage based on query history ingested from QLI, MDE, and Compose queries

No

Direct lineage

Extraction of lineage from system tables during MDE

No

Column-level lineage

Extraction of lineages on the column level

No

Compose

Not supported

Object Mapping

DynamoDB Object

SQL Object

Collection

Table

Document

Rows or Records within Table