Intended Audience for this Guide

All Users

Understanding the Asset Lifecycle

To make better use of the catalog, such as checking the status of assets or setting detailed conditions in asset searches, it may be necessary to understand the asset lifecycle. The asset lifecycle varies depending on the type of data source.

agent Data Source Assets

This applies when a connector automatically creates and updates assets by retrieving catalog-targeted metadata from user organization services (e.g., data warehouses).

Operations marked as manual execution in the diagram below are performed by operators or administrators.

1. Data Source Creation by Connector

The connector creates a new data source on its first execution.

2. Asset Creation/Update and Detailed Metadata Update by Connector

When the connector is executed, it retrieves catalog-targeted metadata from user organization services (e.g., data warehouses) and creates them as assets in the data source. The technical metadata of the assets obtained by the connector is also updated.
Detailed metadata that cannot be obtained from the data warehouse is updated by the operator using QDIC functions.

3. Regular Connector Execution

It is assumed that the connector will be executed regularly. The retrieval of cataloging metadata and asset creation, as well as detailed metadata updates by connector execution, are assumed to be repeated regularly as part of operations.

4. Assets with Missing Source Data

For example, if an asset representing a table that exists in the catalog is not retrieved from the data warehouse during the latest connector execution, this asset is detected as Missing Source Data and becomes an asset with missing source data. The missing source data status is also displayed in the workspace. Situations where missing source data occurs include the following:

• The table was physically deleted from the data warehouse before connector execution.
• The table still exists in the data warehouse but was excluded from cataloging by a decision from the administrator or environment operator.

An asset with missing source data means an asset that is now eligible for deletion.

5. Asset Deletion

Operators delete assets that have become missing source data. In some cases, assets with missing source data detected by the latest connector execution may be deleted automatically. (This is determined by the operator.)

6. Data Source Deletion

Operators may delete data sources depending on the data warehouse situation or catalog operation management policy.

csv Data Source Assets

This applies when an operator creates or updates assets by outputting catalog-targeted metadata from user organization services (e.g., data warehouses) to a file. This also applies when creating custom data sources from files for the user organization.

Operations marked as manual execution in the diagram below are performed by operators or administrators.

1. Data Source Creation

The operator manually creates a new data source in the tenant using a file and creates assets in it.

2. Asset Creation and Metadata Update

The operator regularly repeats asset creation and metadata updates.

3. Archiving Assets

The operator can manually archive assets. Archiving an asset is considered a pre-processing step before asset deletion. Archived assets can be unarchived if they have not yet been deleted.

4. Deleting Archived Assets

The operator can manually delete archived assets.

5. Bulk Asset Deletion

The operator may manually bulk delete specific assets.

6. Data Source Deletion

The operator may delete data sources depending on the catalog operation management policy.