This page has been machine-translated and may contain inaccuracies in phrasing or product terminology. If discrepancies exist, the original Japanese version takes precedence.
This document explains the mechanism of the calculation for "Anomaly Detection" displayed in the detailed metadata of assets. (Anomaly detection targets assets of data sources for which statistical information is obtained by connectors (Intelligence Agent).)
Intended Audience for this Guide
What is Statistical Information?
Statistical values calculated within services (such as data warehouses) in the user's organization through connectors are registered in the catalog as statistical information. Statistical information is also one type of property (metadata). Users can view column-specific statistical information from the asset side drawer.
For details, please refer to the following: About Asset Statistical Information
About Anomaly Detection
QDIC retains the statistical values registered in the catalog as a time series. Using these time series values, the connector determines whether the most recently obtained value is an anomaly. For example, in the case of the average value, a box plot is calculated using previously computed average values (the calculation of the box plot uses past values within the anomaly detection window size set by the operator in the data quality management screen). The most recently obtained average value is compared to whether it exceeds the outlier range of the box plot to estimate if it is an anomaly, which is then displayed on the workspace.
Please note that to confirm whether a value is truly an anomaly, it is necessary to check the actual data within services (such as data warehouses) in the user's organization.
* IQR = 3rd quartile - 1st quartile