DataViz Migration - an interoperable semantic layer - {oa-lake-ctx}

Thu, 02 Jul 2026 00:00:00 +0000

Legacy decision-support platforms typically relied on a semantic layer that let users work with business objects rather than tables or SQL queries.

“Customers”, “contracts”, “products”, or “revenue” were exposed in a functional form, independent of the underlying physical storage structures.

To address this need, we designed {oa-lake-ctx}.

{oa-lake-ctx} is the semantic layer of the {oa-lake} architecture. It centralizes business objects, dimensions, measures, KPIs, calculation rules, and security mechanisms.

Key point: consuming tools work with business objects. The physical structures stay hidden.

Architecture

{oa-lake-ctx} centralizes dimensions, measures, hierarchies, KPIs, and calculation rules.

These components are defined once and then exposed to consuming tools.

Unlike some semantic layers on the market, {oa-lake-ctx} requires no specific physical model, no pivot table, and no imposed navigation path.

Relationships are calculated dynamically from the existing model.

Key point: a single business model for all data and all consuming tools.

Data sources

{oa-lake-ctx} can use several data systems simultaneously.

Data can come from Oracle, SQL Server, PostgreSQL, DB2, Snowflake, BigQuery, Redshift databases, files, or other SQL-compatible sources.

Business objects can combine several sources within a single functional definition.

Typical use case: build a KPI from data spread across several platforms.

Business objects

The catalogs exposed by {oa-lake-ctx} group together dimensions, measures, KPIs, hierarchies, and business nomenclature.

Relationships between objects are defined in the semantic layer, not in the reporting tools.

Catalogs can be organized independently of the databases’ physical structures.

SQL generation

SQL queries are generated dynamically from the selected business objects, dimensions, measures, and filters.

Joins, aggregations, and calculation rules are applied by the engine.

User filters are propagated all the way to the source systems via the SQL Push-Down mechanism: no systematic copying of data — queries run against the existing systems, respecting the architectures already in place.

The generated SQL can be executed by {oa-lake} or by any compatible SQL engine.

Key point: users work with business objects; the SQL is generated automatically.

Smart Cache

Data stays in the source systems whenever possible.

The most heavily used datasets can be materialized in Parquet format.

The cache is driven by {oa-lake}’s Cache Manager, managed automatically based on observed usage.

Cache management remains independent of consuming tools.

Key point: the most expensive queries can be accelerated without modifying the source systems.

SQL Engine

{oa-lake} relies on a columnar, in-memory SQL engine.

The queries generated by {oa-lake-ctx} are executed by this engine or propagated to the source systems depending on the context.

The SQL remains portable, standard, and executable on different engines, independent of the storage technologies in use — which limits technology lock-in.

Key point: the SQL engine forms the execution layer of the architecture.

Security

Security is managed at the semantic layer level.

Access restrictions, user profiles, business catalogs, and security filters are applied before queries are generated.

This approach makes it possible to reproduce the security mechanisms found in platforms such as SAP BusinessObjects or IBM Cognos.

Key point: security rules are defined once and then applied to all connected tools.

Reporting

The catalogs exposed by {oa-lake-ctx} can be consumed by any tool capable of querying a PostgreSQL source.

Power BI, Looker, Spotfire, Tableau, Qlik, or other platforms access the same business objects, dimensions, measures, and KPIs.

The functional definitions remain identical regardless of the tool being used.

Result: several platforms can share the same business catalog with no duplication of calculation rules.

Governance

Business objects, KPIs, dimensions, measures, and calculation rules are centralized in a single repository.

Functional changes are made in {oa-lake-ctx} and are then immediately available to all connected tools.

Key point: {oa-lake-ctx} is the central point for defining business objects and KPIs.

Data lineage - feeding third-party data catalogs to expose technical data lineage

Thu, 02 Jul 2026 00:00:00 +0000

{openAudit} lets you browse data lineage through its Mosaïc web portal, but the consolidated metadata can also be published to various data catalogs on the market, which often have significant limitations when it comes to data lineage. The benefit is having a single tool for optimal data governance.

The data lineage is reconstructed in the {openAudit} technical repository, then exposed to data catalogs via dedicated connectors.

This architecture makes it possible to decouple the data lineage collection and reconstruction mechanisms from the exposure layer used by governance teams.

Key point: the technical repository stays independent from the target catalog. The collection and lineage-reconstruction (and connection) mechanisms don’t depend on the data catalog in use.

Architecture

The collected metadata is consolidated in the oA.mdb columnar database after processing by the oA.Job engine.

The central repository notably contains:

Field-level data lineage.
Usage data.
Associated governance information, where applicable.

The catalogs are then fed from this single repository via exposure connectors independent of the data catalog solutions in use.

Exposure connectors

The connectors handle:

Automated publication of the lineage.
Continuous synchronization.
Restriction management.
Support for the various Workspaces.

Data lineage exposure can be complete or partial depending on the display capabilities of the target catalog, keeping in mind that the data lineage produced upstream is itself available at field level.

Result: catalogs only consume the consolidated metadata, not the original technical sources.

Supported catalogs

The available connectors currently support the following data catalogs:

DataGalaxy
Zeenea
Collibra

Usage management

When audit logs are available, observed usage is integrated into the technical repository and then exposed to the catalog.

This lets users distinguish:

“Live” flows, consumed by tools or queried.
Flows — and therefore objects — that see no consumption (often more than 80% of the total).

Typical use case: enrich the data catalog with usage information to identify objects that are candidates for rationalization or decommissioning.

Governance |

DataViz Migration - an interoperable semantic layer - {oa-lake-ctx}

Architecture

Data sources

Business objects

SQL generation

Smart Cache

SQL Engine

Security

Reporting

Governance

Data lineage - feeding third-party data catalogs to expose technical data lineage

Architecture

Exposure connectors

Supported catalogs

Usage management