Ellipsys — Data lineage & BI/ETL migration toward open architectures |

ETL Migration - from legacy ETL to an open SQL ETL - {oa.tbx}

Thu, 02 Jul 2026 00:00:00 +0000

Once ETL processes have been converted to SQL by {openAudit}, they can be run inside the {oa.tbx} container.

{oa.tbx} provides an execution environment for SQL, Python, or Java processes, driven by configuration and orchestration repositories.

Key point: the migrated processes become SQL instructions that can be executed independently of the original ETL platform.

Principles

SQL is encapsulated in open containers, in ETL mode.

Containers: transformations run inside an external container, {oa.tbx}, detached from the target database. Temporary containers act as intermediate storage areas where data is loaded before transformation. Jobs are tested to verify correct behavior, and possibly the data itself, to guarantee its integrity.
Orchestration with Airflow: Airflow drives the execution of the containers, step by step. Each transformation, within a container, is defined as a job.

The scheduling of an ETL migrated to Airflow automatically picks up all of the original instructions and scheduling logic.

Execution chain

ETL jobs are converted to SQL by {openAudit}.

The generated processes are integrated into a configuration repository and a control repository used by {oa.tbx}.

The workflows are then orchestrated by tools such as Airflow or Dagster.

Execution container

{oa.tbx} runs the processes described in the workflows.

Dataflows can combine:

SQL;
Python;
Java;
others.

The engine can also rely on embedded databases such as DuckDB for certain intermediate processing steps.

Sources and targets

Input data can come from any technology accessible via SQL or JDBC.

Results can be loaded into any target system compatible with standard write mechanisms.

The architecture remains independent of source and target technologies.

Orchestration

Orchestration remains separate from the processes themselves.

Dependencies, execution parameters, schedules, and triggers are handled by the workflow layer.

Business logic stays contained within the SQL scripts and dataflows executed by {oa.tbx}.

Benefits

{oa.tbx} runs open SQL pipelines, orchestrated by Airflow and independent of proprietary Cloud platform runtimes.

Across all migrated jobs — with volumes ranging from a few thousand to more than 50 million rows — execution times are often just a few seconds, with an almost systematic OK status.

Governance and data lineage

Once ETL processes have been converted to SQL, {oa.tbx} keeps the complete trace between the original logic and the executed SQL. This data lineage is exposed at two levels of detail, suited to different needs.

Key point: a synthetic view for governance, an exhaustive detail view for maintenance.

High-level pipeline view: only source and target tables are visible; intermediate ETL stages are grouped into a single SQL block. A readable view, even for complex pipelines.

Detail on click for a transformation: clicking a stage (a Transformer, for example) displays its generated SQL equivalent, along with its inputs/outputs and its original metadata (type, stage ID). The mapping between the original ETL logic and the executed SQL is immediate.

Result: complete governance of the migrated pipeline, from the overview down to the executed SQL, regardless of the original ETL platform.

SQL as the pivot format

The SQL produced by {openAudit} becomes the primary vehicle for the transformations.

Business rules, joins, aggregations, and calculations are expressed in an open, version-controllable format, independent of legacy ETL tools.

Result: a SQL-based ETL/ELT chain, orchestrated by workflows and run in an open environment.

DataViz Migration - an upstream/downstream SQL interoperability layer - {oa-lake}

Fri, 03 Jul 2026 00:00:00 +0000

Decision-support platforms often rely on several databases, several data warehouses / data lakes, and several reporting tools.

{oa-lake} introduces an intermediate SQL layer between data sources and dataviz tools.

Business rules, business objects, dimensions, measures, and calculations are centralized in this layer and then exposed to consuming tools.

Key point: data sources and reporting tools become independent of one another.

Architecture

{oa-lake} connects simultaneously to multiple data systems: relational databases, data warehouses, lakehouses, files, or APIs.

Queries are federated by a distributed SQL engine that performs the necessary joins, calculations, and transformations.

The resulting SQL remains standard and executable across different engines, which limits technology lock-in.

Business objects are defined once and then exposed to consuming tools.

Data sources

The data stays in its original systems.

Oracle, SQL Server, PostgreSQL, DB2, BigQuery, Snowflake, Redshift, Azure SQL (and others), or files can be queried within a single request.

The SQL Push-Down mechanism avoids any systematic copying of data: queries run directly against the existing systems, respecting the architectures already in place.

Typical use case: build a dashboard from data spread across several platforms with no prior migration.

SQL business layer

Dimensions, measures, hierarchies, and calculation rules are centralized in {oa-lake}.

This layer becomes the single entry point for reporting tools.

Business rules are no longer duplicated across every dataviz platform.

Key point: a single definition of indicators for the entire decision-support system.

Parquet cache

Data can be materialized in Parquet format.

This caching layer reduces response times and limits load on source systems. It’s managed automatically based on observed usage, with no manual intervention.

Typical use case: speed up the most expensive analytical processing.

Data visualization

Reporting tools connect to the model exposed by {oa-lake}.

Power BI, Looker, Spotfire, Tableau, Qlik, SAP BusinessObjects, or any SQL-compatible tool can use the same business objects.

Switching dataviz tools no longer requires rebuilding dimensions, measures, and calculation rules.

Progressive migration

Several tools can coexist during a transition phase.

The same business model can be exposed simultaneously to several reporting platforms.

Result: progressive migration of usage with no change to the data systems.

Data Lineage

The data lineage reconstructed by {openAudit} remains available across the whole chain.

Dependencies are documented from the source systems all the way to the indicators exposed in the dashboards.

Key point: transformations, calculations, and usage remain traceable across the entire architecture.

DataViz Migration - an interoperable semantic layer - {oa-lake-ctx}

Thu, 02 Jul 2026 00:00:00 +0000

Legacy decision-support platforms typically relied on a semantic layer that let users work with business objects rather than tables or SQL queries.

“Customers”, “contracts”, “products”, or “revenue” were exposed in a functional form, independent of the underlying physical storage structures.

To address this need, we designed {oa-lake-ctx}.

{oa-lake-ctx} is the semantic layer of the {oa-lake} architecture. It centralizes business objects, dimensions, measures, KPIs, calculation rules, and security mechanisms.

Key point: consuming tools work with business objects. The physical structures stay hidden.

Architecture

{oa-lake-ctx} centralizes dimensions, measures, hierarchies, KPIs, and calculation rules.

These components are defined once and then exposed to consuming tools.

Unlike some semantic layers on the market, {oa-lake-ctx} requires no specific physical model, no pivot table, and no imposed navigation path.

Relationships are calculated dynamically from the existing model.

Key point: a single business model for all data and all consuming tools.

Data sources

{oa-lake-ctx} can use several data systems simultaneously.

Data can come from Oracle, SQL Server, PostgreSQL, DB2, Snowflake, BigQuery, Redshift databases, files, or other SQL-compatible sources.

Business objects can combine several sources within a single functional definition.

Typical use case: build a KPI from data spread across several platforms.

Business objects

The catalogs exposed by {oa-lake-ctx} group together dimensions, measures, KPIs, hierarchies, and business nomenclature.

Relationships between objects are defined in the semantic layer, not in the reporting tools.

Catalogs can be organized independently of the databases’ physical structures.

SQL generation

SQL queries are generated dynamically from the selected business objects, dimensions, measures, and filters.

Joins, aggregations, and calculation rules are applied by the engine.

User filters are propagated all the way to the source systems via the SQL Push-Down mechanism: no systematic copying of data — queries run against the existing systems, respecting the architectures already in place.

The generated SQL can be executed by {oa-lake} or by any compatible SQL engine.

Key point: users work with business objects; the SQL is generated automatically.

Smart Cache

Data stays in the source systems whenever possible.

The most heavily used datasets can be materialized in Parquet format.

The cache is driven by {oa-lake}’s Cache Manager, managed automatically based on observed usage.

Cache management remains independent of consuming tools.

Key point: the most expensive queries can be accelerated without modifying the source systems.

SQL Engine

{oa-lake} relies on a columnar, in-memory SQL engine.

The queries generated by {oa-lake-ctx} are executed by this engine or propagated to the source systems depending on the context.

The SQL remains portable, standard, and executable on different engines, independent of the storage technologies in use — which limits technology lock-in.

Key point: the SQL engine forms the execution layer of the architecture.

Security

Security is managed at the semantic layer level.

Access restrictions, user profiles, business catalogs, and security filters are applied before queries are generated.

This approach makes it possible to reproduce the security mechanisms found in platforms such as SAP BusinessObjects or IBM Cognos.

Key point: security rules are defined once and then applied to all connected tools.

Reporting

The catalogs exposed by {oa-lake-ctx} can be consumed by any tool capable of querying a PostgreSQL source.

Power BI, Looker, Spotfire, Tableau, Qlik, or other platforms access the same business objects, dimensions, measures, and KPIs.

The functional definitions remain identical regardless of the tool being used.

Result: several platforms can share the same business catalog with no duplication of calculation rules.

Governance

Business objects, KPIs, dimensions, measures, and calculation rules are centralized in a single repository.

Functional changes are made in {oa-lake-ctx} and are then immediately available to all connected tools.

Key point: {oa-lake-ctx} is the central point for defining business objects and KPIs.

Data lineage - connection protocols to the various analytics technologies

Thu, 02 Jul 2026 00:00:00 +0000

Metadata acquisition relies on the native interfaces exposed by the technologies analyzed by {openAudit}.

Depending on the platform, collection uses database protocols, REST APIs, SDKs, technical repositories, or standardized exports.

Key point: the collectors use the mechanisms officially exposed by the vendors (JDBC, ODBC, REST API, SDK, technical repositories, exports).

Technical principles

Dedicated service accounts.
Access limited to metadata.
On-premise, Cloud, and hybrid compatibility.
Use of platforms’ native interfaces.
Password encryption after first use.

Databases

Relational databases are analyzed via:

JDBC.
ODBC.
Proprietary connectors.

Collection covers:

Schemas.
Tables.
Views.
Stored procedures.
Usage statistics (when analyzed).

ETL / ELT

ETL/ELT platforms are analyzed via:

Technical repositories.
Administration APIs.
XML or JSON exports.

The collected metadata makes it possible to reconstruct:

Jobs.
Transformations.
Dependencies.
Data flows.

Data Visualization Tools

Data visualization tools are analyzed via:

SDKs.
REST APIs.
Internal repositories.

Collection covers:

Data sources.
Semantic layers.
Dashboards.
Business rules.

This information makes it possible to extend the data lineage all the way to the reporting layer.

Typical use case: build data lineage in the sources and connect it with the data lineage in the data visualization layer, all the way to the dashboard cell.

Secure environments

Collection can be securely supervised via:

VMware Horizon.
OpenOTP (two-factor authentication).
Administration bastion.
Logging and traceability of operations.

Access remains limited to the components needed for metadata collection.

For information: the same collection mechanism applies in Cloud, on-premise, or hybrid environments.

Data lineage - feeding third-party data catalogs to expose technical data lineage

Thu, 02 Jul 2026 00:00:00 +0000

{openAudit} lets you browse data lineage through its Mosaïc web portal, but the consolidated metadata can also be published to various data catalogs on the market, which often have significant limitations when it comes to data lineage. The benefit is having a single tool for optimal data governance.

The data lineage is reconstructed in the {openAudit} technical repository, then exposed to data catalogs via dedicated connectors.

This architecture makes it possible to decouple the data lineage collection and reconstruction mechanisms from the exposure layer used by governance teams.

Key point: the technical repository stays independent from the target catalog. The collection and lineage-reconstruction (and connection) mechanisms don’t depend on the data catalog in use.

Architecture

The collected metadata is consolidated in the oA.mdb columnar database after processing by the oA.Job engine.

The central repository notably contains:

Field-level data lineage.
Usage data.
Associated governance information, where applicable.

The catalogs are then fed from this single repository via exposure connectors independent of the data catalog solutions in use.

Exposure connectors

The connectors handle:

Automated publication of the lineage.
Continuous synchronization.
Restriction management.
Support for the various Workspaces.

Data lineage exposure can be complete or partial depending on the display capabilities of the target catalog, keeping in mind that the data lineage produced upstream is itself available at field level.

Result: catalogs only consume the consolidated metadata, not the original technical sources.

Supported catalogs

The available connectors currently support the following data catalogs:

DataGalaxy
Zeenea
Collibra

Usage management

When audit logs are available, observed usage is integrated into the technical repository and then exposed to the catalog.

This lets users distinguish:

“Live” flows, consumed by tools or queried.
Flows — and therefore objects — that see no consumption (often more than 80% of the total).

Typical use case: enrich the data catalog with usage information to identify objects that are candidates for rationalization or decommissioning.

Data lineage - on-premise architecture

Thu, 02 Jul 2026 00:00:00 +0000

In the most demanding environments in terms of security (banking, insurance, etc.), {openAudit} can be deployed directly within the Client’s environment, whether on-premise or in a Cloud tenant of the Client’s choice.

The architecture relies on a dedicated VM hosting the various components used to collect, process, and expose the data lineage.

Key point: all processing, metadata, and repositories stay hosted within the Client’s environment. No external SaaS component is required.

Technical principles

Dedicated VM hosted at the Client’s site.
Collectors containerized per technology.
Dedicated service accounts.
Password encryption after first use.
Production of standardized CSV files.
Parsing and consolidation by oA.Job.
Technical repository oA.mdb (DuckDB).
Exposure via the Mosaïc web portal.
Audit logs that may contain sensitive information can be anonymized before processing.

Good to know: this data lineage can feed a third-party solution, typically a data catalog that needs to address data lineage more specifically.

Metadata collection

Metadata is collected using containers specialized by technology. These components connect to databases, ETL/ELT platforms, data visualization tools, and audit databases via dedicated service accounts. Passwords are encrypted after first use.

The collectors produce standardized CSV files containing the metadata needed to reconstruct the lineage. These files can be reviewed and validated by the Client before processing.

Where audit logs are used, they can be anonymized through a dedicated container before being integrated into the repository.

Processing

The files are then sent to the oA.Job server, responsible for parsing and consolidating the metadata. The results are stored in the oA.mdb columnar database (DuckDB), which forms the technical repository used by the Mosaïc web portal.

This repository can be dumped in the format requested by the client to run specific analyses: the repository is extremely rich, and various pre-built queries are made available to make exploring it easier.

Exposure

The data lineage can be browsed via the proprietary Mosaïc web portal, hosted on the dedicated VM.

Overall flow mapping

Automatic reconstruction of dependencies between databases, ETL/ELT processes, and usage in data visualization tools.

Typical use case: impact analysis before modifying a table or a business indicator.

Usage analysis

Cross-referencing data lineage with audit logs to identify objects that are actually used and branches that see no consumption.

Result: quick identification of unused flows and simplification opportunities.

Detailed data lineage

Navigation through the data lineage down to field level, with visualization of the transformations (drill-through).

Key point: instantly drill into the code driving the transformations to identify and fix an issue in a flow.

Data lineage in dashboards

Complete traceability between source data, business rules, and the dashboard cell. Dashboard source data is linked into the data lineage as far back as its feeds, for end-to-end impact analysis.

Typical use case: validating a business indicator or tracing the origin of a value displayed in a dashboard.

Data lineage - SaaS architecture

Thu, 02 Jul 2026 00:00:00 +0000

In a SaaS deployment, the collection components remain hosted within the Client’s environment, while the processing, consolidation, and exposure components are hosted on dedicated infrastructure operated by Ellipsys.

The extractors are deployed as containers specialized by technology. They collect metadata from databases, ETL/ELT platforms, data visualization tools, and audit databases using dedicated service accounts.

The collected metadata is produced as standardized CSV files. These files can be reviewed by the Client before being transferred to Ellipsys’s infrastructure.

The files are then sent to the oA.Job server, responsible for parsing and reconstructing dependencies. Metadata consolidation happens in the oA.mdb technical repository, based on DuckDB (a columnar database).

The data lineage is then exposed via the proprietary Mosaïc web portal or via third-party solutions, typically off-the-shelf data catalog products.

Where audit logs are used, they can be anonymized through a dedicated container before transfer.

Key point: the metadata needed to reconstruct the lineage is transferred to the SaaS platform in CSV format and can be reviewed beforehand.

Technical principles

Containerized extractors deployed within the Client’s environment.
Service accounts dedicated to collection.
Password encryption after first use.
Production of standardized CSV files.
Optional anonymization of audit logs.
Parsing and consolidation via oA.Job.
Source database for the data lineage: oA.mdb (DuckDB).
Exposure via the Mosaïc web portal or in data catalog solutions.

Good to know: business data is never used. Only technical metadata is collected.

Metadata collection

The collectors can analyze:

Databases.
ETL/ELT platforms.
Data visualization tools.
Audit databases.

Each technology is handled by a dedicated component.

Processing

The oA.Job server performs:

Metadata parsing.
Dependency reconstruction.
Consolidation of technical objects.
Integration of usage data from audit logs.

The results are stored in the oA.mdb repository.

Exposure

The data lineage can be viewed:

Via the Mosaïc web portal.
Via exposure of {openAudit}’s data lineage into third-party solutions.
Via the Client’s own integration mechanisms.

Key point: the collection and lineage-reconstruction mechanisms are independent from the exposure mechanisms.

Overall flow mapping

Mapping of dependencies between databases, ETL/ELT processes, and data usage.

Typical use case: impact analysis before modifying a technical object.

Usage analysis

Cross-referencing data lineage with audit logs to identify flows that are actually used.

Result: detection of unused flows, i.e. dead branches.

Detailed data lineage

Navigation through the data lineage down to field level, with visualization of intermediate transformations.

Key point: dependencies can be explored iteratively, from operational sources all the way to usage.

Data lineage in dashboards

Traceability between source data, transformations, calculation rules, and reporting objects.

Typical use case: tracing the origin of a business indicator or validating a business rule.

Data lineage - the principles of connector containerization

Thu, 02 Jul 2026 00:00:00 +0000

The {openAudit} architecture relies on containerized components deployed within the Client’s environment.

Each component is specialized for a given technology and handles the collection of the metadata needed to reconstruct the data lineage: databases, ETL/ELT platforms, data visualization tools, and audit databases.

The extractors produce standardized CSV files containing the collected metadata. These files can be reviewed by the Client before being transferred to the central processing components.

Where anonymization is required, a dedicated container is inserted into the collection chain to replace strings, SQL constants, and other sensitive information with variables before the exchange files are produced.

Key point: extractors and anonymizers are decoupled from the processing engines. Each component can be deployed and updated independently.

Technical principles

One dedicated container per technology.
Metadata collected as close as possible to the analyzed systems.
Optional anonymization before transfer.
Production of standardized CSV files.
Docker and Podman compatibility.
On-premise, SaaS, or hybrid deployment.
Independent updates for each component.

Extraction

The extractors connect to the analyzed technologies using dedicated service accounts.

The metadata collected relates to:

Databases.
ETL / ELT platforms.
Data visualization tools.
Audit databases.
The scheduler.

Each extractor produces a consistent output format regardless of the technology being analyzed.

Anonymization

An anonymization container can be positioned between metadata collection and transfer.

The following elements can be replaced with technical variables:

Strings.
Application parameters.
Literal values.

The goal is to preserve technical dependencies while removing sensitive information.

Key point: anonymization happens before the transferred files are generated.

Exchange files

The components produce standardized CSV files containing:

Technical objects.
Dependencies.
Transformations.
Usage information, where available.

These files can be archived, reviewed, or validated by the Client before being transferred.

Processing

The generated files are used by {openAudit}’s central components to:

Reconstruct dependencies.
Consolidate metadata.
Integrate usage from audit logs.
And produce the data lineage.

Result: a consistent collection chain, independent of the analyzed technologies and deployment modes.

DataViz Migration - migrating from Cognos to Power BI

Thu, 02 Jul 2026 00:00:00 +0000

IBM Cognos Analytics platforms typically hold several years’ worth of business modeling: Frameworks, hierarchies, calculations, dimensions, measures, variables, and reports.

{openAudit} analyzes Cognos Frameworks, reports, and usage logs to reconstruct the dependencies between data sources, business objects, and reporting outputs.

Key point: in most Cognos estates, the value lies more in the Frameworks and business rules than in the reports themselves.

Analysis of the Cognos estate

The analysis covers Framework Manager, reports, queries, dimensions, hierarchies, calculations, filters, and variables.

The collected metadata makes it possible to reconstruct:

dependencies between objects;
relationships with data sources;
business objects and indicators exposed in reports.

Scope rationalization

Usage logs are cross-referenced with the platform’s metadata.

This phase identifies unused reports, duplicates, and obsolete Frameworks.

Result: the scope that’s actually used is isolated before migration.

Conversion to Power BI

Cognos Frameworks are analyzed and then converted to the target Power BI model.

Dimensions, hierarchies, measures, calculations, variables, and relationships between objects are reconstructed in the target semantic model.

Cognos expressions are converted to SQL or DAX depending on their nature.

Dependencies between Frameworks, reports, and data sources are preserved in the generated model.

Key point: business objects and calculation rules are reconstructed directly from the Cognos Frameworks.

Validation

Migrations can be carried out with dual Cognos / Power BI execution.

The indicators produced by both platforms are compared during acceptance testing phases.

Typical use case: progressive validation of reports before cutover.

Usage-driven migration

Reports can be migrated in batches based on how often they’re used and their business criticality.

Observed usage forms the basis for project planning and development prioritization.

Result: a reduced volume to migrate, with effort focused on active usage.

DataViz Migration - migrating from SAP BO to GCP Looker

Thu, 02 Jul 2026 00:00:00 +0000

SAP BusinessObjects estates typically hold several years’ worth of modeling: universes, Web Intelligence reports, business objects, calculations, filters, prompts, and semantic layers.

{openAudit} analyzes these components to reconstruct the dependencies between data sources, universes, reports, and observed usage.

Key point: universes and business objects are generally the primary source of information for reconstructing the target model.

SAP BusinessObjects analysis prerequisites

The analysis relies on access to the SAP BusinessObjects services, technical repositories, and the audit database.

The extractors run within the Client’s environment. The collected metadata can be anonymized before processing.

Good to know: only technical metadata is used.

Estate analysis

The analysis covers Web Intelligence reports, UNV and UNX universes, Data Providers, business objects, variables, filters, prompts, and graphical components.

Technical dependencies and observed usage are consolidated into a repository.

Rationalization

Audit logs are cross-referenced with the platform’s metadata.

Unused or broken reports, duplicated reports, unconsumed objects, obsolete universes, etc. are identified before migration.

Result: a massive reduction in the volume to migrate.

Conversion to Looker

Data Providers, business objects, and queries are converted into Looker components.

Dimensions, measures, joins, and calculations are reconstructed in the LookML models.

Dashboards, filters, prompts, tables, charts, and navigation mechanisms are reconstructed from the SAP BusinessObjects reports.

The functional organization of the universes carries over into the target platform’s folders, Explores, and business catalogs.

Key point: functional objects and reporting components are reconstructed from the SAP BusinessObjects universes and reports.

Validation

Migrations can be carried out with dual SAP BusinessObjects / Looker execution.

The indicators produced by both platforms are compared during acceptance testing phases.

SAP BusinessObjects prompts

SAP BusinessObjects prompts are converted into Looker’s native filters.

Value lists and input parameters are preserved.

Business objects and LookML models

Queries, business objects, dimensions, measures, and joins are converted into LookML components.

Functional structure

The organization of the universes is rebuilt as Explores, dimensions, and measures.

Pivots and matrix analyses

Cross-tabs and multidimensional analyses are converted into Looker’s pivot mechanisms.

Charts

Trend lines and comparative analyses are rebuilt from the indicators identified in the reports.

Bar charts

Bar charts are adapted to Looker’s visualization components.

Pie charts

Pie charts are converted into Looker’s native visualizations.

DataViz Migration - migrating from SAP BO to Power BI

Thu, 02 Jul 2026 00:00:00 +0000

SAP BusinessObjects estates typically hold several years’ worth of modeling: universes, Web Intelligence reports, business objects, calculations, filters, prompts, and semantic layers.

{openAudit} analyzes universes, reports, and usage logs to reconstruct the dependencies between data sources, business objects, and visualizations.

Key point: the migration covers universes, business objects, and business rules just as much as reports.

SAP BusinessObjects analysis prerequisites

The analysis requires access to the SAP BusinessObjects services, technical repositories, and the audit database.

The extractors run within the Client’s environment. The collected metadata can be anonymized before processing.

Good to know: only technical metadata is used.

Estate analysis

The analysis covers Web Intelligence reports, UNV and UNX universes, Data Providers, business objects, variables, filters, prompts, and graphical components.

Technical dependencies and observed usage are consolidated into a repository.

Rationalization

Audit logs are cross-referenced with the platform’s metadata.

Unused reports, duplicates, unconsumed objects, and obsolete universes are identified before migration.

Result: a reduced volume to migrate.

Conversion to Power BI

Data Providers are converted into Power BI datasets.

Queries, calculations, and variables are converted into SQL or DAX. Business objects are reconstructed as dimensions and measures.

Dependencies between universes, reports, and data sources are preserved in the target model.

Tables, charts, filters, sections, and layout elements are analyzed and then reconstructed in Power BI.

Key point: the migration covers both the business layer and the reporting layer.

Validation

Migrations can be carried out with dual SAP BusinessObjects / Power BI execution.

The indicators produced by both platforms are compared during acceptance testing phases.

Tables, filters, and slicers

Conversion of SAP BusinessObjects tables, filters, and sections into Power BI components.

Pie charts

Conversion of SAP BusinessObjects pie charts into Power BI visuals.

Charts

Conversion of SAP BusinessObjects charts into Power BI visuals.

Dashboards

Conversion of SAP BusinessObjects dashboards into Power BI.

DataViz Migration - SAP BO, Power BI, Spotfire, or Cognos to an open, interoperable architecture

Thu, 02 Jul 2026 00:00:00 +0000

Data visualization platforms evolve regularly.

SAP BusinessObjects, IBM Cognos, Power BI, or Spotfire may succeed one another within the same Information System. Business objects, calculations, indicators, and business rules generally remain identical from one platform to the next.

{openAudit} analyzes technical repositories, semantic layers, and reports to reconstruct the dependencies between data sources, intermediate processes, and the objects exposed in dashboards.

Key point: the functional components of reporting platforms are extracted and then rebuilt in a format independent of the target technology.

Analysis of the decision-support estate

The parser-based analysis mechanisms cover SAP BusinessObjects universes, Cognos Frameworks, Power BI models, Spotfire Information Links, as well as the associated reports, dashboards, calculations, filters, and business objects.

The collected metadata makes it possible to reconstruct technical dependencies, semantic layers, and business rules.

Rationalization

Technical repositories are cross-referenced with usage logs. Unused reports, duplicates, and obsolete components are identified before the migration phase.

Result: a reduced scope for the actual migration.

SQL transcription

Queries, calculations, filters, aggregations, joins, variables, and business rules are reconstructed in the form of SQL.

The resulting SQL forms an open representation of the functional logic originally carried by the data visualization platform.

Key point: SQL becomes the pivot format for representing business objects.

{oa-lake}

The SQL processes are integrated into {oa-lake}.

{oa-lake} centralizes business objects, indicators, dimensions, business rules, and SQL transformations. The engine federates several data systems and exposes a single model to reporting tools.

Data can be materialized in Parquet format. Queries are generated against the physical systems or executed from the cache when needed.

Key point: business logic is defined once and then exposed to several data visualization platforms.

ETL Migration - from legacy ETL to dbt

Thu, 02 Jul 2026 00:00:00 +0000

Migrating to dbt means transforming proprietary ETL processes into SQL models run directly in the target data engine.

Before any conversion, {openAudit} analyzes flows, dependencies, and usage to identify the scope that is actually worth migrating.

Key point: the migration starts with a rationalization phase of the existing estate.

Flow analysis

ETL processes are analyzed to reconstruct dependencies, transformations, and usage.

Data lineage is used to identify active flows, dead branches, and unconsumed components.

This phase reduces the volume of processes to be migrated.

Result: a migration scope refocused on the processes that are actually used.

Conversion to SQL

ETL jobs are broken down into elementary operations: filters, joins, aggregations, lookups, business calculations, and transformations.

These components are converted into documented SQL, independent of the source technology.

SQL becomes the pivot representation of the processes.

Key point: business logic is rebuilt in SQL before the dbt project is generated.

Generating dbt models

The SQL processes are organized into dbt models.

Dependencies between models are automatically generated from the flows identified during the analysis.

The project structure, references between models, and documentation can be produced automatically.

Progressive validation

Migration is carried out in batches.

The generated models are validated and then progressively integrated into the target environment.

Discrepancies can be checked process by process during the transition phase.

Typical use case: progressive migration of several hundred ETL jobs to dbt.

Target architecture

The processes become version-controllable, maintainable SQL models.

Dependencies are explicit, transformations are visible, and processes can be run directly by Snowflake, BigQuery, Databricks SQL, Redshift, PostgreSQL, or any other compatible engine.

Key point: the processes no longer depend on the legacy ETL platform but on a dbt project built on open SQL.

ETL Migration - from legacy ETL to SQL

Thu, 02 Jul 2026 00:00:00 +0000

Legacy ETL platforms mainly implement filters, joins, aggregations, mappings, and transformations that can be expressed in SQL.

{openAudit} analyzes ETL jobs, reconstructs the flows, and generates a SQL equivalent independent of the source technology.

Key point: the processes are rebuilt as documented, version-controllable SQL.

Process analysis

ETL repositories are analyzed to reconstruct sources, targets, transformations, parameters, and dependencies between processes.

The collected metadata makes it possible to reconstruct how the flows actually work.

Unused or unconsumed components can be identified before migration.

Result: a migration scope reduced to the processes that are actually used.

Flow reconstruction

Processing chains are reconstructed from the ETL jobs.

Dependencies between processes, intermediate flows, variables, and parameters are integrated into the target model.

This mapping is the entry point for the conversion.

Rationalization

Cross-referencing data lineage (down to field level) with actually observed usage makes it possible to identify the data that is genuinely consumed and set aside what isn’t.

Each job is also analyzed to assess its complexity: number of steps, depth of sub-processes, types of transformations, and external dependencies. These metrics are aggregated into a complexity score, used to prioritize and plan the migration.

Result: a migration scope reduced to the processes that are actually used, prioritized according to their real complexity.

SQL generation

The detected transformations are converted into SQL.

Filters, joins, aggregations, lookups, mappings, and calculations are translated into successive, flat SQL statements.

The resulting SQL preserves the logical structure of the original processes.

Key point: each processing step remains identifiable in the generated SQL.

Example: IBM DataStage jobs

For a DataStage estate, each stage type is translated according to its own logic:

Runtime Column Propagation (RCP): the column list isn’t fixed in the job; it’s generated dynamically in the SQL, driven by the variables passed to the job.
Oracle connectors (read / write): SELECT, UPDATE, MERGE, or INSERT generated from the stage’s options.
Change Capture (CDC): Before/After comparison on primary keys, detecting new, modified, and deleted rows.
CDC Transformer: conditional routing to a dedicated flow based on the detected change code.
Shared Container: a factored logical block, rendered as a standardized SQL block reused across several jobs.
Column Generator: calculated columns translated into typed SQL expressions.
Modify: type conversions translated into SQL CAST (e.g. string to timestamp).
Funnel: merging several flows via SQL UNION, with output schema harmonization.

Observed result: across an estate of several hundred DataStage jobs, more than 95% of the SQL translations succeed on the very first run.

Adapting to target platforms

The generated SQL can be adapted to the dialects of the target platforms.

Functions, data types, and specific mechanisms are taken into account during generation.

The same processes can be produced for Snowflake, BigQuery, Databricks SQL, Redshift, PostgreSQL, SQL Server, or other SQL-compatible engines.

Orchestration

Dependencies and process scheduling are preserved.

The generated processes can be integrated into the orchestration tools already present in the Information System.

Typical use case: integration into Airflow, Control-M, or any other existing scheduler.

SQL as the pivot format

SQL becomes the common representation of the processes.

Transformations no longer depend on the proprietary components of the original ETL platform.

Processes can be versioned, reviewed, tested, and deployed independently of the source technology.

Key point: the transformation logic stays under the company’s control and no longer depends on a specific ETL engine.

ETL Migration - legacy ETL connectors for SQL migration

Thu, 02 Jul 2026 00:00:00 +0000

ETL platforms rely on connectors, transformation components, parameterization mechanisms, and execution strategies specific to each vendor.

{openAudit} analyzes these components, reconstructs their behavior, and then generates a SQL representation independent of the source technology.

Key point: the conversion is based on analyzing the ETL components, not on a simple syntactic translation.

Component analysis

Technical repositories are analyzed to identify connectors, transformations, parameters, dependencies, and execution mechanisms.

The components are converted into a common intermediate model before SQL generation.

This approach is identical regardless of the source technology.

Connectors

Read and write connectors are analyzed to reconstruct the flows between systems.

Connection parameters, queries, data formats, and mappings are integrated into the migration model.

Flows can originate from relational databases, files, APIs, or Cloud platforms.

Typical use case: reconstructing exchanges between Oracle, SQL Server, PostgreSQL, flat files, or Cloud storage.

Transformations

Transformation components are converted into SQL instructions.

Filters, joins, aggregations, lookups, mappings, type conversions, column calculations, and conditional expressions are reconstructed in the target SQL model.

Key point: each transformation becomes an explicit SQL operation.

Parameterization

Environment variables, job parameters, shared libraries, and reusable components are analyzed during the conversion.

Dependencies between processes and parameterization mechanisms are preserved in the generated model.

Supported technologies

The migration engines cover in particular IBM DataStage, Talend, SAP BODS, SSIS, ODI, Informatica PowerCenter, Stambia, Semarchy xDI, and Ab Initio.

The methodology stays the same: component analysis, flow reconstruction, then SQL generation.

Result: a consistent approach applied to different ETL technologies.

SQL as the pivot format

Once the components have been reconstructed, the processes are converted into a SQL model independent of the source platform.

The generated SQL can then be adapted to Snowflake, BigQuery, Databricks SQL, Redshift, PostgreSQL, SQL Server, or any other compatible engine.

Key point: SQL becomes the common format for representing the processes.

Privacy Policy

Thu, 02 Jul 2026 00:00:00 +0000

This page describes how Ellipsys processes the personal data of visitors to this site.

Data controller

Ellipsys is the data controller for the data collected through this site. For any question, you can contact us via the form on the Contact page.

Data collected

This site does not set any cookies and does not use any tracking or audience analytics tools.

The only personal data collected is the data you voluntarily submit via the contact form (name, email address, company, message). This information is used solely to respond to your request.

Data retention

Data submitted via the contact form is kept for as long as needed to process your request, and then deleted.

Your rights

Under the General Data Protection Regulation (GDPR), you have the right to access, rectify, and erase your personal data. To exercise these rights, contact us via the Contact page.

Changes

This privacy policy may be updated from time to time. The date of the last update appears at the top of this page.

{oa-lake} — interoperability layer

Wed, 01 Jul 2026 00:00:00 +0000

{oa.tbx} — a SQL ETL

Wed, 01 Jul 2026 00:00:00 +0000

About

Wed, 01 Jul 2026 00:00:00 +0000

Contact

Wed, 01 Jul 2026 00:00:00 +0000

{openAudit} - Migration

Tue, 30 Jun 2026 00:00:00 +0000

{openAudit} - Data Lineage

Mon, 29 Jun 2026 00:00:00 +0000

Terms of Service

Fri, 01 Dec 2023 00:00:00 +0000

Add your company legal terms here…