Legacy ETL/ELT and data visualization tools have accumulated a tremendous amount of complexity over time.
Some of these technologies are outdated and no longer meet the requirements of modern data stacks: scalability,
ease of use, and interoperability.
Migration is necessary. Unfortunately, the sheer volume of data and the inherent complexity of most of these technologies make
these projects excessively costly and risky.
{openAudit}, by leveraging the extreme granularity of its reverse engineering capabilities, enables "as is" migrations
for ETL/ELT and certain data visualization technologies in an automated way.
This allows for faster time to market, reduced migration costs, and greater buy-in from business users.
We have developed a unique methodology capable of deconstructing the business logic encapsulated in ELT/ETL jobs from numerous legacy solutions (BODS, DataStage, SSIS, ODI, Informatica, AB Initio, Genio, etc.) and converting it into pure or enriched SQL. This logic can then be executed directly on modern databases (GCP, Azure SQL, etc.). In addition, {openAudit} can rebuild all these instructions within modern data transformation technologies (for example, dbt).
Step-by-step migrations from ETL / ELT to SQL make it possible to break down complex processes into readable and maintainable blocks by flattening the nested SQL of legacy systems. This allows for explicit control points and intermediate tables to track and better understand the data flow.
The transformations generated by {openAudit} are converted into enriched SQL, either to adapt directly to the target database (GCP, Redshift), or to be encapsulated within the target technology (such as dbt). Orchestration via Dagster, for example, makes it possible to visualize dependencies, schedule, and monitor pipeline executions. This approach reduces latency, improves maintainability, and fully leverages modern data warehouses.
{oa-lake} acts as an orchestration layer between data systems and visualization tools. Based on a distributed SQL engine and Parquet storage, it centralizes business logic, reduces the load on source systems, and simplifies end-to-end data flow governance.
{openAudit}, by relying on the exceptional introspection capabilities of certain data visualization technologies, makes it possible to automate the process of “reconstruction” of the platform in the target technology: intelligence and layout. In particular, we have extensive experience in migrations between SAP BO and Power BI from Microsoft, or SAP BO to Looker from Google.
{openAudit} will automatically recover all of the instructions that are passed into the DataViz technology at the source, at the intelligence or layout level.
{openAudit} reconstructs intelligence in a fully automated way based on the target technology: variables, expressions, etc. The structure of the initial dashboard is also reproduced in the target technology.
Part of the intelligence from the source dashboards is moved to the target platform's data warehouse, to streamline data preparation and thus significantly reduce complexity, but also to substantially improve response times.
{openAudit} runs the dashboards in bulk, before migration and after migration, then compare the results (content and form) to validate the quality of the migration.