ETL Migration - legacy ETL connectors for SQL migration

ETL platforms rely on connectors, transformation components, parameterization mechanisms, and execution strategies specific to each vendor.

{openAudit} analyzes these components, reconstructs their behavior, and then generates a SQL representation independent of the source technology.

Key point: the conversion is based on analyzing the ETL components, not on a simple syntactic translation.

Component analysis

Technical repositories are analyzed to identify connectors, transformations, parameters, dependencies, and execution mechanisms.

The components are converted into a common intermediate model before SQL generation.

This approach is identical regardless of the source technology.

Connectors

Read and write connectors are analyzed to reconstruct the flows between systems.

Connection parameters, queries, data formats, and mappings are integrated into the migration model.

Flows can originate from relational databases, files, APIs, or Cloud platforms.

Typical use case: reconstructing exchanges between Oracle, SQL Server, PostgreSQL, flat files, or Cloud storage.

Transformations

Transformation components are converted into SQL instructions.

Filters, joins, aggregations, lookups, mappings, type conversions, column calculations, and conditional expressions are reconstructed in the target SQL model.

Key point: each transformation becomes an explicit SQL operation.

Parameterization

Environment variables, job parameters, shared libraries, and reusable components are analyzed during the conversion.

Dependencies between processes and parameterization mechanisms are preserved in the generated model.

Supported technologies

The migration engines cover in particular IBM DataStage, Talend, SAP BODS, SSIS, ODI, Informatica PowerCenter, Stambia, Semarchy xDI, and Ab Initio.

The methodology stays the same: component analysis, flow reconstruction, then SQL generation.

Result: a consistent approach applied to different ETL technologies.

SQL as the pivot format

Once the components have been reconstructed, the processes are converted into a SQL model independent of the source platform.

The generated SQL can then be adapted to Snowflake, BigQuery, Databricks SQL, Redshift, PostgreSQL, SQL Server, or any other compatible engine.

Key point: SQL becomes the common format for representing the processes.