ETL Migration - from legacy ETL to dbt
Migrating to dbt means transforming proprietary ETL processes into SQL models run directly in the target data engine. The migration starts with a rationalization of the existing estate.
Flow analysis
{openAudit} reconstructs the dependencies, transformations, and usage of ETL processes; data lineage identifies active flows, dead branches, and unconsumed components.
Result: a migration scope refocused on the processes that are actually used.
Conversion to SQL
ETL jobs are broken down into elementary operations (filters, joins, aggregations, lookups, business calculations, transformations), converted into documented SQL independent of the source technology, before the dbt project is generated.
Generating dbt models
The SQL processes are organized into dbt models. Dependencies between models are generated from the flows identified during the analysis; project structure, references between models, and documentation can be produced automatically.
Progressive validation
Models are generated, validated, and integrated in batches into the target environment, with discrepancies checked process by process during the transition.
Typical use case: progressive migration of several hundred ETL jobs to dbt.
Target architecture
The processes become version-controllable, maintainable SQL models with explicit dependencies, run directly by Snowflake, BigQuery, Databricks SQL, Redshift, PostgreSQL, or any other compatible engine, with no dependency on the legacy ETL platform.
{oa-tbx}: the recommended target
Rewriting DataStage, PowerCenter, BODS, Talend, ODI, or SSIS jobs in another proprietary ETL means renewing the licenses, the dedicated infrastructure, and the vendor lock-in.
{oa-tbx} is an open, serverless SQL ETL with an embedded DuckDB engine. Its core is free; advanced features are licensed. The jobs automatically transcribed into SQL by {openAudit} run there as pipelines orchestrated by Airflow, with no dedicated infrastructure.
dbt and {oa-tbx} are complementary, not competing: {oa-tbx} handles extraction and loading (the E and the L of ETL), while the generated dbt models carry the transformation (the T), lock-in free, executed by the embedded columnar / in-memory SQL engine of {oa-tbx}.
Key point: with {oa-tbx}, migrated flows become open SQL, readable and auditable line by line, and executable on any environment.
{oa-tbx} benefits in detail → · Discover {oa-tbx} → · ETL migration to {oa-tbx} in detail →
