ETL Migration - from legacy ETL to dbt
Migrating to dbt means transforming proprietary ETL processes into SQL models run directly in the target data engine.
Before any conversion, {openAudit} analyzes flows, dependencies, and usage to identify the scope that is actually worth migrating.
Key point: the migration starts with a rationalization phase of the existing estate.
Flow analysis
ETL processes are analyzed to reconstruct dependencies, transformations, and usage.
Data lineage is used to identify active flows, dead branches, and unconsumed components.
This phase reduces the volume of processes to be migrated.
Result: a migration scope refocused on the processes that are actually used.
Conversion to SQL
ETL jobs are broken down into elementary operations: filters, joins, aggregations, lookups, business calculations, and transformations.
These components are converted into documented SQL, independent of the source technology.
SQL becomes the pivot representation of the processes.
Key point: business logic is rebuilt in SQL before the dbt project is generated.
Generating dbt models
The SQL processes are organized into dbt models.
Dependencies between models are automatically generated from the flows identified during the analysis.
The project structure, references between models, and documentation can be produced automatically.
Progressive validation
Migration is carried out in batches.
The generated models are validated and then progressively integrated into the target environment.
Discrepancies can be checked process by process during the transition phase.
Typical use case: progressive migration of several hundred ETL jobs to dbt.
Target architecture
The processes become version-controllable, maintainable SQL models.
Dependencies are explicit, transformations are visible, and processes can be run directly by Snowflake, BigQuery, Databricks SQL, Redshift, PostgreSQL, or any other compatible engine.
Key point: the processes no longer depend on the legacy ETL platform but on a dbt project built on open SQL.
