It's definitely better than raw 500+ lines of SQL composed with 300+ lines of Python for some bespoke business transformation without clean versioning, which is what SQL transformations tend to converge towards without something like dbt.
Yeah that's fair. In practice, the projects I've worked on have always seemed to end up with a mix of all of dbt, SQL strings in python, and dataframe manipulation. I've always felt like we should be able to minimize one of these in favor of the others, but in practice it seems like people find each of these things useful for different things.
It's definitely better than raw 500+ lines of SQL composed with 300+ lines of Python for some bespoke business transformation without clean versioning, which is what SQL transformations tend to converge towards without something like dbt.
Yeah that's fair. In practice, the projects I've worked on have always seemed to end up with a mix of all of dbt, SQL strings in python, and dataframe manipulation. I've always felt like we should be able to minimize one of these in favor of the others, but in practice it seems like people find each of these things useful for different things.
Versioning seems like a separate problem, no? I mean, you can version control your python program generating SQL queries