This is a multi-part article series where I dive into the ideal stack you’d use for a data engineering pipeline given constraints around what software providers to use. I aim to provide some indications of cost, ease of use, and functional limitations / cool features.
Original article: Azure
This article focus: Databricks
The Databricks edition
It is really, really worthwhile looking into a platform like Databricks, and that’s because you can do pretty much anything in it. Personally, although I work on Orchestra which you wouldn’t need if you use an “all in one” platform, I think it makes complete sense using an “all in one” platform. Nothing even comes close to Databricks apart from possibly Microsoft Fabric. So let’s dive in and see why it’s so sick.