More than a year ago, we decided to leverage Databricks and the Delta Lake as the core for our data platform. Fast-forward to Microsoft Build 2023, when Microsoft announces their new data platform Microsoft Fabric. We were positively surprised by Fabric’s highlighted key points and design choices, many of which overlap with the reasons why we decided to use Databricks in the first place, such as:
- Separation of compute and storage, underpinned by a Data Lake
- Data Lakehouse supported by a centralized unified catalog, governance and security
- Multiple ways of building and consuming data products: Spark, SQL, ML, PowerBI, …