Google launched Materialized Views for BigQuery & BigLake
Amazon calls it the War on ETL but also other big players on the market like Microsoft and Google want to ease data integration. The latter has therefore launched BigLake which allows cloud independent data analysis via SQL. More connectors and platforms which ease data source integration are only the first step of the No-ETL approach which also wants to ease the data transformation and cleaning process. Here, no additional ETL-tools are needed to use build-in connectors for transferring data or even like Google BigLake allowing to query the data directly on the source system.
A problem on this approach is the fact that querying the data directly might be suitable for PoC and Data Science tasks because for things like dashboards, reports & Co. you often need transformed and cleaned data. For these use cases, Google has now released materialized views over BigLake metadata cache-enabled tables that can reference structured data stored in Cloud Storage. Google states that these materialized views function like materialized views over BigQuery-managed storage tables, including the benefits of automatic refresh and smart tuning. Another cool fact about is that this feature is already generally available[1].
0 Comments