Reverse ETL integration with Google BigQuery

Today, we introduce our latest integration - a reverse integration with Google BigQuery, a cloud-based data warehousing solution. Unlike our previous integration that allowed sending data from Synerise to BigQuery, this feature allows you to effortlessly retrieve data from BigQuery directly into Synerise.

 

What is Google BigQuery?

Google BigQuery is a cloud-based data warehouse solution that enables users to store, manage and query data in a highly scalable and cost-effective way. It provides a fully managed, serverless infrastructure that eliminates the need for users to manage their hardware or software.

 

What advantages does this integration create?

  • Dynamic Querying: You define the data to be retrieved yourself in the integration through an SQL query. At Synerise, we've taken it a step further and enable the construction of dynamic queries using non-profile Jinjava tags, such as metrics or references to data from the catalogs
  • Incremental Data Retrieval: In the process of exchanging information, it is crucial to precisely specify the scope and quantity of the retrieved data. Our integration also allows for incremental data retrieval. This means that during synchronization, you only fetch new records or those that have been modified
  • Data Transformation Opportunities: You don't have to worry about the column names of the data you want to retrieve. Thanks to the automation capabilities in the data transformation area, you can tailor them to the import requirements

 

How to perform this import?

We have created a dedicated “Get Data - Reverse ETL” node, which enables pulling data from BigQuery to Synerise.

The process of creating the export is simple and consists of just a few steps.

1.  Build a workflow: To build a workflow, you need to select a trigger and establish a connection between Synerise and Google BigQuery. Use “Get Data - Reverse ETL” node.

‍

‍

2. Select Connection: To authenticate yourself in Google BigQuery, create a new connection using the extracted private key from the BigQuery tool.

‍

‍

3. Set up Dataset name and Query: Specify the data set name from BigQuery. Define the data you wish to extract.

During the configuration of the node, you can use Jinjava inserts which let you dynamically refer to non-profile tags like e.g. metrics (meaning you cannot make references to aggregates, attributes, or expressions).

Use the preview option to check the first 10 records and verify the query's accuracy.

‍

4. Choose the type of imported data: In the last step, choose the type of imported data - depending on whether they are profiles, events, etc. by selecting the appropriate node from the Synerise collection. Add the End node and run the workflow.

For more detailed information about prerequisites and the complete configuration process of the pulling data from Google BigQuery to Synerise in any form, please refer to our User Guide.  

Read our documentation to learn more!

We hope you enjoy our inspiration of the week!