Google BigQuery

The BigQuery extractor loads data from BigQuery and brings it into Keboola Connection (KBC).

Create New Configuration

Find Google Analytics in the list of extractors. Create a new configuration and name it.

Screenshot - Big Query Authorization

Initial Setup

Service Account

To access and extract data from your BigQuery dataset, you need to set up a Google Service Account.

Screenshot - Create service account

  • Then add the BigQuery Data Editor, BigQuery Job User and Storage Object Admin roles.

Screenshot - Create service account

  • Finally, create a new JSON key (click + Create key) and download it to your computer (click Create).

Screenshot - Create service account

  • Go back to your BigQuery extractor configuration.
  • In the Google Service Account Key section, click Set Service Account Key.

Screenshot - Set Service Account Key

  • Open the downloaded key in a text editor, copy & paste it in the input field, click Submit and then Save.

Screenshot - Copy & Paste Service Account Key

Important: The private key is stored in an encrypted form and only the non-sensitive parts are visible in the UI for your verification. The key can be deleted or replaced by a new one at any time.

Screenshot - Copy & Paste Service Account Key

Bucket

The extractor uses Google Storage Bucket as a temporary storage for off-loading the data from BigQuery.

Screenshot - Google Cloud Platform

  • Go back to your BigQuery extractor configuration.
  • In the Unload Configuration section, fill Cloud Storage Bucket Name you have created, and select the correct Dataset Location.
  • Click Save.

Screenshot - Google Cloud Platform

Configure Extraction

Start by clicking the green + Add Query button.

Screenshot - Big Query Bucket Configured

Name the query and click Create.

Screenshot - Big Query Bucket Configured

Specify your requirements in the SQL Query field and Save the query configuration.

Screenshot - Finished Configuration

Now run the configuration to bring the data to KBC.

Screenshot - Finished Configuration

Running the extractor creates a background job that

  • executes the queries in Google BigQuery.
  • saves the results to Google Cloud Storage.
  • exports the results from Google Cloud Storage and stores them in specified tables in Keboola Connection Storage.
  • removes the results from Google Cloud Storage.

Note: Using the Google BigQuery extractor is also described in our Getting Started Tutorial.