BigQuery

Category: Google

|

Runtime: 0.9+

|

Available: Free

Overview

BigQuery node by Kumologica integrates seamlessly with Google BigQuery.

Setup

Prerequisites

For BigQuery node to work properly, you need to have a Google cloud account (opens new window) and Service account credential file from Google cloud console. For more details refer service account keys (opens new window).

Installation

To install the node from designer:

1. Go to Add more nodes option on the palette section.
   
2. Check for the BigQuery node and click install.
   
3. Once installed, the designer is required to be restarted.
   

To install the node from CLI:

1. Go to your project workspace where you can see your package.json file.
   
2. Run the below give npm command.
   npm i @kumologica/kumologica-contrib-bigquery

Technical Details

Properties

• ProjectId (Required)
  ProjectId of your project in Google cloud account.
• KeyFile (Required)
  Service Account credential file generated from Google cloud console. This can be JSON or a PEM file.
• Query (Optional)
  Query for fetching the data from BigQuery.
• DataSet (Optional)
  Dataset associated with the table against which the query to be executed.
• Table (Optional)
  Table against which the query needs to execute.
  

Note
The query statement(currently BigQuery only supports SELECT statement) must be specified in the node property. 512 mb is the minimum memory requirement for your AWS lambda deployment when using BigQuery node.

Supported Operations

• Query
  Query operation helps in querying the data in BigQuery.
• Insert
  Insert operation helps in inserting the data into BigQuery.

Throws

BigQuery node throws the following exception.

• BigQueryOnQueryError
• BigQueryOnInsertError
• BigQueryInvalidMessageError

Returns

Payload from this node will be overwritten by Insert operation. The payload can be accessed in the subsequent node using msg.payload expression. For Query operation the payload will not be overwritten.