Bruin - GSheet to BigQuery Template
This pipeline is a simple example of a Bruin pipeline that copies data from GSheet to BigQuery. It demonstrates how to use the bruin
CLI to build and run a pipeline.
The pipeline includes two sample assets already:
gsheet_raw.customers
: A simple ingestr asset that copies a table from GSheet to BigQuery
Setup
Example Sheet: https://docs.google.com/spreadsheets/d/1p40qR9t6DM5a1IskTkqEX9eZYZmBeILzUX_AdMkg__A/edit?usp=sharing
The pipeline already includes an empty .bruin.yml
file, fill it with your connections and environments. You can read more about connections here.
Here's a sample .bruin.yml
file:
yaml
default_environment: default
environments:
default:
connections:
google_cloud_platform:
- name: "gcp-default"
service_account_file: "<Path to your Google Cloud service account JSON file>"
project_id: "bruin-common-health-check"
google_sheets:
- name: "gsheet-default"
credentials_path: "<Path to your Google Sheets credentials JSON file>"
Running the pipeline
bruin CLI can run the whole pipeline or any task with the downstreams:
shell
❯ bruin run ./templates/gsheet-bigquery/ (bruin)
Analyzed the pipeline 'bruin-init' with 1 assets.
Pipeline: bruin-init (.)
No issues found
✓ Successfully validated 1 assets across 1 pipeline, all good.
Starting the pipeline execution...
Executed 1 tasks in 9.656s
You can also run a single task:
shell
❯ bruin run ./templates/gsheet-bigquery/ (bruin)
Analyzed the pipeline 'bruin-init' with 1 assets.
Pipeline: bruin-init (.)
No issues found
✓ Successfully validated 1 assets across 1 pipeline, all good.
Starting the pipeline execution...
Executed 1 tasks in 9.656s
You can optionally pass a --downstream
flag to run the task with all of its downstreams.
That's it, good luck!