Inspecting BigQuery Public Datasets from Google App Engine

BigQuery hosts over 200 public datasets that you can use for free up to some pretty generous bandwidth limits. They are particularly useful for experimentation because you can query them without copying them into your GCP project.

The code below demonstrates how to access one of these datasets, and then gets metadata for the dataset's tables using INFORMATION_SCHEMA.

import json
from django.http import HttpResponse
from google.cloud import bigquery

def index(request):
    dataset = "bigquery-public-data.new_york_taxi_trips"
    query = f"SELECT * FROM {dataset}.INFORMATION_SCHEMA.TABLES"

    bq_client = bigquery.Client()
    query_job = bq_client.query(query)
    results = query_job.result()
    rows = []
    for row in results:
      for key, val in row.items():
        rows.append(f"{key}: {val}")
    return HttpResponse(json.dumps(rows))

Resources