BigQuery

Connect Google BigQuery to Memex for data warehouse analytics.

What is BigQuery?

BigQuery is Google Cloud's enterprise data warehouse service. It handles petabyte-scale analytics workloads without requiring infrastructure management, using a serverless architecture that scales automatically. When you connect BigQuery to Memex, you can query massive datasets, create visualizations, and generate insights—all through natural conversation.

What You'll Need

To connect BigQuery to Memex, you'll need a Google Cloud service account with the appropriate permissions:

Credential	Description
Service Account JSON	A JSON key file containing your service account credentials

Setting Up BigQuery Access

Step 1: Create or Select a Google Cloud Project

1. Go to the Google Cloud Console

2. Create a new project or select an existing one with BigQuery data

Step 2: Enable the BigQuery API

Before creating a service account, ensure the BigQuery API is enabled:

1. In the Google Cloud Console, go to APIs & Services → Library

2. Search for "BigQuery API"

3. Click on it and press Enable (if not already enabled)

Most projects have BigQuery API enabled by default, but it's worth checking if you encounter "API not enabled" errors.

Step 3: Create a Service Account

1. In the Google Cloud Console, go to IAM & Admin → Service Accounts

2. Click Create Service Account

3. Enter a Name (e.g., "Memex BigQuery Access")

4. Enter a Description (optional)

5. Click Create and Continue

Button grayed out? If the "Create Service Account" button is grayed out, you may not have sufficient permissions in this Google Cloud project. Contact your project administrator to request the "Service Account Admin" role.

Step 4: Grant Required Permissions

Add the following roles to your service account:

Role	Purpose
BigQuery Data Viewer	Read access to query tables and view data
BigQuery Job User	Permission to run queries

1. Click Select a role

2. Search for and select BigQuery Data Viewer

3. Click Add another role

4. Search for and select BigQuery Job User

5. Click Continue, then Done

Principle of Least Privilege

These two roles provide read-only access to BigQuery. Only add additional roles if you need Memex to create or modify tables.

Step 5: Create and Download the JSON Key

1. Click on the service account you just created

2. Go to the Keys tab

3. Click Add Key → Create new key

4. Select JSON format

5. Click Create

The JSON key file will download automatically. Keep this file secure.

Keep your key file secure! This file provides access to your BigQuery data. Don't share it or commit it to version control.

Connecting to Memex

1. Open the Memex Hub from the sidebar

2. Click on the Connectors tab

3. Click on the BigQuery card

4. Enter a Connection Name (e.g., "Analytics Warehouse")

5. Open your downloaded JSON key file in a text editor

6. Copy the entire contents and paste into the Service Account JSON field

7. Click Add Connection

Your BigQuery data warehouse is now connected! Memex can access it across all your conversations.

Verifying Your Connection

After connecting, try asking Memex:

What datasets do I have in BigQuery?

If Memex returns your dataset names, the connection is working correctly.

Using Your Connection

Once connected, you can ask Memex to work with your BigQuery data:

What datasets are available in my BigQuery project?

Query my BigQuery data warehouse for the top 10 customers by revenue

Using my BigQuery connection, show me daily active users for the past 90 days

Using my BigQuery data, write a query to find users who haven't logged in for 30 days

Security Considerations

Service Account Security

• Once connected, you can delete the JSON file from your downloads—Memex stores the credentials securely

• Never commit service account keys to version control

• Use separate service accounts for different applications

• Regularly audit which service accounts have access to your data

Limiting Access

You can restrict what the service account can access:

• Grant roles at the dataset level instead of project level for finer control

• Use BigQuery Data Viewer for read-only access

• Avoid granting BigQuery Admin unless absolutely necessary

Granting Access at Dataset Level

For more granular control, you can grant access to specific datasets instead of the entire project:

1. Go to BigQuery in the Google Cloud Console

2. In the Explorer panel, click on the dataset name

3. Click Sharing → Permissions

4. Click Add Principal

5. Enter the service account email

6. Select BigQuery Data Viewer role

7. Click Save

This limits Memex to only the datasets you explicitly share.

Key Rotation

Google recommends rotating service account keys periodically:

1. Create a new key in the Google Cloud Console

2. Update your Memex connection with the new key

3. Delete the old key from Google Cloud

If you suspect your service account key has been compromised, delete it immediately in the Google Cloud Console and create a new one.

Troubleshooting

You can ask the Memex agent to help troubleshoot, but here are a few common problems:

"Permission denied" Error

• Verify the service account has both BigQuery Data Viewer and BigQuery Job User roles

• Check that roles are granted at the correct level (project or dataset)

• Ensure you're querying datasets the service account can access

"Invalid JSON" Error

• Make sure you copied the entire contents of the JSON file

• Check that you didn't accidentally include extra characters

• Verify the JSON is properly formatted (no missing braces or commas)

"Project not found" Error

• Confirm the project ID in your JSON matches an existing project

• Verify the BigQuery API is enabled for the project

• Check that the service account belongs to the correct project

Can't See Certain Datasets

• The service account only sees datasets it has permission to access

• Grant BigQuery Data Viewer on specific datasets if needed

• Ask your GCP administrator to verify permissions

Learn More

• BigQuery Documentation

• BigQuery IAM Roles

• Creating Service Accounts