Skip to content

InsightsData platforms

How to use BigQuery and Databricks together on one copy of your data

Use BigQuery for SQL and BI, add Databricks for Spark and ML, and share one copy of your data through Iceberg and catalog federation. Checked Nov 11, 2026.

Written by
CloudMagic engineering team
Published
Reading time
11 minutes

The short answer

Use BigQuery for SQL analytics, BI and governed reporting, and add Databricks on Google Cloud where your team needs Spark engineering, notebooks and machine learning on the same data, as many fraud, risk and large reporting workloads do. Since November 2, 2026, BigQuery can query Iceberg tables governed by Databricks Unity Catalog without copying them, and Databricks can read BigQuery and Lakehouse tables through its own federation, so one copy of the data in Cloud Storage can serve both engines. Databricks is bought through Google Cloud Marketplace and appears on your Google Cloud bill; confirm with your Google account team whether that spend counts toward your commit before you sign.

Checked November 11, 2026. Product names, prices and release status are as Google's and Databricks' own pages showed them that day.

When a team that reports in BigQuery asks for Databricks, the first worry is a second copy of the data. Since November 2, one copy can serve both engines.

Our financial services page describes the pattern we build: Databricks for fraud models, risk analytics and large reporting jobs, often paired with BigQuery. We have no benchmark of our own on this topic, so every number below is a published price, limit or date with its source.

When is BigQuery enough, and when do we need Databricks as well?

BigQuery is enough when the work is SQL: dashboards, scheduled reports, ad hoc analysis, and models an analyst can build with BigQuery ML. Add Databricks when your engineers write most pipelines and models in Spark and Python notebooks, or when part of the business already runs Databricks.

WorkloadStart withWhy
BI dashboards and governed reportingBigQuerySQL, with row-level security and masking outside Standard edition
Ad hoc SQL by analystsBigQueryPay per TiB scanned, or move to slots once usage is steady
Models an analyst can write in SQLBigQuery MLIncluded in Enterprise, Enterprise Plus and on-demand pricing
Fraud model training and feature engineeringDatabricksSpark and notebooks over the full history; scores published for BigQuery
Risk analytics and large batch reporting jobsDatabricksLong Spark jobs in Lakeflow Jobs, results written as Iceberg tables
Spark jobs for a team that does not use DatabricksManaged Service for Apache SparkGoogle's managed Spark (formerly Dataproc), which can also query Unity Catalog tables

BigQuery Standard edition has no BigQuery ML, no capacity commitments and no fine-grained security controls (column-level access control, row-level security and data masking), and it caps a reservation at 1,600 slots. A reporting workload that needs row-level security has ruled out Standard before Databricks comes up.

Can BigQuery and Databricks read the same tables without copying data?

Yes, in both directions, through two federation features. Google's side became generally available on November 2, 2026, after a preview from April 22.

Databricks writes Parquet files with Iceberg metadata to Cloud Storage, as Unity Catalog managed Iceberg tables or Delta tables with UniForm. BigQuery reaches them through Lakehouse remote catalog federation, which talks to Unity Catalog over the Apache Iceberg REST Catalog API. At query time Unity Catalog returns the current table state and vends short-lived credentials that give BigQuery temporary read access to the files. New data appears after the next metadata refresh, which Google says can run as often as every five minutes. It works with Unity Catalog on AWS, Azure or Google Cloud.

Going the other way, Databricks has two read paths, shown in the table.

DirectionFeatureAccessWatch for
BigQuery reads Unity Catalog tablesLakehouse remote catalog federation, GA November 2, 2026Read, with short-lived vended credentialsMetadata refresh lag; egress if the catalog is on AWS or Azure
Databricks reads native BigQuery tablesLakehouse Federation, Runtime 16.1 or laterRead through the Storage APIViews and external tables are always materialized, with BigQuery compute charges
Databricks reads Google Cloud Lakehouse Iceberg tablesGoogle Cloud Lakehouse federation, Runtime 18 LTS or laterRead-onlyThe docs give no release status, so check it before production use

Treat all three paths as reads. Give every shared table one engine that writes it, and let the other read it. When BigQuery produces a table Databricks needs, write it in BigQuery and let Databricks read it through federation.

Should we standardize on Iceberg or Delta if both engines use the data?

Make every Databricks table that BigQuery reads readable as Iceberg, and keep plain Delta for tables only Databricks uses. Unity Catalog managed Iceberg tables have been generally available since May 21, 2026, and Unity Catalog's Iceberg REST endpoint lets outside engines such as Apache Spark read and write them.

Delta tables with Iceberg reads (UniForm) suit pipelines that already write Delta. Outside engines can only read them through that endpoint, which is fine for tables BigQuery only reads.

Three details from the vendors' documentation shape the design:

  • Databricks managed Iceberg tables do not support partition transforms; Databricks recommends liquid clustering instead.
  • For BigQuery to prune partitions, compare partition columns with literal values in the query.
  • Google's walkthrough runs OPTIMIZE on Databricks to compact small files and keeps a seven-day delta.deletedFileRetentionDuration before VACUUM. That housekeeping belongs to the writing engine.

Tables BigQuery writes can stay native, since Databricks reads those through Lakehouse Federation.

Who governs what: Unity Catalog, Knowledge Catalog, or both?

Both, divided by which engine writes the table. Unity Catalog governs the tables Databricks writes and decides who gets credentials to their files. Knowledge Catalog, formerly Dataplex Universal Catalog, catalogs data on the Google side, and BigQuery enforces its own policies on every table it queries, federated ones included.

Google's guide says that once users are granted access to the Lakehouse table entity, BigQuery applies row-level security, column-level security and dynamic data masking to the external Iceberg tables. With user identity federation, the end user's identity travels with each query, so Unity Catalog can check and log the person behind it.

Plan for one gap. Databricks' documentation says tables with Unity Catalog row filters or column masks cannot be accessed through the Iceberg REST catalog, except for reads under what it calls cross-engine ABAC, and Google's guide describes the federation as using that API. Either publish a filtered or masked table for BigQuery to read, or set the policy in BigQuery on the federated table, then test from both engines as a user who should see less. Our post on keeping sensitive data out of an AI agent's reach uses the same idea with masked BigQuery views.

How is Databricks on Google Cloud billed, and does it count toward our Google Cloud commit?

Databricks on Google Cloud is bought through Google Cloud Marketplace and appears on your Google Cloud bill. Whether that spend counts toward your Google Cloud commit is a question for your Google account team, and you want the answer in writing before you sign.

Subscribing to the Databricks listing creates the Databricks account, tied to one Google billing account. A free trial becomes pay-as-you-go when it ends, and companies with a contract use a Marketplace private offer set up by their Databricks account team. Since April 14, 2023, compute usage has appeared in Marketplace under one SKU, "Databricks Consumption Units".

Restrict who can cancel. Databricks' docs say anyone in your organization with the required permission on the billing account can cancel the subscription, and cancelling deletes every workspace in the account, irreversibly.

Google's commit drawdown page says 100% of spend on Google Cloud Marketplace services, other than listed exclusions, may count toward minimum commitment obligations. The exclusions include services not running on Google Cloud and professional services, and Google tells buyers to ask the vendor which of its services run on Google Cloud. The page does not name Databricks, and we found no published statement on Databricks spend, so ask your Google and Databricks account teams to confirm it.

What will our workloads cost: bytes scanned, BigQuery slots or Databricks DBUs?

BigQuery charges $6.25 per TiB scanned on demand, or $0.04 to $0.10 per slot-hour by edition in us-central1, and Databricks charges in Databricks Units (DBUs) at rates its account team quotes. Price one real workload on each engine, because list prices alone do not tell you which costs less.

BigQuery pricing, checked November 11, 2026Price (USD)
On-demand queries$6.25 per TiB scanned; first 1 TiB each month free
Standard edition, pay as you go, us-central1$0.04 per slot-hour
Enterprise edition, pay as you go, us-central1$0.06 per slot-hour
Enterprise Plus edition, pay as you go, us-central1$0.10 per slot-hour

Capacity is billed per second with a one-minute minimum.

The DBUs a Databricks workload consumes depend on factors such as the compute type and the Google Cloud machine type. Databricks' Google Cloud pricing page, checked the same day, did not publish per-DBU rates; it pointed to a price calculator and to sales, so get your rates from your Databricks account team. Classic compute runs as Compute Engine VMs in your own project, so count those VMs too. Serverless compute runs in Databricks' account, in your workspace's region.

Crossing between engines adds two charges. Databricks reading a BigQuery view or external table makes BigQuery materialize it and bill the compute. Reading from a Unity Catalog on AWS or Azure still incurs remote-cloud egress charges, which Google's Lakehouse caching, in preview, can reduce.

To measure, tag the workload's jobs on both sides. In BigQuery, INFORMATION_SCHEMA.JOBS records total_bytes_billed and total_slot_ms for each job, with 180 days of history. In Databricks, system.billing.usage records DBUs against each job or SQL warehouse, with your custom tags. A few weeks of both gives you cost per run for the same output.

What changed in 2025 and 2026 that older comparisons still get wrong?

Databricks classic compute on Google Cloud now runs on Compute Engine VMs, several products have new names, and the two catalogs now federate. Many of the pages that rank for this question were written before these changes.

Older name or claimAs of November 11, 2026When
A GKE deployment for DatabricksClassic compute on Compute Engine VMs; Databricks' docs call GKE the "legacy GKE deployment model"Page updated September 11, 2026
BigLakeLakehouse; APIs, client libraries, CLI and IAM names still say BigLakeApril 20, 2026
Dataplex Universal CatalogKnowledge Catalog; API and IAM names unchangedApril 10, 2026
Data CatalogPhased shutdownFrom June 1, 2026
Iceberg tables in BigQueryApache Iceberg managed tablesCurrent BigQuery docs
DataprocManaged Service for Apache SparkCurrent product page
DLT (Delta Live Tables)Lakeflow Spark Declarative PipelinesJune 11 and November 12, 2025
Databricks Jobs (Workflows in the old menu)Lakeflow JobsJune 11, 2025
Unity Catalog Iceberg tables in previewManaged and foreign Iceberg tables GAMay 21, 2026
BigQuery cannot query Unity CatalogRemote catalog federation GANovember 2, 2026
Genie spacesGenie AgentsJuly 2026

What do the first 60 days of running both look like?

Move one workload end to end with one writer per table, and measure its cost and check its results before a second workload follows. This is our plan for a team adding Databricks next to BigQuery:

WeeksWorkDone when
1 and 2Pick one workload, such as a monthly risk report. Name each table's writing engine, check regions, settle the subscription, restrict cancellation and get the commit answer.Every table has one writer and one owning team
3 and 4Create the shared tables as Unity Catalog managed Iceberg tables, connect BigQuery with user identity federation, and set policies on both sides.A restricted test user sees the same rows from both engines
5 and 6Run Spark on Databricks and reporting in BigQuery, with every job tagged.Totals match the current report, signed off by its owners
7 and 8Compare cost per run, write the runbook and choose the next workload.Your team, or ours, runs it from the runbook

Weeks 5 and 6 decide whether people trust the new path, so reconcile the report line by line before anyone retires the old job. For the rest of what we build, see our Databricks and Google Cloud pages and our data and analytics work, or talk to us.

Frequently asked questions

Can we use Gemini models inside Databricks?

Yes. Databricks announced on June 12, 2025 that Gemini models are available in Databricks through SQL queries and model endpoints, and that customers can pay for Gemini usage through their Databricks contract.

Is Databricks serverless available in our region?

Databricks lists serverless notebooks, jobs, pipelines and SQL warehouses in 15 Google Cloud regions, including us-central1, us-east1, us-east4, us-west1 and us-west4.

Does BigQuery Standard edition include BigQuery ML?

No. BigQuery ML is included in Enterprise, Enterprise Plus and on-demand pricing. Standard edition also carries a 99.9% monthly service level objective and does not support customer-managed encryption keys.

Can BigQuery write to Unity Catalog tables?

Plan on reads. Google's October 30 guide speaks of read and write interoperability, but the steps it spells out give BigQuery temporary read access. If a Google-side job must write, test a Spark write through Unity Catalog's Iceberg REST endpoint first, or have BigQuery write its own table.

What happens to row-level security across both?

Each engine enforces its own rules: BigQuery's row-level security and masking on federated tables (not in Standard edition), and Unity Catalog's row filters and column masks on Databricks. Test every rule from both sides.

Sources

Bring us the question you are working on.

A senior engineer reads every message and replies within one business day.