How to backfill data in BigQuery (marketplace)

Once you’ve set up a data transfer to BigQuery, use a data backfill to add historical data to your dataset.

This guide applies only to BigQuery transfers. See our separate guide for backfilling data to other destinations.

Before you begin

Before you can backfill data, a data transfer to BigQuery has to be completed successfully.

Instructions

1. Log in to the Google Cloud Platform.
2. Navigate to BigQuery → Data transfers.
3. Click on the transfer you want to use for the data backfill.
4. On the transfer details page, click Schedule backfill in the top right corner.
5. Set the start and end dates for the historical data you want to backfill and click Ok.

Once your backfill has been saved, you’ll be taken to the Run history of the transfer. Depending on the amount of data in your backfill, completing it may take some time.

Limitations

• You can transfer up to 6 months’ worth of data with 1 backfill. If you need to backfill data from a longer time period, create separate backfills in 6-month batches.
• Different data sources have varying data retention policies, so you may only be able to fetch data from their maximum time range of historical data.
• Trial licenses can only backfill up to 14 days.

More resources

• How to monitor your data transfers
• How to visualize BigQuery data in Looker Studio (formerly Data Studio)