{"id":8714,"date":"2026-09-17T08:01:28","date_gmt":"2026-09-17T08:01:28","guid":{"rendered":"https:\/\/prwatech.in\/blog\/?p=8714"},"modified":"2026-09-18T16:41:42","modified_gmt":"2026-09-18T16:41:42","slug":"adding-local-file-into-big-query","status":"publish","type":"post","link":"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/adding-local-file-into-big-query\/","title":{"rendered":"Adding Local File into Big Query"},"content":{"rendered":"\r\n

Uploading a Local File Into BigQuery<\/h1>\r\n

BigQuery lets you upload a file straight from your computer and turn it into a table in one step, without touching Cloud Storage first, as long as the file is small enough. This walks through uploading a sample CSV this way and querying the result.<\/p>\r\n

Integrating local files into BigQuery represents a vital aspect of expanding data accessibility and leveraging the platform’s analytical capabilities. This introduction will elucidate the process of seamlessly incorporating local files into BigQuery, elucidating the methods, considerations, and advantages inherent in this operation.<\/p>\r\n

While BigQuery excels in processing vast datasets stored in Google Cloud Storage or other cloud-based repositories, there are scenarios where local files, residing within an organization’s infrastructure or on individual workstations, contain valuable data that warrants inclusion in analytical workflows.<\/p>\r\n

Step by Step Process of Adding Local Files into BigQuery<\/h2>\r\n

The process of adding local files into BigQuery involves a few essential steps, such as preparing the data in a compatible format, uploading it to Google Cloud Storage, and then utilizing BigQuery’s native capabilities to ingest the data into its tables. Supported file formats include CSV, JSON, Avro, Parquet, and more, ensuring flexibility and compatibility with various data sources.<\/p>\r\n

Step One: Open BigQuery<\/h3>\r\n

Open the console, then go to BigQuery, then SQL Workspace.<\/p>\r\n\r\n\r\n\r\n

Open Menu > Bigquery<\/a> > SQL Workspace<\/p>\r\n\r\n\r\n\r\n

\"\"<\/figure>\r\n\r\n\r\n\r\n

Step Two: Select Your Project<\/h3>\r\n

In Resources, select your project.<\/p>\r\n\r\n\r\n\r\n

\"\"<\/figure>\r\n\r\n\r\n\r\n

Step Three: Create a Dataset<\/h3>\r\n

Click Create Dataset.<\/p>\r\n\r\n\r\n\r\n

\"\"<\/figure>\r\n\r\n\r\n\r\n

Give it a name. The default settings are fine for this. Click Create Dataset.<\/p>\r\n\r\n\r\n\r\n

\"\"<\/figure>\r\n\r\n\r\n\r\n

Step Four: Download a Sample File<\/h3>\r\n

Download this sample CSV, a small subset of New York taxi trip data:<\/p>\r\n\r\n\r\n\r\n

https:\/\/storage.googleapis.com\/cloud-training\/OCBL013\/nyc_tlc_yellow_trips_2018_subset_1.csv<\/a><\/p>\r\n\r\n\r\n\r\n

In the created dataset, click create Table.<\/p>\r\n\r\n\r\n\r\n

\"\"<\/figure>\r\n\r\n\r\n\r\n

Step Five: Upload the File as a New Table<\/h3>\r\n

In your new dataset, click Create Table.<\/p>\r\n\r\n\r\n\r\n

\"\"<\/figure>\r\n\r\n\r\n\r\n

Choose Upload as the source.<\/p>\r\n\r\n\r\n\r\n

\"\"<\/figure>\r\n\r\n\r\n\r\n

Click Browse and select the downloaded file. BigQuery will usually detect the file format automatically. If it does not, set it to CSV manually.<\/p>\r\n\r\n\r\n\r\n

\"\"<\/figure>\r\n\r\n\r\n\r\n

The project and dataset fields are normally filled in for you already. Give the table a name, tick Auto Detect Schema, and click Create Table.<\/p>\r\n\r\n\r\n\r\n

\"\"<\/figure>\r\n\r\n\r\n\r\n

The table is created with the uploaded file’s contents.<\/p>\r\n

Step Six: Query the Table<\/h3>\r\n

In the query editor, replace the dataset and table names below with whatever you actually named yours, then run it:<\/p>\r\n\r\n\r\n\r\n

SELECT
\u00a0\u00a0*
FROM
\u00a0\u00a0your_dataset.your_table_name
ORDER BY
\u00a0\u00a0fare_amount DESC
LIMIT
\u00a0\u00a010<\/p>\r\n

This returns the ten highest fares in the sample, most to least expensive.<\/p>\r\n\r\n\r\n\r\n

\"\"<\/figure>\r\n\r\n\r\n\r\n

The 10 MB Limit on Direct Upload<\/h2>\r\n

Uploading a file straight from your computer through the console only works up to 10 MB for an uncompressed file. This is a hard limit on the direct upload path specifically, separate from the much larger limits that apply when loading data through Cloud Storage or the bq command line tool, which can handle files up into the terabytes.<\/p>\r\n

Once a file is too large for direct upload, the actual fix is exactly what the introduction of the live page was describing, just as a specific threshold rather than a rule that applies to every local file: upload the file to a Cloud Storage bucket first, then load it into BigQuery from there.<\/p>\r\n

Common Mistakes to Avoid<\/h2>\r\n