700개 이상의 실습 및 과정 이용하기

Ingesting New Datasets into BigQuery v1.5

실습 1시간 30분 universal_currency_alt 크레딧 5개 show_chart 입문
info 이 실습에는 학습을 지원하는 AI 도구가 통합되어 있을 수 있습니다.
700개 이상의 실습 및 과정 이용하기

Overview

BigQuery is Google's fully managed, NoOps, low cost analytics database. With BigQuery you can query terabytes and terabytes of data without having any infrastructure to manage or needing a database administrator. BigQuery uses SQL and can take advantage of the pay-as-you-go model. BigQuery allows you to focus on analyzing data to find meaningful insights.

The dataset you'll use is an ecommerce dataset that has millions of Google Analytics records for the Google Merchandise Store loaded into BigQuery. You have a copy of that dataset for this lab and will explore the available fields and row for insights.

In this lab you will ingest several types of datasets into tables inside of BigQuery.

Setup and requirements

For each lab, you get a new Google Cloud project and set of resources for a fixed time at no cost.

  1. Sign in to Qwiklabs using an incognito window.

  2. Note the lab's access time (for example, 1:15:00), and make sure you can finish within that time.
    There is no pause feature. You can restart if needed, but you have to start at the beginning.

  3. When ready, click Start lab.

  4. Note your lab credentials (Username and Password). You will use them to sign in to the Google Cloud Console.

  5. Click Open Google Console.

  6. Click Use another account and copy/paste credentials for this lab into the prompts.
    If you use other credentials, you'll receive errors or incur charges.

  7. Accept the terms and skip the recovery resource page.

Open BigQuery Console

  1. In the Google Cloud Console, select Navigation menu > BigQuery.

The Welcome to BigQuery in the Cloud Console message box opens. This message box provides a link to the quickstart guide and lists UI updates.

  1. Click Done.

Task 1. Create a new dataset to store tables

  1. In the BigQuery console, click on the 3 vertical dots next to your project, then click Create Dataset.

The Create dataset option highlighted on the UI

  1. Set the Dataset ID to ecommerce. Leave the other fields at their default values.

  2. Click Create dataset.

You'll now see the ecommerce dataset under your project name.

Task 2. Ingest a new dataset from a CSV

Scenario: Your marketing team is looking to you to help guide them with what products should be up for promotions based on inventory stock levels. They have also asked how each product is trending in customer sentiment based on the product reviews.

Your existing ecommerce transactional dataset does not have inventory stock levels or product review data in it, but your operations and marketing teams have provided you with new datasets for you to analyze.

Here is how you get started:

  1. Download the product stock level dataset locally onto your computer.

  2. Select the ecommerce dataset; then click Create Table.

The highlighted Create Table button and ecommerce dataset

  1. Specify the below table options:

Source:

  • Create table from: Upload
  • Select file: select the file you downloaded locally earlier
  • File format: CSV

Destination:

  • Table name: products

  • Leave other settings at their default value.

Schema:

  • Check Auto Detect for Schema and input parameters

Tip: Not seeing the checkbox? Ensure the file format is CSV and not Avro.

  • Partition and Cluster settings: Leave at default settings

Advanced Options:

  • Leave at default settings
  1. Click Create Table.

You should now see the products table below the ecommerce dataset.

  1. Select the products table and view details:

The Details tab for the products table

  1. Select Preview and confirm all columns have been loaded (sampled below):

SKU

name

orderedQuantity

stockLevel

restockingLeadTime

GGOEGDHQ014899

20 oz Stainless Steel Insulated Tumbler

499

652

2

GGOEGOAB022499

Satin Black Ballpoint Pen

403

477

2

GGOEYHPB072210

Twill Cap

1429

1997

2

GGOEGEVB071799

Pocket Bluetooth Speaker

214

246

2

You have successfully loaded in a CSV file into a new BigQuery table.

Exploring newly loaded data with SQL

Next, practice with a basic query to gain insights from the new products table.

  • In the +SQL query, write a query to list the top 5 products with the highest stockLevel:
#standardSQL SELECT * FROM ecommerce.products ORDER BY stockLevel DESC LIMIT 5

Task 3. Ingest data from Google Cloud Storage

  1. Select the ecommerce dataset and click Create Table.

  2. Specify the below table options:

Source:

  • Create table from: Google Cloud Storage
  • Select file from GCS bucket: cloud-training/data-insights-course/exports/products.csv
  • File format: CSV

Destination:

  • Table name: products

  • Leave all other settings as default.

Schema:

  • Check Auto Detect for Schema and input parameters.

Advanced Options:

  • Leave at default settings
  1. Click Create Table.

Does it work? No

  1. Click Cancel to close the message, then click Yes,quit in the Create table dialog.

  2. Click on the upward arrow in the bottom right and click, Project history and select the error message.

The Project history tab reflecting the highlighted error message

  1. Click the Repeat load job button.

  2. In the Create table form, click on Advanced Options and in the Write Preference dropdown menu, select Overwrite table.

  3. Now click Create Table.

  4. Confirm the table was executed successfully.

Task 4. Ingest a new dataset from a Google Spreadsheet

  1. Select Compose New Query.

  2. Execute this next query to show which products are in the greatest restocking need based on inventory turnover and how quickly they can be resupplied:

#standardSQL SELECT *, SAFE_DIVIDE(orderedQuantity,stockLevel) AS ratio FROM ecommerce.products WHERE # include products that have been ordered and # are 80% through their inventory orderedQuantity > 0 AND SAFE_DIVIDE(orderedQuantity,stockLevel) >= .8 ORDER BY restockingLeadTime DESC Note: If you specify a relative project name path like ecommerce.products instead of project_id.ecommerce.products, BigQuery will assume the current project.

Task 5. Saving Data to Google Sheets

Scenario: You want to provide your supply chain management team with a way to notate whether or not they have contacted the supplier to reorder inventory, and to make any notes on the items. You decide on using a Google Spreadsheet for a quick survey.

Now you'll create it:

  1. In Query Results, select Save Results > Google Sheets.

The expanded Save Results dropdown menu

  1. A popup will appear with a link to Open the spreadsheet, select Open.

  2. In your spreadsheet, in column G add a new field titled comments and for the first product row type new shipment on the way.

Column G highlighted

  1. In Google Sheets, select Share and Get Shareable Link then copy the link.

  2. Return to your BigQuery tab.

  3. Click on the ecommerce dataset, then Create Table.

  4. Specify the these table options:

Source:

  • Create table from: Drive
  • Select Drive URI: put-your-spreadsheet-url-here
  • File format: Google Sheet

Destination:

  • Table type: Leave as default (External table)
  • Table name: products_comments

Schema:

  • Check Auto Detect for Schema and input parameters

The Create table dialog

Advanced options:

  • Set Header rows to skip: to 1.
  • Leave all options at their default.
  1. Click Create Table.
Note: There is no load job for creating an externally-linked table since the data is not ingested.

Query data from an external spreadsheet

  1. Click Compose New Query.

  2. Add the below query then Run:

#standardSQL SELECT * FROM ecommerce.products_comments WHERE comments IS NOT NULL

Wait for the query to execute. You will see that the new comments field is now returned.

SKU

name

orderedQuantity

stockLevel

restockingLeadTime

ratio

comments

GGOENEBB078899

Cam Indoor Security Camera - USA

2139

2615

42

0.8179732314

new shipment on the way

  1. Navigate back to your Google Spreadsheet tab.

  2. Type in more comments in the Comments field.

  3. Navigate back to BigQuery and execute the query again by clicking Run.

  4. Confirm the new data properly shows in the results.

You have successfully created an external table connection into BigQuery from Google Spreadsheets.

Task 6. External table performance and data quality considerations

Linking external tables to BigQuery (e.g. Google Spreadsheets or directly from Google Cloud Storage) has several limitations. Two of the most significant are:

  • Data consistency is not guaranteed if the data values in the source are changed while querying.
  • Data sources stored outside of BigQuery lose the performance benefits of having BigQuery manage your data storage (including but not limited to auto-optimization of your query execution path, certain wildcard functions are disabled, etc.).

Congratulations!

You've successfully created a new dataset and ingested new external data sources into BigQuery from CSV, Google Cloud Storage, and Google Drive.

End your lab

When you have completed your lab, click End Lab. Google Cloud Skills Boost removes the resources you’ve used and cleans the account for you.

You will be given an opportunity to rate the lab experience. Select the applicable number of stars, type a comment, and then click Submit.

The number of stars indicates the following:

  • 1 star = Very dissatisfied
  • 2 stars = Dissatisfied
  • 3 stars = Neutral
  • 4 stars = Satisfied
  • 5 stars = Very satisfied

You can close the dialog box if you don't want to provide feedback.

For feedback, suggestions, or corrections, please use the Support tab.

Copyright 2025 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.

시작하기 전에

  1. 실습에서는 정해진 기간 동안 Google Cloud 프로젝트와 리소스를 만듭니다.
  2. 실습에는 시간 제한이 있으며 일시중지 기능이 없습니다. 실습을 종료하면 처음부터 다시 시작해야 합니다.
  3. 화면 왼쪽 상단에서 실습 시작을 클릭하여 시작합니다.

시크릿 브라우징 사용

  1. 실습에 입력한 사용자 이름비밀번호를 복사합니다.
  2. 비공개 모드에서 콘솔 열기를 클릭합니다.

콘솔에 로그인

    실습 사용자 인증 정보를 사용하여
  1. 로그인합니다. 다른 사용자 인증 정보를 사용하면 오류가 발생하거나 요금이 부과될 수 있습니다.
  2. 약관에 동의하고 리소스 복구 페이지를 건너뜁니다.
  3. 실습을 완료했거나 다시 시작하려고 하는 경우가 아니면 실습 종료를 클릭하지 마세요. 이 버튼을 클릭하면 작업 내용이 지워지고 프로젝트가 삭제됩니다.

현재 이 콘텐츠를 이용할 수 없습니다

이용할 수 있게 되면 이메일로 알려드리겠습니다.

감사합니다

이용할 수 있게 되면 이메일로 알려드리겠습니다.

한 번에 실습 1개만 가능

모든 기존 실습을 종료하고 이 실습을 시작할지 확인하세요.

시크릿 브라우징을 사용하여 실습 실행하기

이 실습을 실행하려면 시크릿 모드 또는 시크릿 브라우저 창을 사용하세요. 개인 계정과 학생 계정 간의 충돌로 개인 계정에 추가 요금이 발생하는 일을 방지해 줍니다.