In this second installment of the Dataflow course series, we are going to be diving deeper on developing pipelines using the Beam SDK. We start with a review of Apache Beam concepts. Next, we discuss processing streaming data using windows, watermarks and triggers. We then cover options for sources and sinks in your pipelines, schemas to express your structured data, and how to do stateful transformations using State and Timer APIs. We move onto reviewing best practices that help maximize your pipeline performance. Towards the end of the course, we introduce SQL and Dataframes to represent your business logic in Beam and how to iteratively develop pipelines using Beam notebooks.
各種規模的機構都開始運用雲端的強大功能和靈活性,徹底改變營運方式。不過,有效管理及擴充雲端資源可能是相當複雜的工作。本課程將探討在雲端中的現代營運、可靠性和韌性的基礎概念,以及 Google Cloud 如何協助您達成這些目標。本課程是 Cloud Digital Leader 學習路徑的一部分,旨在促進個人職能發展,並打造企業的未來。
許多傳統企業使用舊版系統和應用程式,無法滿足現今客戶的期望。企業主管經常需要在維護老舊的 IT 系統,以及投資新產品/服務之間做出選擇。本課程將探討這些挑戰,並提供解決方案,說明如何運用雲端技術克服難題。 本課程是 Cloud Digital Leader 學習路徑的一部分,旨在促進個人職能發展,並打造企業的未來。
雲端技術是強大的資產,與資料搭配後,將成為創新和提升客戶體驗的催化劑。「運用 Google Cloud 探索資料轉換功能」課程將探討機構如何運用雲端,讓資料更易於存取、做為行動依據和更有價值。 這門課程是 Cloud Digital Leader 學習路徑的一部分,旨在協助個人在職務上成長,並打造企業的未來。
數位轉型是現代組織必經的關鍵旅程,而建立穩固的雲端運算基礎,是跨出有意義創新的第一步。本課程「透過 Google Cloud 進行數位轉型」,將會介紹核心技術和策略框架,協助組織革新營運方式,並探討基本的雲端概念、全球網路基礎架構和共同責任模式,幫助主管在雲端之路上自信前行。本課程是 Cloud Digital Leader 學習路徑的一部分,旨在促進個人職能發展,並打造企業的未來。
This course explores what ML is and what problems it can solve. The course also discusses best practices for implementing machine learning. You’re introduced to Vertex AI, a unified platform to quickly build, train, and deploy AutoML machine learning models. The course discusses the five phases of converting a candidate use case to be driven by machine learning, and why it’s important to not skip them. The course ends with recognizing the biases that ML can amplify and how to recognize them.
完成「建立及管理 Bigtable 執行個體」技能徽章入門課程,證明您具備下列技能:建立執行個體、設計結構定義、 查詢資料,以及在 Bigtable 執行管理工作,包括監控效能、設定自動調度節點資源和複製作業。
完成「建立及管理 Cloud Spanner 執行個體」技能徽章入門課程,即可證明自己具備下列技能: 建立 Cloud Spanner 執行個體和資料庫,並與其互動; 使用各種技術載入 Cloud Spanner 資料庫; 備份 Cloud Spanner 資料庫;定義結構定義及瞭解查詢計畫;以及 部署連線至 Cloud Spanner 執行個體的現代化網頁應用程式。
This course is intended to give architects, engineers, and developers the skills required to help enterprise customers architect, plan, execute, and test database migration projects. Through a combination of presentations, demos, and hands-on labs participants move databases to Google Cloud while taking advantage of various services. This course covers how to move on-premises, enterprise databases like SQL Server to Google Cloud (Compute Engine and Cloud SQL) and Oracle to Google Cloud bare metal.
「Google Cloud 基礎知識:核心基礎架構」介紹了在使用 Google Cloud 時會遇到的重要概念和術語。本課程會透過影片和實作實驗室,介紹並比較 Google Cloud 的多種運算和儲存服務,同時提供重要的資源和政策管理工具。
This course is part 1 of a 3-course series on Serverless Data Processing with Dataflow. In this first course, we start with a refresher of what Apache Beam is and its relationship with Dataflow. Next, we talk about the Apache Beam vision and the benefits of the Beam Portability framework. The Beam Portability framework achieves the vision that a developer can use their favorite programming language with their preferred execution backend. We then show you how Dataflow allows you to separate compute and storage while saving money, and how identity, access, and management tools interact with your Dataflow pipelines. Lastly, we look at how to implement the right security model for your use case on Dataflow.
Incorporating machine learning into data pipelines increases the ability to extract insights from data. This course covers ways machine learning can be included in data pipelines on Google Cloud. For little to no customization, this course covers AutoML. For more tailored machine learning capabilities, this course introduces Notebooks and BigQuery machine learning (BigQuery ML). Also, this course covers how to productionalize machine learning solutions by using Vertex AI.
In this course you will get hands-on in order to work through real-world challenges faced when building streaming data pipelines. The primary focus is on managing continuous, unbounded data with Google Cloud products.
In this intermediate course, you will learn to design, build, and optimize robust batch data pipelines on Google Cloud. Moving beyond fundamental data handling, you will explore large-scale data transformations and efficient workflow orchestration, essential for timely business intelligence and critical reporting. Get hands-on practice using Dataflow for Apache Beam and Serverless for Apache Spark (Dataproc Serverless) for implementation, and tackle crucial considerations for data quality, monitoring, and alerting to ensure pipeline reliability and operational excellence. A basic knowledge of data warehousing, ETL/ELT, SQL, Python, and Google Cloud concepts is recommended.
只要修完「在 Google Cloud 設定應用程式開發環境」課程,就能獲得技能徽章。 在本課程中,您將學會如何使用以下技術的基本功能,建構和連結以儲存空間為中心的雲端基礎架構:Cloud Storage、Identity and Access Management、Cloud Functions 和 Pub/Sub。
Big data, machine learning, and scientific data? It sounds like the perfect match. In this course, you will get hands-on practice with GCP services like BigQuery, Managed Service for Apache Spark, and Tensorflow by applying them to use cases that employ real-life, scientific data sets. By getting experience with tasks like earthquake data analysis and satellite image aggregation, Scientific Data Processing will expand your skill set in big data and machine learning so you can start tackling your own problems across a spectrum of scientific disciplines.
In this course you will learn how to use several BigQuery ML features to improve retail use cases. Predict the demand for bike rentals in NYC with demand forecasting, and see how to use BigQuery ML for a classification task that predicts the likelihood of a website visitor making a purchase.
While the traditional approaches of using data lakes and data warehouses can be effective, they have shortcomings, particularly in large enterprise environments. This course introduces the concept of a data lakehouse and the Google Cloud products used to create one. A lakehouse architecture uses open-standard data sources and combines the best features of data lakes and data warehouses, which addresses many of their shortcomings.
Learn how to use Gemini Notebook to create a personalized study guide for the Professional Data Engineer certification exam. You'll review Gemini Notebook features, create a notebook in Gemini Notebook, and learn how to use a study guide to practice for a certification exam.
This course introduces the Google Cloud big data and machine learning products and services that support the data-to-AI lifecycle. It explores the processes, challenges, and benefits of building a big data pipeline and machine learning models with Vertex AI on Google Cloud.
完成 在 Google Cloud 為機器學習 API 準備資料 技能徽章入門課程,即可證明您具備下列技能: 使用 Dataprep by Trifacta 清理資料、在 Dataflow 執行資料管道、在 Managed Service for Apache Spark 建立叢集和執行 Apache Spark 工作,以及呼叫機器學習 API,包含 Cloud Natural Language API、Google Cloud Speech-to-Text API 和 Video Intelligence API。
大數據、機器學習和人工智慧 (AI) 是時下熱門的 電腦相關話題,但這些領域相當專業,就算想要入門 也難以取得教材或資料。幸好,Google Cloud 提供了此領域的多種服務,而且容易使用。 參加這堂入門課程,您就能踏出第一步, 開始學習運用 BigQuery、Cloud Speech API 以及 Video Intelligence 等工具。
完成建立及管理 AlloyDB 執行個體技能徽章入門課程, 即可證明自己具備下列技能:執行主要 AlloyDB 作業 和工作、從 PostgreSQL 遷移至 AlloyDB、管理 AlloyDB 資料庫,以及 使用 AlloyDB 資料欄引擎加快數據分析查詢。
不想花費大把時間,想在幾分鐘內只靠 SQL,就建立好機器學習模型嗎?透過 BigQuery ML,資料分析師可以運用現有的 SQL 工具和技巧,建立、訓練、評估模型, 並使用模型進行預測,降低機器學習的使用門檻。在 本系列的實驗室,您會測試不同類型的模型,瞭解 優良模型應具備的條件。
在這堂入門課程,您將實際練習使用 Google Cloud 的基礎工具和服務。本課程包含可選擇觀賞的影片, 針對實驗室涵蓋的概念提供更多背景資訊,協助您複習。「Google Cloud 必備知識」 是適合 Google Cloud 學員的第一堂課, 即使您尚未學習或不熟悉雲端知識, 也能從這堂課獲得實務經驗,並應用於第一項 Google Cloud 專案。不管是撰寫 Cloud Shell 指令 和部署第一部虛擬機器,還是在 Kubernetes Engine 或透過負載平衡執行應用程式, 「Google Cloud 必備知識」都是認識平台基本功能的最佳入門資源。
In this fundamental-level course, you will learn the ins and outs of Google Cloud's operations suite running on Google Kubernetes Engine, an important service for generating insights into the health of your applications. It provides a wealth of information in application monitoring, report logging, and diagnoses. The labs in this course will give you hands-on practice with and will teach you how to monitor virtual machines, generate logs and alerts, and create custom metrics for application data. It is recommended that the students have at least earned a Badge by completing the Google Cloud Essentials course. Additional lab experience with the labs in the Baseline - Infrastructure course will also be useful. Looking for a hands-on challenge lab to demonstrate your skills and validate your knowledge? On completing this course, enroll in and finish the additional challenge lab at the end of this course to receive an exclusive Google Cloud digital badge.
Want to scale your data analysis efforts without managing database hardware? Learn the best practices for querying and getting insights from your data warehouse with this interactive series of BigQuery labs. BigQuery is Google's fully managed, NoOps, low cost analytics database. With BigQuery you can query terabytes and terabytes of data without having any infrastructure to manage or needing a database administrator. BigQuery uses SQL and can take advantage of the pay-as-you-go model. BigQuery allows you to focus on analyzing data to find meaningful insights.
In this series of labs you will learn how to use BigQuery to analyze NCAA basketball data with SQL. Build a Machine Learning Model to predict the outcomes of NCAA March Madness basketball tournament games.
In this quest you will get hands-on experience writing infrastructure as code with Terraform.