Axel VILLANUEVA
成为会员时间:2024
黄金联赛
41870 积分
成为会员时间:2024
In the last installment of the Dataflow course series, we will introduce the components of the Dataflow operational model. We will examine tools and techniques for troubleshooting and optimizing pipeline performance. We will then review testing, deployment, and reliability best practices for Dataflow pipelines. We will conclude with a review of Templates, which makes it easy to scale Dataflow pipelines to organizations with hundreds of users. These lessons will help ensure that your data platform is stable and resilient to unanticipated circumstances.
Learn how to use Gemini Notebook to create a personalized study guide for the Professional Data Engineer certification exam. You'll review Gemini Notebook features, create a notebook in Gemini Notebook, and learn how to use a study guide to practice for a certification exam.
In this second installment of the Dataflow course series, we are going to be diving deeper on developing pipelines using the Beam SDK. We start with a review of Apache Beam concepts. Next, we discuss processing streaming data using windows, watermarks and triggers. We then cover options for sources and sinks in your pipelines, schemas to express your structured data, and how to do stateful transformations using State and Timer APIs. We move onto reviewing best practices that help maximize your pipeline performance. Towards the end of the course, we introduce SQL and Dataframes to represent your business logic in Beam and how to iteratively develop pipelines using Beam notebooks.
This course is part 1 of a 3-course series on Serverless Data Processing with Dataflow. In this first course, we start with a refresher of what Apache Beam is and its relationship with Dataflow. Next, we talk about the Apache Beam vision and the benefits of the Beam Portability framework. The Beam Portability framework achieves the vision that a developer can use their favorite programming language with their preferred execution backend. We then show you how Dataflow allows you to separate compute and storage while saving money, and how identity, access, and management tools interact with your Dataflow pipelines. Lastly, we look at how to implement the right security model for your use case on Dataflow.
Incorporating machine learning into data pipelines increases the ability to extract insights from data. This course covers ways machine learning can be included in data pipelines on Google Cloud. For little to no customization, this course covers AutoML. For more tailored machine learning capabilities, this course introduces Notebooks and BigQuery machine learning (BigQuery ML). Also, this course covers how to productionalize machine learning solutions by using Vertex AI.
In this course you will get hands-on in order to work through real-world challenges faced when building streaming data pipelines. The primary focus is on managing continuous, unbounded data with Google Cloud products.
完成入门级技能徽章课程在 Google Cloud 上为机器学习 API 准备数据,展示以下技能: 使用 Dataprep by Trifacta 清理数据、在 Dataflow 中运行数据流水线、在 Managed Service for Apache Spark 中创建集群和运行 Apache Spark 作业,以及调用机器学习 API,包括 Cloud Natural Language API、Google Cloud Speech-to-Text API 和 Video Intelligence API。
完成中级技能徽章课程利用 BigQuery ML 构建预测模型时的数据工程处理, 展示自己在以下方面的技能:利用 Dataprep by Trifacta 构建 BigQuery 数据转换流水线; 利用 Cloud Storage、Dataflow 和 BigQuery 构建提取、转换和加载 (ETL) 工作流; 以及利用 BigQuery ML 构建机器学习模型。
In this intermediate course, you will learn to design, build, and optimize robust batch data pipelines on Google Cloud. Moving beyond fundamental data handling, you will explore large-scale data transformations and efficient workflow orchestration, essential for timely business intelligence and critical reporting. Get hands-on practice using Dataflow for Apache Beam and Serverless for Apache Spark (Dataproc Serverless) for implementation, and tackle crucial considerations for data quality, monitoring, and alerting to ensure pipeline reliability and operational excellence. A basic knowledge of data warehousing, ETL/ELT, SQL, Python, and Google Cloud concepts is recommended.
完成保护 Lakehouse 数据这一入门级技能徽章课程,展示您在以下方面的技能:使用 IAM、BigQuery、Lakehouse 和 Knowledge Catalog 中的 Data Catalog 创建和保护 Lakehouse 表。
完成入门技能徽章课程使用 Knowledge Catalog 构建数据网格,展示以下方面的技能:使用 Knowledge Catalog 构建数据网格, 以在 Google Cloud 上实现数据安全、治理和发现。您将在 Knowledge Catalog 中练习和测试自己在标记资产、分配 IAM 角色和评估数据质量方面的技能。
完成入门级技能徽章课程“使用 Dataplex 整理和治理数据”,展示您在以下方面的技能:创建 Dataplex 资产、创建切面类型,以及将切面应用于 Dataplex 中的条目。 技能徽章通过动手实验和挑战赛形式的评估,检验您对特定产品的实际知识掌握情况。完成课程即可获得徽章, 也可直接参加实验室挑战赛,快速获得徽章。徽章可证明您掌握技能的熟练程度,提升您的专业形象,最终助您获得更多职业机会。欢迎访问您的个人资料,并跟踪您已获得的徽章。
This course provides an introduction to using Terraform for Google Cloud. It enables learners to describe how Terraform can be used to implement infrastructure as code and to apply some of its key features and functionalities to create and manage Google Cloud infrastructure. Learners will get hands-on practice building and managing Google Cloud resources using Terraform.
完成在 Google Cloud 上使用 Terraform 构建基础设施技能徽章中级课程, 展示您在以下方面的技能:在使用 Terraform 时遵循基础设施即代码 (IaC) 原则;利用 Terraform 配置 来预配和管理 Google Cloud 资源;管理有效状态(本地和远程);以及将 Terraform 代码模块化,以方便重复使用和整理。
各种规模的组织都在利用云的强大功能和灵活性来改变运营方式。然而,有效管理和扩缩云资源可能是一项复杂的任务。本课程探讨了云中现代运营、可靠性和弹性方面的基本概念,以及 Google Cloud 如何帮助支持这些工作。 作为 Cloud Digital Leader 学习路线的一部分,本课程旨在帮助个人在自己的岗位上成长,并为企业谋求未来发展。
随着组织将数据和应用迁移到云端,他们必须妥善应对快速变化的安全挑战格局。本课程将探讨云安全基础知识、Google Cloud 基于安全性设计的基础设施的价值以及纵深防御策略,并重点介绍依托 AI 的运营和合规工具如何帮助组织满足严格的全球监管要求。 作为“Cloud Digital Leader 学习路线”的一部分,本课程旨在帮助个人提升职业素养,并为企业谋求未来发展。
许多传统企业都在使用无法满足现代客户期望的旧式系统和应用。业务领导者往往需要在维护老旧的 IT 系统和投资新产品与服务之间做出选择。本课程探讨了这些挑战,并提供了利用云技术克服这些挑战的解决方案。 作为 Cloud Digital Leader 学习路线的一部分,本课程旨在帮助个人在工作中成长,并为企业打造美好未来。
人工智能 (AI) 和机器学习 (ML) 代表着信息技术领域的重要演变,正在迅速改变各行各业。《利用 Google Cloud 人工智能进行创新》探讨了组织如何使用 AI 和机器学习来转变业务流程。 本课程是“Cloud Digital Leader 学习路线”的一部分,旨在帮助个人在自己的角色中不断成长,并为企业谋求未来发展。
云技术是一项强大的资产,与数据相结合后,它将成为创新和提升客户体验的催化剂。《使用 Google Cloud 探索数据转换》课程探讨了组织如何利用云技术让数据更易于访问、更具实用价值和商业价值。 作为“Cloud Digital Leader 学习路线”的一部分,本课程旨在帮助个人提升职业素养,并为企业谋求未来发展。
数字化转型是现代组织的必经之路,而扎实掌握云计算基础知识,是推动实质性创新的第一步。《利用 Google Cloud 实现数字化转型》课程将讲解核心技术与战略框架,助力组织实现运营现代化。本课程介绍关于云的基本概念、全球网络基础设施和责任共担模式,帮助领导者从容推进上云进程。 作为 Cloud Digital Leader 学习路线的一部分,本课程旨在帮助学员提升岗位能力,从而为企业打造美好未来。
完成中级技能徽章课程使用 BigQuery 构建数据仓库,展示以下技能: 联接数据以创建新表、排查联接故障、使用并集附加数据、创建日期分区表, 以及在 BigQuery 中使用 JSON、数组和结构体。
While the traditional approaches of using data lakes and data warehouses can be effective, they have shortcomings, particularly in large enterprise environments. This course introduces the concept of a data lakehouse and the Google Cloud products used to create one. A lakehouse architecture uses open-standard data sources and combines the best features of data lakes and data warehouses, which addresses many of their shortcomings.
This course introduces the Google Cloud big data and machine learning products and services that support the data-to-AI lifecycle. It explores the processes, challenges, and benefits of building a big data pipeline and machine learning models with Vertex AI on Google Cloud.