Viacheslav Melnyk
Jest członkiem od 2022
Liga złota
25305 pkt.
Jest członkiem od 2022
In the last installment of the Dataflow course series, we will introduce the components of the Dataflow operational model. We will examine tools and techniques for troubleshooting and optimizing pipeline performance. We will then review testing, deployment, and reliability best practices for Dataflow pipelines. We will conclude with a review of Templates, which makes it easy to scale Dataflow pipelines to organizations with hundreds of users. These lessons will help ensure that your data platform is stable and resilient to unanticipated circumstances.
In this second installment of the Dataflow course series, we are going to be diving deeper on developing pipelines using the Beam SDK. We start with a review of Apache Beam concepts. Next, we discuss processing streaming data using windows, watermarks and triggers. We then cover options for sources and sinks in your pipelines, schemas to express your structured data, and how to do stateful transformations using State and Timer APIs. We move onto reviewing best practices that help maximize your pipeline performance. Towards the end of the course, we introduce SQL and Dataframes to represent your business logic in Beam and how to iteratively develop pipelines using Beam notebooks.
This course is part 1 of a 3-course series on Serverless Data Processing with Dataflow. In this first course, we start with a refresher of what Apache Beam is and its relationship with Dataflow. Next, we talk about the Apache Beam vision and the benefits of the Beam Portability framework. The Beam Portability framework achieves the vision that a developer can use their favorite programming language with their preferred execution backend. We then show you how Dataflow allows you to separate compute and storage while saving money, and how identity, access, and management tools interact with your Dataflow pipelines. Lastly, we look at how to implement the right security model for your use case on Dataflow.
Complete the intermediate Build a Data Warehouse with BigQuery skill badge course to demonstrate skills in the following: joining data to create new tables, troubleshooting joins, appending data with unions, creating date-partitioned tables, and working with JSON, arrays, and structs in BigQuery.
Complete the intermediate Implement Cloud Security Fundamentals on Google Cloud skill badge course to demonstrate skills in the following: creating and assigning roles with Identity and Access Management (IAM); creating and managing service accounts; enabling private connectivity across virtual private cloud (VPC) networks; restricting application access using Identity-Aware Proxy; managing keys and encrypted data using Cloud Key Management Service (KMS); and creating a private Kubernetes cluster.
Complete the introductory Migrate MySQL Data to Cloud SQL Using Database Migration Service skill badge course to demonstrate skills in the following: migrating MySQL data to Cloud SQL using different job types and connectivity options available in Database Migration Service and migrating MySQL user data when running Database Migration Service jobs.
Complete the intermediate Perform Predictive Data Analysis in BigQuery skill badge course to demonstrate skills in the following: creating datasets in BigQuery by importing CSV and JSON files; harnessing the power of BigQuery with sophisticated SQL analytical concepts, including using BigQuery ML to train an expected goals model on soccer event data and evaluate the impressiveness of World Cup goals.
Earn a skill badge by completing the Share Data Using Google Data Cloud skill badge course, where you will gain practical experience with Google Cloud Data Sharing Partners, which have proprietary datasets that customers can use for their analytics use cases. Customers subscribe to this data, query it within their own platform, then augment it with their own datasets and use their visualization tools for their customer facing dashboards.
Complete the intermediate Manage Data Models in Looker skill badge course to demonstrate skills in the following: maintaining LookML project health; utilizing SQL runner for data validation; employing LookML best practices; optimizing queries and reports for performance; and implementing persistent derived tables and caching policies.
Complete the introductory Build LookML Objects in Looker skill badge course to demonstrate skills in the following: building new dimensions and measures, views, and derived tables; setting measure filters and types based on requirements; updating dimensions and measures; building and refining Explores; joining views to existing Explores; and deciding which LookML objects to create based on business requirements.
Complete the intermediate Create ML Models with BigQuery ML skill badge to demonstrate skills in creating and evaluating machine learning models with BigQuery ML to make data predictions.
Ukończ szkolenie wprowadzające Uzyskiwanie statystyk z danych BigQuery, aby zdobyć odznakę potwierdzającą zdobycie następujących umiejętności: pisanie zapytań SQL, tworzenie zapytań dotyczących tabel publicznych, wczytywanie przykładowych danych w BigQuery, naprawianie typowych błędów składniowych przy użyciu walidatora zapytań w BigQuery oraz tworzenie raportów w Data Studio przez tworzenie połączenia z danymi BigQuery.
Complete the introductory Prepare Data for Looker Dashboards and Reports skill badge course to demonstrate skills in the following: filtering, sorting, and pivoting data; merging results from different Looker Explores; and using functions and operators to build Looker dashboards and reports for data analysis and visualization.
Complete the intermediate Develop Serverless Apps with Firebase skill badge course to demonstrate skills in the following: architecting and building serverless web applications with Firebase, utilizing Firestore for database management, automating deployment processes using Cloud Build, and integrating Google Assistant functionality into your applications.
Complete the intermediate Develop Serverless Applications on Cloud Run skill badge course to demonstrate skills in the following: integrating Cloud Run with Cloud Storage for data management, architecting resilient asynchronous systems using Cloud Run and Pub/Sub, constructing REST API gateways powered by Cloud Run, and building and deploying services on Cloud Run.
Complete the intermediate Optimize Costs for Google Kubernetes Engine skill badge course to demonstrate skills in the following: creating and managing multi-tenant clusters, monitoring resource usage by namespace, configuring cluster and pod autoscaling for efficiency, setting up load balancing for optimal resource distribution, and implementing liveness and readiness probes to ensure application health and cost-effectiveness.
Complete the intermediate Build Infrastructure with Terraform on Google Cloud skill badge to demonstrate skills in the following: Infrastructure as Code (IaC) principles using Terraform, provisioning and managing Google Cloud resources with Terraform configurations, effective state management (local and remote), and modularizing Terraform code for reusability and organization.
Learn how to use Gemini Notebook to create a personalized study guide for the Professional Cloud Architect certification exam. You'll review Gemini Notebook features, create a notebook in Gemini Notebook, and learn how to use a study guide to practice for a certification exam.
Complete the intermediate Deploy Kubernetes Applications on Google Cloud skill badge course to demonstrate skills in the following: Configuring and building Docker container images.Creating and managing Google Kubernetes Engine (GKE) clusters.Utilizing kubectl for efficient cluster management.Deploying Kubernetes applications with robust continuous delivery (CD) practices.
Ukończ szkolenie wprowadzające Wdrażanie równoważenia obciążenia Cloud Load Balancing w Compute Engine, aby zdobyć odznakę potwierdzającą zdobycie następujących umiejętności: tworzenie i wdrażanie maszyn wirtualnych w Compute Engine oraz konfigurowanie sieciowych systemów równoważenia obciążenia i systemów równoważenia obciążenia aplikacji.
Aby zdobyć odznakę umiejętności potwierdzającą opanowanie podstaw, ukończ kurs wprowadzający Tworzenie witryny w Google Cloud. Jest on oparty na serii materiałów wideo Get Cooking in Cloud i obejmuje te tematy: Wdrażanie witryny w Cloud RunHostowanie aplikacji internetowej w Compute EngineTworzenie, wdrażanie i skalowanie witryny w Google Kubernetes EngineMigracja z aplikacji monolitycznej do architektury mikroserwisów przy użyciu Cloud Build
Complete the intermediate Engineer Data for Predictive Modeling with BigQuery ML skill badge to demonstrate skills in the following: building data transformation pipelines to BigQuery using Dataprep by Trifacta; using Cloud Storage, Dataflow, and BigQuery to build extract, transform, and load (ETL) workflows; and building machine learning models using BigQuery ML.
Earn a skill badge by completing the Set Up a Google Cloud Network skill badge course, where you will learn how to perform basic networking tasks on Google Cloud Platform - create a custom network, add subnets firewall rules, then create VMs and test the latency when they communicate with each other.
Aby zdobyć odznakę umiejętności, ukończ szkolenie Budowanie sieci w Google Cloud, w trakcie którego poznasz różne sposoby wdrażania i monitorowania aplikacji i dowiesz się, jak: przeglądać role uprawnień, dodawać/usuwać dostęp do projektu, tworzyć sieci VPC, wdrażać i monitorować maszyny wirtualne Compute Engine, pisać zapytania SQL oraz wdrażać aplikacje przy użyciu różnych metod w Kubernetes.
Ukończ szkolenie wprowadzające Przygotowywanie danych do użycia z interfejsami ML w Google Cloud, aby zdobyć odznakę potwierdzającą zdobycie następujących umiejętności: czyszczenie danych przy użyciu usługi Dataprep firmy Trifacta, uruchamianie potoków danych w Dataflow, tworzenie klastrów i uruchamianie zadań Apache Spark w Managed Service for Apache Spark, a także wywoływanie interfejsów API dotyczących uczenia maszynowego, w tym Cloud Natural Language API, Google Cloud Speech-to-Text API oraz Video Intelligence API.
Aby zdobyć odznakę umiejętności, ukończ szkolenie Konfigurowanie środowiska programistycznego w Google Cloud, w trakcie którego dowiesz się, jak utworzyć i podłączyć infrastrukturę w chmurzę do przechowywania danych przy użyciu podstawowych funkcji technologii Cloud Storage, Identity and Access Management, Cloud Functions oraz Pub/Sub.