Viacheslav Melnyk
Membro dal giorno 2022
Campionato Oro
25305 punti
Membro dal giorno 2022
In the last installment of the Dataflow course series, we will introduce the components of the Dataflow operational model. We will examine tools and techniques for troubleshooting and optimizing pipeline performance. We will then review testing, deployment, and reliability best practices for Dataflow pipelines. We will conclude with a review of Templates, which makes it easy to scale Dataflow pipelines to organizations with hundreds of users. These lessons will help ensure that your data platform is stable and resilient to unanticipated circumstances.
In this second installment of the Dataflow course series, we are going to be diving deeper on developing pipelines using the Beam SDK. We start with a review of Apache Beam concepts. Next, we discuss processing streaming data using windows, watermarks and triggers. We then cover options for sources and sinks in your pipelines, schemas to express your structured data, and how to do stateful transformations using State and Timer APIs. We move onto reviewing best practices that help maximize your pipeline performance. Towards the end of the course, we introduce SQL and Dataframes to represent your business logic in Beam and how to iteratively develop pipelines using Beam notebooks.
This course is part 1 of a 3-course series on Serverless Data Processing with Dataflow. In this first course, we start with a refresher of what Apache Beam is and its relationship with Dataflow. Next, we talk about the Apache Beam vision and the benefits of the Beam Portability framework. The Beam Portability framework achieves the vision that a developer can use their favorite programming language with their preferred execution backend. We then show you how Dataflow allows you to separate compute and storage while saving money, and how identity, access, and management tools interact with your Dataflow pipelines. Lastly, we look at how to implement the right security model for your use case on Dataflow.
Complete the intermediate Build a Data Warehouse with BigQuery skill badge course to demonstrate skills in the following: joining data to create new tables, troubleshooting joins, appending data with unions, creating date-partitioned tables, and working with JSON, arrays, and structs in BigQuery.
Complete the intermediate Implement Cloud Security Fundamentals on Google Cloud skill badge course to demonstrate skills in the following: creating and assigning roles with Identity and Access Management (IAM); creating and managing service accounts; enabling private connectivity across virtual private cloud (VPC) networks; restricting application access using Identity-Aware Proxy; managing keys and encrypted data using Cloud Key Management Service (KMS); and creating a private Kubernetes cluster.
Complete the introductory Migrate MySQL Data to Cloud SQL Using Database Migration Service skill badge course to demonstrate skills in the following: migrating MySQL data to Cloud SQL using different job types and connectivity options available in Database Migration Service and migrating MySQL user data when running Database Migration Service jobs.
Complete the intermediate Perform Predictive Data Analysis in BigQuery skill badge course to demonstrate skills in the following: creating datasets in BigQuery by importing CSV and JSON files; harnessing the power of BigQuery with sophisticated SQL analytical concepts, including using BigQuery ML to train an expected goals model on soccer event data and evaluate the impressiveness of World Cup goals.
Earn a skill badge by completing the Share Data Using Google Data Cloud skill badge course, where you will gain practical experience with Google Cloud Data Sharing Partners, which have proprietary datasets that customers can use for their analytics use cases. Customers subscribe to this data, query it within their own platform, then augment it with their own datasets and use their visualization tools for their customer facing dashboards.
Complete the intermediate Manage Data Models in Looker skill badge course to demonstrate skills in the following: maintaining LookML project health; utilizing SQL runner for data validation; employing LookML best practices; optimizing queries and reports for performance; and implementing persistent derived tables and caching policies.
Complete the introductory Build LookML Objects in Looker skill badge course to demonstrate skills in the following: building new dimensions and measures, views, and derived tables; setting measure filters and types based on requirements; updating dimensions and measures; building and refining Explores; joining views to existing Explores; and deciding which LookML objects to create based on business requirements.
Complete the intermediate Create ML Models with BigQuery ML skill badge to demonstrate skills in creating and evaluating machine learning models with BigQuery ML to make data predictions.
Completa il corso introduttivo con badge delle competenze Genera insight dai dati BigQuery per dimostrare le tue competenze nei seguenti ambiti: scrivere query SQL, eseguire query su tabelle pubbliche, caricare dati di esempio in BigQuery, risolvere i problemi di sintassi comuni con lo strumento di convalida query in BigQuery e creare report in Data Studio collegando ai dati di BigQuery.
Complete the introductory Prepare Data for Looker Dashboards and Reports skill badge course to demonstrate skills in the following: filtering, sorting, and pivoting data; merging results from different Looker Explores; and using functions and operators to build Looker dashboards and reports for data analysis and visualization.
Complete the intermediate Develop Serverless Apps with Firebase skill badge course to demonstrate skills in the following: architecting and building serverless web applications with Firebase, utilizing Firestore for database management, automating deployment processes using Cloud Build, and integrating Google Assistant functionality into your applications.
Complete the intermediate Develop Serverless Applications on Cloud Run skill badge course to demonstrate skills in the following: integrating Cloud Run with Cloud Storage for data management, architecting resilient asynchronous systems using Cloud Run and Pub/Sub, constructing REST API gateways powered by Cloud Run, and building and deploying services on Cloud Run.
Completa il badge delle competenze di livello intermedio Ottimizza i costi per Google Kubernetes Engine per dimostrare le tue competenze nei seguenti ambiti: creazione e gestione di cluster multi-tenant, monitoraggio dell'utilizzo delle risorse per spazio dei nomi, configurazione della scalabilità automatica di cluster e pod per l'efficienza, impostazione del bilanciamento del carico per la distribuzione ottimale delle risorse e implementazione di probe di attività e idoneità per garantire l'integrità e la convenienza economica dell'applicazione.
Completa il corso intermedio con badge delle competenze Crea l'infrastruttura con Terraform su Google Cloud per dimostrare le tue competenze nei seguenti ambiti: principi di Infrastructure as Code (IaC) utilizzando Terraform, provisioning e gestione di risorse Google Cloud con configurazioni Terraform, gestione efficace dello stato (locale e remoto) e modularizzazione del codice Terraform per la riusabilità e l'organizzazione.
Lo scopo di questo corso è aiutare coloro che sono qualificati ad avere confidenza per tentare l'esame e aiutare le persone non ancora qualificate a sviluppare il proprio piano per la preparazione.
Complete the intermediate Deploy Kubernetes Applications on Google Cloud skill badge course to demonstrate skills in the following: Configuring and building Docker container images.Creating and managing Google Kubernetes Engine (GKE) clusters.Utilizing kubectl for efficient cluster management.Deploying Kubernetes applications with robust continuous delivery (CD) practices.
Completa il corso introduttivo con badge delle competenze Implementazione di Cloud Load Balancing per Compute Engine per dimostrare le tue competenze nei seguenti ambiti: creazione ed esecuzione del deployment di macchine virtuali in Compute Engine e configurazione di bilanciatori del carico di rete e delle applicazioni.
Ottieni il badge delle competenze introduttivo completando il corso con badge delle competenze Crea un sito web su Google Cloud. Questo corso si basa sulla serie Get Cooking in Cloud e tratta i seguenti argomenti:Deployment di un sito web su Cloud RunHosting di un'app web su Compute EngineCreazione, deployment e scalabilità del tuo sito web su Google Kubernetes EngineMigrazione da un'applicazione monolitica a un'architettura di microservizi utilizzando Cloud Build
Complete the intermediate Engineer Data for Predictive Modeling with BigQuery ML skill badge to demonstrate skills in the following: building data transformation pipelines to BigQuery using Dataprep by Trifacta; using Cloud Storage, Dataflow, and BigQuery to build extract, transform, and load (ETL) workflows; and building machine learning models using BigQuery ML.
Earn a skill badge by completing the Set Up a Google Cloud Network skill badge course, where you will learn how to perform basic networking tasks on Google Cloud Platform - create a custom network, add subnets firewall rules, then create VMs and test the latency when they communicate with each other.
Guadagna un badge delle competenze completando il corso Sviluppa la tua rete Google Cloud, in cui apprenderai diversi modi per eseguire il deployment e il monitoraggio delle applicazioni, tra cui: esplorare i ruoli IAM e aggiungere/rimuovere l'accesso ai progetti, creare reti VPC, eseguire il deployment e il monitoraggio delle VM di Compute Engine, scrivere query SQL, eseguire il deployment e il monitoraggio delle VM in Compute Engine ed eseguire il deployment delle applicazioni utilizzando Kubernetes con più approcci al deployment.
Ottieni il corso intermedio con badge delle competenze Prepara i dati per le API ML su Google Cloud per dimostrare le tue competenze nei seguenti ambiti: pulizia dei dati con Dataprep di Trifacta, esecuzione delle pipeline di dati in Dataflow, creazione dei cluster ed esecuzione dei job Apache Spark in Managed Service for Apache Spark e richiamo delle API ML tra cui l'API Cloud Natural Language, l'API Google Cloud Speech-to-Text e l'API Video Intelligence.
Ottieni un badge delle competenze completando il corso Configura un ambiente di sviluppo di app su Google Cloud, in cui imparerai a creare e connettere un'infrastruttura cloud incentrata sull'archiviazione utilizzando le funzionalità di base delle seguenti tecnologie: Cloud Storage, Identity and Access Management, Cloud Functions e Pub/Sub.