Francisco Colomer
Member since 2023
Diamond League
56397 points
Member since 2023
In this advanced challenge lab, you act as a Data Engineer for Cymbal Direct, a retail company integrating real-time movie review data into a marketing pipeline. You are responsible for building two distinct streaming architectures. First, you will implement a direct, code-free ingestion path using Pub/Sub BigQuery subscriptions. Second, you will deploy a sophisticated Dataflow pipeline that uses JavaScript User-Defined Functions (UDFs) to transform raw text into numerical data before it reaches BigQuery, all while managing high-velocity data generated by a simulated stream.
Complete the Orchestrate Data Lifecycle Automation with Data Agents skill badge to demonstrate your proficiency in modernizing data infrastructure using AI-driven automation. Emphasis is placed on acting as an 'Agent Orchestrator'—leveraging specialized Data Agents to build resilient Dataform pipelines, enforce Dataplex governance, and accelerate data science workflows in Colab Enterprise. A skill badge is an exclusive digital badge issued by Google Cloud in recognition of your proficiency with Google Cloud products and services and tests your ability to apply your knowledge in an interactive hands-on environment. Complete the assessment challenge lab to receive a skill badge that you can share with your network. When you complete this course, you can earn the badge displayed here and claim it on Credly! Boost your cloud career by showing the world the skills you have developed!
In this introductory course, you'll learn how Looker can help you explore, analyze, and visualize your data to drive better decisions. Through a combination of video lectures and demos, you'll discover how to connect to various data sources, build interactive dashboards, and perform effective data analysis. Whether you're a data analyst, BI analyst, data scientist or business user, this course will equip you with the foundational knowledge to start using Looker effectively, regardless of your background.
Create your first Gemini Enterprise application to earn a skill badge! Connect diverse data sources to your application to build a powerful, unified search and analysis engine. Master advanced capabilities like deep research agents, multi-agent ideation, and NotebookLM for focused analysis.
Discover how AI agents drive business impact. You’ll map agent types to your KPIs and explore use cases that solve real bottlenecks. Then, learn how Gemini Enterprise empowers you to build and orchestrate the right agents—from no-code to high-code solutions.
AI Agents represent a major shift beyond traditional large language models (LLMs): instead of simply generating text-based solutions, they can also act autonomously to execute them. This course introduces the fundamentals of AI Agents, how they differ from LLM APIs, and where they add value in the real world. Based on Google’s agents whitepaper, it provides the theoretical foundation needed before writing your first lines of agent code—ideal for developers, architects, and technical decision-makers who want to understand AI systems through the lens of autonomous, goal-directed behavior (and not just text generation). Join the community forum for questions and discussions.
Gain a conceptual overview of AI Agents. Discover how AI Agents use autonomous action and reasoning to solve complex problems. You’ll explore the technical architecture—models, tools, and orchestration—that enables agents to learn, plan, and achieve goals on your behalf.
This course introduces you to event-based applications and teaches you how to use service orchestration and choreography to coordinate microservices. Using lectures and hands-on labs, you learn how to use Workflows, Eventarc, Cloud Tasks, and Cloud Scheduler to build microservices applications on Google Cloud.
In this course, you learn the fundamentals of application development on Google Cloud. You learn best practices for cloud applications, and how to select compute and data options to match your application use cases. You're introduced to generative AI and how it's used to help build applications. You learn about authentication and authorization, application deployment, continuous integration and delivery, and monitoring and performance tuning for your applications running in Google Cloud. Using lectures and hands-on labs, you learn how to get started building and running applications on Google Cloud.
Learn about how you can use Agent Development Kit (ADK) to build complex, production-ready AI agents. This course covers ADK’s open-source framework, moving from simple prompt engineering to a code-first, structured software development approach suitable for enterprise-grade, multi-agent systems.
Complete the introductory Develop AI-Powered Prototypes in Google AI Studio skill badge to demonstrate skills in the following: crafting effective prompts, leveraging multimodal capabilities for image and video analysis, prototyping functional AI-driven applications from templates and text prompts, and utilizing API keys to build and deploy custom AI solutions.
In this video, you'll learn to use Gemini in Google Sheets for advanced data analysis. You'll learn how to create data visualizations, like a scatter plot, using a simple prompt. You'll also learn how to ask Gemini to analyze your data for strategic insights and recommendations, such as how to save money.
In this video, you'll learn to use Gemini in Google Sheets to automatically generate organized trackers from unstructured data. You'll learn how to write a prompt that references other Google Drive files using the "@" symbol. You'll also see how Gemini builds a fully formatted, color-coded table with drop-down menus based on the data in your notes, saving you significant manual effort.
In this video, you'll learn to use Gemini in Google Workspace to manage your calendar from your Gmail inbox. You'll learn how to ask Gemini about your schedule in the side panel and how to create new events without switching apps. You'll also learn how to use Gemini's one-click scheduling feature when it automatically detects event details in an email.
In this video, you'll learn to use Gemini in Google Slides to build polished presentations efficiently. You'll see how to create slides by referencing content directly from your Google Drive files using the "@" symbol. You'll also learn to generate custom, professional images from a simple text prompt to visually enhance your presentation.
In this video, you'll learn to use the "Ask Gemini" feature in Google Sheets to manage your data with natural language. You'll see how to ask Gemini to create dropdowns, highlight sales data, create a pivot table, filter by month, and sort by revenue.
In this video, you'll learn to use the =AI() function in Google Sheets to automate your work. You'll see how to generate text like slogans, summarize paragraphs of customer feedback, and categorize data by classifying inquiries and analyzing sentiment.
In this video, you'll learn to create "what if" scenarios using AI's reasoning. You'll see how to set a "before" scene with a person holding a cake, prompt an action by asking "what would happen if they tripped?", and let the AI generate the plausible "after" image of the cake falling.
In this video, you'll learn the "creative mashup artist" technique for combining images. You'll see how to generate a subject and a scene as two separate images, and then use a final blending prompt to fuse the astronaut from the first image and the court from the second into one new picture.
In this video, you'll learn to art direct your images in the Gemini app. You'll see how to use multi-turn editing to furnish an empty room step-by-step, and how to "remix" a photo by applying the color and texture of a butterfly's wings to a pair of rainboots.
In this video, you'll learn to create imaginative portraits in the Gemini app that maintain your likeness. You'll see how to use the "Subject, Action, Scene, Style" formula for better prompts, transport yourself into a vintage photo, and combine a photo of yourself and your dog into one new image.
In this video, you'll learn to use NotebookLM to manage a large-scale research project. You'll see how to upload all your documents to get summaries, ask deeper strategic questions to find key insights, and use the audio feature to listen to your findings on the go.
In this video, you'll learn to use NotebookLM to analyze raw data and find compelling stories. You'll see how to ground the AI in your specific documents, prompt it to generate newsworthy PR claims, and ask it to find hidden correlations between different data points to uncover new insights.
In this video, you'll learn to use an AI assistant to streamline large-scale event planning. You'll see how to prompt an AI to brainstorm off-site ideas, create fair groups for icebreakers, build a shuttle schedule based on arrival times, and analyze survey feedback for gains and losses.
In this video, you'll learn pro tips for conducting thorough competitive research with Gemini Deep Research. You'll see how to enhance a basic prompt by assigning a persona, specifying a table format for the output, and directing the AI to search specific sources like Reddit for richer insights.
In this video, you'll learn to use Gemini Deep Research to make complex purchasing decisions with confidence. You'll see how to write a nuanced, personal prompt about your specific needs and receive a synthesized, comparative report that cuts through the clutter of online reviews.
In this video, you'll learn to create an animated bar chart race using Gemini Canvas. You'll see how to write a prompt to visualize data over time and then use a follow-up prompt to refine the animation, such as slowing it down for a presentation.
In this video, you'll learn to create complex animated art using Gemini Canvas. You'll see how to write a descriptive prompt to generate a kinetic typography animation with distortion effects, and then use a follow-up prompt to completely change its color style.
In this video, you'll learn to build a reusable, custom "Study Guide Gem." You'll see how to write the core instructions for your Gem once, and then use it again and again to transform messy notes and articles into a perfectly structured study guide with a summary, outline, and glossary.
In this video, you'll learn to turn your study guides into interactive quizzes using Gemini Canvas. You'll see how to upload your notes and use a simple prompt to generate a quiz with multiple choice and fill-in-the-blank questions that provide immediate feedback.
In this video, you'll learn to use Gemini Deep Research to conduct competitive marketing analysis. You'll see how to submit a research prompt, approve the personalized research plan Gemini creates, and receive a detailed, synthesized report on competitor campaigns to inform your strategy.
In this video, you'll learn to build a working prototype of a personalized weather app using Gemini Canvas. You'll see how to specify the cities for a 7-day forecast and then use a follow-up prompt to change the app's theme to a minimal, dark mode interface.
In this video, you'll learn to create an interactive 3D simulation using Gemini Canvas. You'll see how to build a 3D model of the solar system from a single sentence and then use follow-up prompts to add interactive and educational features, like clickable planets and fun facts.
In this video, you'll learn to build a custom personal utility app using Gemini Canvas. You'll see how to write a detailed prompt to create a Pomodoro timer tailored to your specific needs, and then use follow-up prompts to adjust its visual design.
In this video, you'll learn to use the Guided Learning experience in Gemini to build a deep understanding of any topic. You'll see how to start a conversation, engage with Gemini's probing questions, and use interactive tools like diagrams and quizzes to guide your learning journey.
In this video, you'll learn to use a simple text prompt in Gemini Canvas to build a playable game. You'll see how to create a Tic-Tac-Toe game, use a follow-up prompt to customize its theme to a retro 8-bit style, and share the final game with a link.
In this video, you'll learn to use creative prompts in Gemini to make complex research fun and easy to understand. You'll see how to ask Gemini to research a topic and present the findings in a creative format, like a sports scouting report and a head-to-head bracket, to better visualize and compare your options.
In this video, you'll learn to use the Create menu in Gemini Canvas to instantly convert a block of text into a variety of visual formats. You’ll see how to take a brainstormed marketing campaign and turn it into both a shareable infographic and an internal webpage with just one click.
In this video, you'll learn to use Gemini Canvas to turn a simple drawing into a working app. You'll upload a sketch, use a simple prompt to generate code, watch a live preview build itself, and make iterative changes to your app prototype using natural language.
This video covers how NotebookLM's Reports feature dynamically suggests formats to help you create customized, trustworthy analyses of your documents with ease.
This video covers how to build a personalized "Work with Me" agent using Gemini Gems, which helps streamline foundational feedback and makes your meetings more strategic and efficient.
AI Boost Bites is a video series designed to help you leverage Google's AI tools in your daily work. Each episode, under 10 minutes, features a quick video demonstrating a real-world AI use case or topic. After the video, you'll get a challenge to apply what you've learned. It's an easy, interactive way to boost your AI skills and improve your productivity.
This video covers how to create a 'project notebook' in NotebookLM by adding all relevant sources to build a central, searchable knowledge hub for your team.
This video covers how to use NotebookLM for common marketing tasks like analyzing customer feedback, conducting market research, and generating content ideas.
This video covers how to use the Video Overviews feature in NotebookLM to automatically generate a short explainer video based on your source documents.
This video covers how to use the 'Discover Sources' feature in NotebookLM to find and import relevant web-based sources directly into your research project.
This video covers how to use the Mind Maps feature in NotebookLM to automatically create a visual representation of your sources, helping you understand connections and key concepts.
This video covers how to use NotebookLM as a personal research assistant by adding sources, asking questions, and generating new content formats based on your documents.
This video covers how to use Gemini in Gmail to draft new emails, refine their tone, respond with context from Drive files, and use smart reply suggestions.
This video covers how to use the 'Help me create' feature in Google Docs to generate a complete, formatted document by referencing content from other files in your Drive.
This video covers five key ways to use Google's AI tools, including Gemini in Workspace, the Gemini app, and NotebookLM, to enhance your daily productivity.
This video covers how to use Gemini in Gmail to summarize emails, find information, and draft replies, helping you manage your inbox more efficiently.
This video covers how to use Gemini in Slides to automatically generate meeting recaps and draft follow-up emails, which can streamline your post-meeting workflow and save you time.
This video covers how you can leverage Gemini's advanced AI capabilities within Google Sheets to effortlessly pull data and generate insights in minutes, all without the need for any technical or coding background.
This video will cover how to leverage Gemini Gems to create authentic social media posts in your leader's unique voice. Learn to overcome the challenge of scaling executive social presence by training a Gem with writing samples and clear instructions. Discover how to generate engaging posts quickly, saving time while amplifying thought leadership and ensuring authenticity.
This video covers how you can create your own Brevity Gem to summarize and transform messy notes or long documents into clear, concise, executive-ready summaries.
This video covers how to use Gemini and Apps Script to automate manual tasks across Google Workspace. You'll learn to prompt Gemini to generate Apps Script code that automatically drafts email reminders in Google Sheets for tasks not marked 'Complete.' Automate your workflow with little to no technical expertise, freeing up time for more important work and eliminating manual follow-ups.
This video covers how you can leverage Notebook LM to "eat the frog" on your to-do list by automating complex tasks like summarizing legislation and mapping services, saving you hours of work.
This video covers how to eliminate tedious manual data entry using Gemini. Learn how to take a picture or screenshot of data (from PDFs, paper, or images) and prompt Gemini to instantly convert it into a structured Google Sheet. Discover this simple hack to save countless hours transcribing data, turning Gemini into your personal data entry assistant. Just snap, prompt, and export!
AI Boost Bites is a video series designed to help you leverage Google's AI tools in your daily work. Each episode, under 10 minutes, features a quick video demonstrating a real-world AI use case or topic. After the video, you'll get a challenge to apply what you've learned. It's an easy, interactive way to boost your AI skills and improve your productivity.
This video will cover how to use NotebookLM to gather and analyze publicly available information, combine it with internal documents, and extract key competitive insights.
This video covers how to personalize your Gemini results in Google Workspace. Learn to incorporate documents and research papers directly into your prompts using the "@" symbol to get more targeted and relevant AI output tailored to your needs.
This video covers how you can use Gemini to summarize long documents in Google Workspace, so you can quickly get the information you need and save time. You'll learn how to use Gemini to summarize entire documents or just selected text, as well as how to use Gemini in Drive to summarize across multiple files.
This video covers prompt engineering fundamentals for effective AI communication. Learn a simple framework (Persona, Task, Context, Format) to craft clear prompts, getting better, faster results from Gemini in Google Workspace. Discover how to use natural language, be specific, and iterate for optimal AI assistance.
This video will cover how you can leverage Gemini's advanced AI capabilities in Google Docs to brainstorm ideas, draft various marketing content, and collaborate with your team.
This video covers how NotebookLM can revolutionize customer insight gathering from call or chat transcripts. You'll learn to upload PDF transcripts of hundreds of conversations (even multilingual ones!) and quickly extract key themes, trending topics, and actionable insights without listening for hours. Discover how to save findings, share notebooks, and even generate interactive podcast summaries of your data.
This video covers how to create your own Gemini Gems, advanced AI capabilities that can automate repetitive tasks and supercharge your productivity.
In this course, you'll learn how to build software with Gemini, Google’s generative AI. The best part is: Anyone can do this. It’s okay if you've never written code before. You just need to be curious and motivated, and we'll help you get from zero to app.
In this video, you'll learn to build a working music synthesizer using a simple text prompt in Gemini Canvas. You'll see how to specify controls like waveform, attack, and sustain, and then interact with the generated instrument to create and experiment with your own unique sounds.
Earn a skill badge by completing the Build a Smart Cloud Application with Vibe Coding and MCP course, where you will learn to leverage the power of Google's AI coding assistant and MCP servers
Complete the Streamline Application Development with Gemini CLI skill badge to demonstrate your proficiency in using the full capabilities of Gemini CLI in application development tasks. You will be tasked with defining multi-step plans, creating a reusable CLI extension, managing context, experimenting with checkpoints and deploying to Cloud Run all from Gemini CLI. A skill badge is an exclusive digital badge issued by Google Cloud in recognition of your proficiency with Google Cloud products and services and tests your ability to apply your knowledge in an interactive hands-on environment. Complete the assessment challenge lab, to receive a skill badge that you can share with your network. When you complete this course, you can earn the badge displayed here and claim it on Credly! Boost your cloud career by showing the world the skills you have developed!
This course is designed for app developers and DevOps engineers who want to work smarter by using Gemini CLI, a generative AI agent made for the terminal and powered by Gemini. This course discusses Gemini CLI installation and configuration, and introduces use cases and security best practices. It explains commands, tools, MCP servers, and extensions. With a hands-on exercise, you'll install and configure Gemini CLI and use it to analyze code and build and modify an app.
Welcome to the "AI Infrastructure: Networking Techniques" course. In this course, you'll learn to leverage Google Cloud's high-bandwidth, low-latency infrastructure to optimize data transfer and communication between all the components of your AI system. By the end, you'll grasp the critical role networking plays across the entire AI pipeline from data ingestion and training to inference and be able to apply best practices to ensure your workloads run at maximum speed.
In this course, you’ll take a comprehensive journey through the storage solutions available on Google Cloud, specifically tailored for AI and high-performance computing (HPC) workloads. You’ll learn how to choose the right storage for each stage of the ML lifecycle. You’ll explore how to optimize for I/O performance during training, manage massive datasets for data preparation, and serve model artifacts with low latency. Through practical examples and demonstrations, you’ll gain the expertise to design robust storage solutions that accelerate your AI innovation.
This course provides a comprehensive guide to deploying, managing, and optimizing AI and high-performance computing (HPC) workloads on Google Cloud. Through a series of lessons and practical demonstrations, you’ll explore diverse deployment strategies, ranging from highly customizable environments using Google Compute Engine (GCE) to managed solutions like Google Kubernetes Engine (GKE). Specifically, you’ll learn how to create clusters and deploy GKE for inference.
Welcome to the Cloud TPUs course. We'll explore the advantages and disadvantages of TPUs in various scenarios and compare different TPU accelerators to help you choose the right fit. You'll learn strategies to maximize performance and efficiency for your AI models and understand the significance of GPU/TPU interoperability for flexible machine learning workflows. Through engaging content and practical demos, we'll guide you step-by-step in leveraging TPUs effectively.
Curious about the powerful hardware behind AI? This module breaks down performance-optimized AI computers, showing you why they're so important. We'll explore how CPUs, GPUs, and TPUs make AI tasks super fast, what makes each one unique, and how AI software gets the most out of them. By the end, you'll know exactly how to pick the right GPU for your AI projects, helping you make smart choices for your AI workloads.
Ready to get started with AI Hypercomputers? This course makes it easy! We'll cover the basics of what they are and how they help AI with AI workloads. You'll learn about the different components inside a hypercomputer, like GPUs, TPUs, and CPUs, and discover how to pick the right deployment approach for your needs.
In this Google DeepMind course you will focus on the training process for machine learning models. You will learn how to spot and mitigate issues when training a model, such as overfitting and underfitting. In practical coding labs, you will implement and evaluate the multilayer perceptron for simple classification tasks. This will provide insights into the mechanics of training a neural network model and the backpropagation algorithm. Research case studies will demonstrate how neural networks power real-world models. Additionally, you will consider the broader social impacts of innovation by looking beyond immediate benefits to anticipate potential risks, safety concerns, and further-reaching societal consequences.
In this Google DeepMind course you will learn how to prepare text data for language models to process. You will investigate the tools and techniques used to prepare, structure, and represent text data for language models, with a focus on tokenization and embeddings. You will be encouraged to think critically about the decisions behind data preparation, and what biases within the data may be introduced into models. You will analyze trade-offs, learn how to work with vectors and matrices, how meaning is represented in language models. Finally, you will practice designing a dataset ethically using the Data Cards process, ensuring transparency, accountability, and respect for community values in AI development.
In this Google DeepMind course, you will learn the fundamentals of language models and gain a high-level understanding of the machine learning development pipeline. You will consider the strengths and limitations of traditional n-gram models and advanced transformer models. Practical coding labs will enable you to develop insights into how machine learning models work and how they can be used to generate text and identify patterns in language. Through real-world case studies, you will build an understanding around how research engineers operate. Drawing on these insights you will identify problems that you wish to tackle in your own community and consider how to leverage the power of machine learning responsibly to address these problems within a global and local context.
The course aims to train Google technical sales partners on the business value discovery process using proprietary content.
Want to learn more about Google Cloud? Grow your Google Cloud knowledge, strengthen your skills to win with customers, and scale your Google Cloud business. Find it here in one handy location.
This course enables system integrators and partners to understand the principles of automated migrations, plan legacy system migrations to Google Cloud leveraging G4 Platform, and execute a trial code conversion.
Complete the introductory Create and Manage AlloyDB Instances skill badge to demonstrate skills in the following: performing core AlloyDB operations and tasks, migrating to AlloyDB from PostgreSQL, administering an AlloyDB database, and accelerating analytical queries using the AlloyDB Columnar Engine.
Complete the introductory Create and Manage Bigtable Instances skill badge to demonstrate skills in the following: creating instances, designing schemas, querying data, and performing administrative tasks in Bigtable including monitoring performance and configuring node autoscaling and replication.
Complete the introductory Create and Manage Cloud Spanner Instances skill badge to demonstrate skills in the following: creating and interacting with Cloud Spanner instances and databases; loading Cloud Spanner databases using various techniques; backing up Cloud Spanner databases; defining schemas and understanding query plans; and deploying a Modern Web App connected to a Cloud Spanner instance.
Complete the introductory Create and Manage Cloud SQL for PostgreSQL Instances skill badge to demonstrate skills in the following: migrating, configuring, and managing Cloud SQL for PostgreSQL instances and databases.
This course introduces Vertex AI Studio, a tool to interact with generative AI models, prototype business ideas, and launch them into production. Through an immersive use case, engaging lessons, and a hands-on lab, you’ll explore the prompt-to-product lifecycle and learn how to leverage Vertex AI Studio for Gemini multimodal applications, prompt design, prompt engineering, and model tuning. The aim is to enable you to unlock the potential of gen AI in your projects with Vertex AI Studio.
This is an introductory level microlearning course aimed at explaining what Generative AI is, how it is used, and how it differs from traditional machine learning methods. It also covers Google Tools to help you develop your own Gen AI apps.
This course introduces you to the world of reliable deep learning, a critical discipline focused on developing machine learning models that not only make accurate predictions but also understand and communicate their own uncertainty. You'll learn how to create AI systems that are trustworthy, robust, and adaptable, particularly in high-stakes scenarios where errors can have significant consequences.
Complete the introductory Migrate MySQL Data to Cloud SQL Using Database Migration Service skill badge course to demonstrate skills in the following: migrating MySQL data to Cloud SQL using different job types and connectivity options available in Database Migration Service and migrating MySQL user data when running Database Migration Service jobs.
This course is intended to give architects, engineers, and developers the skills required to help enterprise customers architect, plan, execute, and test database migration projects. Through a combination of presentations, demos, and hands-on labs participants move databases to Google Cloud while taking advantage of various services. This course covers how to move on-premises, enterprise databases like SQL Server to Google Cloud (Compute Engine and Cloud SQL) and Oracle to Google Cloud bare metal.
In this course, you learn to analyze and choose the right database for your needs, to effectively develop applications on Google Cloud. You explore relational and NoSQL databases, dive into Cloud SQL, AlloyDB, and Spanner, and learn how to align database strengths with your application requirements, including those of generative AI. Gain hands-on experience configuring Vector Search and migrating applications to the cloud.
Google Cloud Fundamentals: Core Infrastructure introduces important concepts and terminology for working with Google Cloud. Through videos and hands-on labs, this course presents and compares many of Google Cloud's computing and storage services, along with important resource and policy management tools.
This course takes a real-world approach to the ML Workflow through a case study. An ML team faces several ML business requirements and use cases. The team must understand the tools required for data management and governance and consider the best approach for data preprocessing. The team is presented with three options to build ML models for two use cases. The course explains why they would use AutoML, BigQuery ML, or custom training to achieve their objectives.
This course explores what ML is and what problems it can solve. The course also discusses best practices for implementing machine learning. You’re introduced to Vertex AI, a unified platform to quickly build, train, and deploy AutoML machine learning models. The course discusses the five phases of converting a candidate use case to be driven by machine learning, and why it’s important to not skip them. The course ends with recognizing the biases that ML can amplify and how to recognize them.
Complete the intermediate Engineer Data for Predictive Modeling with BigQuery ML skill badge to demonstrate skills in the following: building data transformation pipelines to BigQuery using Dataprep by Trifacta; using Cloud Storage, Dataflow, and BigQuery to build extract, transform, and load (ETL) workflows; and building machine learning models using BigQuery ML.
Complete the intermediate Build a Data Warehouse with BigQuery skill badge course to demonstrate skills in the following: joining data to create new tables, troubleshooting joins, appending data with unions, creating date-partitioned tables, and working with JSON, arrays, and structs in BigQuery.
Complete the introductory Prepare Data for ML APIs on Google Cloud skill badge to demonstrate skills in the following: cleaning data with Dataprep by Trifacta, running data pipelines in Dataflow, creating clusters and running Apache Spark jobs in Managed Service for Apache Spark, and calling ML APIs including the Cloud Natural Language API, Google Cloud Speech-to-Text API, and Video Intelligence API.
In the last installment of the Dataflow course series, we will introduce the components of the Dataflow operational model. We will examine tools and techniques for troubleshooting and optimizing pipeline performance. We will then review testing, deployment, and reliability best practices for Dataflow pipelines. We will conclude with a review of Templates, which makes it easy to scale Dataflow pipelines to organizations with hundreds of users. These lessons will help ensure that your data platform is stable and resilient to unanticipated circumstances.
In this second installment of the Dataflow course series, we are going to be diving deeper on developing pipelines using the Beam SDK. We start with a review of Apache Beam concepts. Next, we discuss processing streaming data using windows, watermarks and triggers. We then cover options for sources and sinks in your pipelines, schemas to express your structured data, and how to do stateful transformations using State and Timer APIs. We move onto reviewing best practices that help maximize your pipeline performance. Towards the end of the course, we introduce SQL and Dataframes to represent your business logic in Beam and how to iteratively develop pipelines using Beam notebooks.
This course is part 1 of a 3-course series on Serverless Data Processing with Dataflow. In this first course, we start with a refresher of what Apache Beam is and its relationship with Dataflow. Next, we talk about the Apache Beam vision and the benefits of the Beam Portability framework. The Beam Portability framework achieves the vision that a developer can use their favorite programming language with their preferred execution backend. We then show you how Dataflow allows you to separate compute and storage while saving money, and how identity, access, and management tools interact with your Dataflow pipelines. Lastly, we look at how to implement the right security model for your use case on Dataflow.
Incorporating machine learning into data pipelines increases the ability to extract insights from data. This course covers ways machine learning can be included in data pipelines on Google Cloud. For little to no customization, this course covers AutoML. For more tailored machine learning capabilities, this course introduces Notebooks and BigQuery machine learning (BigQuery ML). Also, this course covers how to productionalize machine learning solutions by using Vertex AI.
In this intermediate course, you will learn to design, build, and optimize robust batch data pipelines on Google Cloud. Moving beyond fundamental data handling, you will explore large-scale data transformations and efficient workflow orchestration, essential for timely business intelligence and critical reporting. Get hands-on practice using Dataflow for Apache Beam and Serverless for Apache Spark (Dataproc Serverless) for implementation, and tackle crucial considerations for data quality, monitoring, and alerting to ensure pipeline reliability and operational excellence. A basic knowledge of data warehousing, ETL/ELT, SQL, Python, and Google Cloud concepts is recommended.
In this course you will get hands-on in order to work through real-world challenges faced when building streaming data pipelines. The primary focus is on managing continuous, unbounded data with Google Cloud products.
While the traditional approaches of using data lakes and data warehouses can be effective, they have shortcomings, particularly in large enterprise environments. This course introduces the concept of a data lakehouse and the Google Cloud products used to create one. A lakehouse architecture uses open-standard data sources and combines the best features of data lakes and data warehouses, which addresses many of their shortcomings.
This course introduces the Google Cloud big data and machine learning products and services that support the data-to-AI lifecycle. It explores the processes, challenges, and benefits of building a big data pipeline and machine learning models with Vertex AI on Google Cloud.
This course helps learners create a study plan for the PDE (Professional Data Engineer) certification exam. Learners explore the breadth and scope of the domains covered in the exam. Learners assess their exam readiness and create their individual study plan.