Use Serverless for Apache Spark to Load BigQuery Reviews

Sashrika A. · Reviewed حوالي سنة ago

Rushikesh N. · Reviewed حوالي سنة ago

Rutik M. · Reviewed حوالي سنة ago

Vimal R. · Reviewed حوالي سنة ago

Arturo Eduardo M. · Reviewed حوالي سنة ago

RANGA C. · Reviewed حوالي سنة ago

Marcos G. · Reviewed حوالي سنة ago

Javier C. · Reviewed حوالي سنة ago

Joel V. · Reviewed حوالي سنة ago

lab takes a while to register that I've completed the step

Adaeze A. · Reviewed حوالي سنة ago

Javier V. · Reviewed حوالي سنة ago

Lars L. · Reviewed حوالي سنة ago

Christopher M. · Reviewed حوالي سنة ago

Jorge M. · Reviewed حوالي سنة ago

Arnaud M. · Reviewed حوالي سنة ago

Arabind M. · Reviewed حوالي سنة ago

Noguchi M. · Reviewed حوالي سنة ago

Edward K. · Reviewed حوالي سنة ago

As a beginner in the world of Date Engineering on GCP, I struggle to understand the point of Dataproc in this lab. Also, I wanted to let you new that during the spark execute command I got the following error a couple of times. After the third try, it worked: ERROR: (gcloud.beta.dataproc.batches.submit.pyspark) Batch job is FAILED. Detail: Insufficient 'CPUS' quota. Requested 12.0, available 11.0. Your resource request exceeds your available quota. See https://cloud.google.com/compute/resource-usage. Use https://cloud.google.com/docs/quotas/view-manage#requesting_higher_quota to request additional quota. Running auto diagnostics on the batch. It may take few minutes before diagnostics output is available. Please check diagnostics output by running 'gcloud dataproc batches describe' command.

Andrea T. · Reviewed حوالي سنة ago

Beini W. · Reviewed حوالي سنة ago

Ronald Alberto R. · Reviewed حوالي سنة ago

Antonio B. · Reviewed حوالي سنة ago

I see opportunities to improve this quick lab: 1) The lab is missing one step which is granting permissions to get the dataproc cluster. I've got the following error message in the VM while executing the batch job: ```` ERROR: (gcloud.beta.dataproc.batches.submit.pyspark) Batch job is FAILED. Detail: Multiple Errors: - Failed to fetch cluster for batch - Permission 'dataproc.clusters.get' denied on resource '//dataproc.googleapis.com/projects/qwiklabs-gcp-00-c467d66f6efc/regions/us-east4/clusters/srvls-batch-1b1a8482-374f-4c44-83d7-6bc417531bed' (or it may not exist). ``` Fortunatelly I was able to figure it out through IAM permissions configuration, but the lab does not provide that guidance. 2) It can be out of scope for a lab, but I think somehow it lacks the explanation for what use cases this solution would be preferred over other ones.

Caio L. · Reviewed حوالي سنة ago

na

Lipsita N. · Reviewed حوالي سنة ago

julian c. · Reviewed حوالي سنة ago