Use Serverless for Apache Spark to Load BigQuery Reviews

Juan C. · Reviewed מעל שנה אחת ago

The below instruction could have been in little more detail. As I found dataproc identification tool not visible. Next, you're going to set a few environment variables into VM instance terminal and execute a Spark template to load data into BigQuery. Set the following environment variables for the Dataproc Serverless environment.

Sameer S. · Reviewed מעל שנה אחת ago

sandeep s. · Reviewed מעל שנה אחת ago

Jason M. · Reviewed מעל שנה אחת ago

error when submitting spark job ERROR: (gcloud.beta.dataproc.batches.submit.pyspark) Batch job is FAILED. Detail: Insufficient 'CPUS' quota. Requested 12.0, available 11.0. Your resource request exceeds your available quota. See https://cloud.google.com/compute/resource-usage. Use https://cloud.google.com/docs/quotas/view-manage#requesting_higher_quota to request additional quota. Running auto diagnostics on the batch. It may take few minutes before diagnostics output is available. Please check diagnostics output by running 'gcloud dataproc batches describe' command.

max s. · Reviewed מעל שנה אחת ago

trevor a. · Reviewed מעל שנה אחת ago

Murilo S. · Reviewed מעל שנה אחת ago

Sergiu D. · Reviewed מעל שנה אחת ago

Thanks Google

Wisdom K. · Reviewed מעל שנה אחת ago

Task 3.2 fails as the spark job needs additional IAM permissions to run the script. After adding the required permission the script passed without any issues. Lab needs to be updated with this additional step.

Vamshi K. · Reviewed מעל שנה אחת ago

Kirit C. · Reviewed מעל שנה אחת ago

there are some errors - just re-try after 2-3 mins

Tomasz M. · Reviewed מעל שנה אחת ago

Maitreyee R. · Reviewed מעל שנה אחת ago

Dawid G. · Reviewed מעל שנה אחת ago

Dmitry P. · Reviewed מעל שנה אחת ago

Task 3 check didn't work, but I was able to see the data in Task 4. Seems like something wrong with the check in Task 3.

Nick F. · Reviewed מעל שנה אחת ago

rida l. · Reviewed מעל שנה אחת ago

Priyambada S. · Reviewed מעל שנה אחת ago

Creative P. · Reviewed מעל שנה אחת ago

abdelhay h. · Reviewed מעל שנה אחת ago

Ujjal P. · Reviewed מעל שנה אחת ago

Raj V. · Reviewed מעל שנה אחת ago

Babita K. · Reviewed מעל שנה אחת ago

MUY LENTO EL PASO DE SPARK

FABRICIO JOCABED V. · Reviewed מעל שנה אחת ago

Hayato S. · Reviewed מעל שנה אחת ago