Distributed Image Processing in Managed Apache Spark Reviews

fatima a. · Reviewed over 5 years ago

Bashar A. · Reviewed over 5 years ago

Khadar Basha S. · Reviewed over 5 years ago

Hieu N. · Reviewed over 5 years ago

GSP010 - Distributed Image Processing in Cloud Dataproc student-02-ee751f7e7ede@devhost:~/cloud-dataproc/codelabs/opencv-haarcascade$ gcloud dataproc jobs submit spark \ > --cluster ${MYCLUSTER} \ > --jar target/scala-2.10/feature_detector-assembly-1.0.jar -- \ > gs://${MYBUCKET}/haarcascade_frontalface_default.xml \ > gs://${MYBUCKET}/imgs/ \ > gs://${MYBUCKET}/out/ Job [19c4459f1c8545e2b234b47718dbbc97] submitted. Waiting for job output... 20/09/07 03:49:51 INFO org.spark_project.jetty.util.log: Logging initialized @2667ms 20/09/07 03:49:51 INFO org.spark_project.jetty.server.Server: jetty-9.3.z-SNAPSHOT, build timestamp: unknown, git hash: unknown 20/09/07 03:49:51 INFO org.spark_project.jetty.server.Server: Started @2744ms 20/09/07 03:49:51 INFO org.spark_project.jetty.server.AbstractConnector: Started ServerConnector@2e1b7a65{HTTP/1.1,[http/1.1]}{0.0.0.0:4040} 20/09/07 03:49:52 WARN org.apache.spark.scheduler.FairSchedulableBuilder: Fair Scheduler configuration file not found so jobs will be scheduled in FIFO order. To use fair scheduling, configure pools in fairscheduler.xml or set spark.scheduler.allocation.file to a file that contains the configuration. 20/09/07 03:49:53 INFO org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at student-02-ee751f7e7ede-qwiklab-m/10.128.0.3:8032 20/09/07 03:49:54 INFO org.apache.hadoop.yarn.client.AHSProxy: Connecting to Application History server at student-02-ee751f7e7ede-qwiklab-m/10.128.0.3:10200 20/09/07 03:49:57 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl: Submitted application application_1599450471776_0001 Number of files found: 3 20/09/07 03:50:12 WARN org.apache.spark.scheduler.TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, student-02-ee751f7e7ede-qwiklab-w-1.c.qwiklabs-gcp-02-1176fbf2a111.internal, executor 2): java.lang.UnsatisfiedLinkError: no jniopencv_videoio in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1860)

Rajeshwar Rao K. · Reviewed over 5 years ago

真央田. · Reviewed over 5 years ago

The job monitoring in Step 3 of Distributed Image Processing in Cloud Dataproc kept failing

Michael W. · Reviewed over 5 years ago

Rajeshwar Rao K. · Reviewed over 5 years ago

This lab needs to be checked. I am getting an error in the last step.

Subhayu G. · Reviewed over 5 years ago

> gs://${MYBUCKET}/out/ERROR: (gcloud.dataproc.jobs.submit.spark) Could not upload file: [student-00-ab65f248b508-image-22519] bucket does not exist. student-00-ab65f248b508@devhost:~/cloud-dataproc/codelabs/opencv-haarcascade$ gsutil ls -R gs://${MYBUCKET} gs://student-00-ab65f248b508-image-617/imgs/: gs://student-00-ab65f248b508-image-617/imgs/african-woman.jpg gs://student-00-ab65f248b508-image-617/imgs/classroom.jpg gs://student-00-ab65f248b508-image-617/imgs/family-of-three.jpg

Rajeshwar Rao K. · Reviewed over 5 years ago

> gs://${MYBUCKET}/out/ERROR: (gcloud.dataproc.jobs.submit.spark) Could not upload file: [student-00-ab65f248b508-image-22519] bucket does not exist. student-00-ab65f248b508@devhost:~/cloud-dataproc/codelabs/opencv-haarcascade$ gsutil ls -R gs://${MYBUCKET} gs://student-00-ab65f248b508-image-617/imgs/: gs://student-00-ab65f248b508-image-617/imgs/african-woman.jpg gs://student-00-ab65f248b508-image-617/imgs/classroom.jpg gs://student-00-ab65f248b508-image-617/imgs/family-of-three.jpg

Rajeshwar Rao K. · Reviewed over 5 years ago

The job monitoring in Step 3 of Distributed Image Processing in Cloud Dataproc kept failing

Michael W. · Reviewed over 5 years ago

Subhayu G. · Reviewed over 5 years ago

Hieu N. · Reviewed over 5 years ago

Yasser A. · Reviewed over 5 years ago

The job monitoring in Step 3 of Distributed Image Processing in Cloud Dataproc kept failing

Michael W. · Reviewed over 5 years ago

Subhayu G. · Reviewed over 5 years ago

error

James Q. · Reviewed over 5 years ago

Rajeshwar Rao K. · Reviewed over 5 years ago

Anh L. · Reviewed over 5 years ago