Serverless Data Processing with Dataflow - CI/CD with Dataflow Rezensionen

needs time to study it but interesting complete CICD

Andreas K. · Vor 6 Monate überprüft

Vishnu M. · Vor 6 Monate überprüft

Riccardo M. · Vor 6 Monate überprüft

too many errors making it impossible to complete in time. for instance, the image in the code does not exist. the DAG keeps failing, and more. I've now tried it two times, and it is impossible to complete.

Trym S. · Vor 6 Monate überprüft

NA

Nenavath R. · Vor 6 Monate überprüft

wordcount-run fails with error: [2024-12-15, 21:15:32 UTC] {beam.py:119} WARNING - WARNING: Request failed with code 400, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://dataflow.googleapis.com/v1b3/projects/qwiklabs-gcp-01-caa5705744db/locations/us-central1/jobs. [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - Exception in thread "main" java.lang.RuntimeException: Failed to create a workflow job: (3c8e5c18af78a2a): 'us-central1' violates constraint 'constraints/gcp.resourceLocations' on the resource 'projects/qwiklabs-gcp-01-caa5705744db'. [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:972) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:186) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.sdk.Pipeline.run(Pipeline.java:315) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.sdk.Pipeline.run(Pipeline.java:301) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.examples.WordCount.runWordCount(WordCount.java:187) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.examples.WordCount.main(WordCount.java:193) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - { [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - "code" : 400, [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - "errors" : [ { [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - "domain" : "global", [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - "message" : "(3c8e5c18af78a2a): 'us-central1' violates constraint 'constraints/gcp.resourceLocations' on the resource 'projects/qwiklabs-gcp-01-caa5705744db'.", [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - "reason" : "failedPrecondition" [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - } ], [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - "message" : "(3c8e5c18af78a2a): 'us-central1' violates constraint 'constraints/gcp.resourceLocations' on the resource 'projects/qwiklabs-gcp-01-caa5705744db'.", [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - "status" : "FAILED_PRECONDITION" [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - } [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:417) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1132) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:515) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:448) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.runners.dataflow.DataflowClient.createJob(DataflowClient.java:61) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:958)

Marcelo B. · Vor 6 Monate überprüft

Riccardo M. · Vor 6 Monate überprüft

Sana K. · Vor 6 Monate überprüft

NA

Nenavath R. · Vor 6 Monate überprüft

Steven B. · Vor 6 Monate überprüft

This had 2 blockers. The 2nd killed my 1st attempt. 1. Airflow version non-existent anymore, had to pick one from https://cloud.google.com/composer/docs/concepts/versioning/composer-versions instead. 2. (stage: watching Airflow DAGs) The default regions setup blows up during the first task. Says constraints/gcp.resourceLocations is violated, because f--k us-central1. The student can't adjust those permissions because they have no editor rights.

Anna E. · Vor 6 Monate überprüft

Kamala Krishnan S. · Vor 6 Monate überprüft

I've tried more than a dozen times. It always fails on the DAG run.

Flavio B. · Vor 6 Monate überprüft

Luke M. · Vor 6 Monate überprüft

Monica C. · Vor 6 Monate überprüft

Simone G. · Vor 6 Monate überprüft

Cecilia Belen B. · Vor 6 Monate überprüft

Simone G. · Vor 6 Monate überprüft

Lots of commands not working on this lab! test_work_count DAG keeps failing. I'm assuming because of incorrect region which should be asia-east1 instead of us-central1. I made sure I run all the commands to set up the region and zone variables to asia-east1 but not sure why region is incorrect in the jar file. Running command: java -jar /tmp/tmpjnvtv59_dataflow_deployment_106c05a3-bb57-41d1-b2ee-386581a6109b.jar --runner=DataflowRunner --project=qwiklabs-gcp-00-436813f80158 --zone=asia-east1-a --region=us-central1 --stagingLocation=gs://qwiklabs-gcp-00-436813f80158-dataflow-staging-test/staging --jobName=wordcount-run-081da930 --autoscalingAlgorithm=THROUGHPUT_BASED --maxNumWorkers=3 --inputFile=gs://qwiklabs-gcp-00-436813f80158-composer-input-test/input.txt --output=gs://qwiklabs-gcp-00-436813f80158-composer-result-test/output '--labels={"airflow-version":"v2-10-2-composer"}'

Oscar F. · Vor 6 Monate überprüft

Puneet P. · Vor 6 Monate überprüft

end

Karthik C. · Vor 6 Monate überprüft

Kevin A. · Vor 6 Monate überprüft

Ashwani Kumar S. · Vor 6 Monate überprüft

Jesus C. · Vor 6 Monate überprüft

Karthik C. · Vor 6 Monate überprüft