Opiniones sobre Procesamiento de datos sin servidores con Dataflow: CI/CD con Dataflow

Vishnu M. · Se revisó hace alrededor de 1 año

Riccardo M. · Se revisó hace alrededor de 1 año

too many errors making it impossible to complete in time. for instance, the image in the code does not exist. the DAG keeps failing, and more. I've now tried it two times, and it is impossible to complete.

Trym S. · Se revisó hace alrededor de 1 año

NA

Nenavath R. · Se revisó hace alrededor de 1 año

wordcount-run fails with error: [2024-12-15, 21:15:32 UTC] {beam.py:119} WARNING - WARNING: Request failed with code 400, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://dataflow.googleapis.com/v1b3/projects/qwiklabs-gcp-01-caa5705744db/locations/us-central1/jobs. [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - Exception in thread "main" java.lang.RuntimeException: Failed to create a workflow job: (3c8e5c18af78a2a): 'us-central1' violates constraint 'constraints/gcp.resourceLocations' on the resource 'projects/qwiklabs-gcp-01-caa5705744db'. [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:972) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:186) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.sdk.Pipeline.run(Pipeline.java:315) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.sdk.Pipeline.run(Pipeline.java:301) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.examples.WordCount.runWordCount(WordCount.java:187) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.examples.WordCount.main(WordCount.java:193) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - { [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - "code" : 400, [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - "errors" : [ { [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - "domain" : "global", [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - "message" : "(3c8e5c18af78a2a): 'us-central1' violates constraint 'constraints/gcp.resourceLocations' on the resource 'projects/qwiklabs-gcp-01-caa5705744db'.", [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - "reason" : "failedPrecondition" [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - } ], [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - "message" : "(3c8e5c18af78a2a): 'us-central1' violates constraint 'constraints/gcp.resourceLocations' on the resource 'projects/qwiklabs-gcp-01-caa5705744db'.", [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - "status" : "FAILED_PRECONDITION" [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - } [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:417) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1132) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:515) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:448) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.runners.dataflow.DataflowClient.createJob(DataflowClient.java:61) [2024-12-15, 21:15:33 UTC] {beam.py:119} WARNING - at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:958)

Marcelo B. · Se revisó hace alrededor de 1 año

Riccardo M. · Se revisó hace alrededor de 1 año

Sana K. · Se revisó hace alrededor de 1 año

NA

Nenavath R. · Se revisó hace alrededor de 1 año

Steven B. · Se revisó hace alrededor de 1 año

This had 2 blockers. The 2nd killed my 1st attempt. 1. Airflow version non-existent anymore, had to pick one from https://cloud.google.com/composer/docs/concepts/versioning/composer-versions instead. 2. (stage: watching Airflow DAGs) The default regions setup blows up during the first task. Says constraints/gcp.resourceLocations is violated, because f--k us-central1. The student can't adjust those permissions because they have no editor rights.

Anna E. · Se revisó hace alrededor de 1 año

Kamala Krishnan S. · Se revisó hace alrededor de 1 año

I've tried more than a dozen times. It always fails on the DAG run.

Flavio B. · Se revisó hace alrededor de 1 año

Luke M. · Se revisó hace alrededor de 1 año

Monica C. · Se revisó hace alrededor de 1 año

Simone G. · Se revisó hace alrededor de 1 año

Cecilia Belen B. · Se revisó hace alrededor de 1 año

Simone G. · Se revisó hace alrededor de 1 año

Lots of commands not working on this lab! test_work_count DAG keeps failing. I'm assuming because of incorrect region which should be asia-east1 instead of us-central1. I made sure I run all the commands to set up the region and zone variables to asia-east1 but not sure why region is incorrect in the jar file. Running command: java -jar /tmp/tmpjnvtv59_dataflow_deployment_106c05a3-bb57-41d1-b2ee-386581a6109b.jar --runner=DataflowRunner --project=qwiklabs-gcp-00-436813f80158 --zone=asia-east1-a --region=us-central1 --stagingLocation=gs://qwiklabs-gcp-00-436813f80158-dataflow-staging-test/staging --jobName=wordcount-run-081da930 --autoscalingAlgorithm=THROUGHPUT_BASED --maxNumWorkers=3 --inputFile=gs://qwiklabs-gcp-00-436813f80158-composer-input-test/input.txt --output=gs://qwiklabs-gcp-00-436813f80158-composer-result-test/output '--labels={"airflow-version":"v2-10-2-composer"}'

Oscar F. · Se revisó hace alrededor de 1 año

Puneet P. · Se revisó hace alrededor de 1 año

end

Karthik C. · Se revisó hace alrededor de 1 año

Kevin A. · Se revisó hace alrededor de 1 año

Ashwani Kumar S. · Se revisó hace alrededor de 1 año

Jesus C. · Se revisó hace alrededor de 1 año

Karthik C. · Se revisó hace alrededor de 1 año

Alok P. · Se revisó hace alrededor de 1 año