리뷰 Dataflow를 사용한 서버리스 데이터 처리 - Apache Beam 및 Dataflow를 사용한 ETL 파이프라인 작성(Python)개
리뷰 11475개
My conclusion from this lab is that I should not use or depend on this type of platform for production loads. It if does not even work in a controlled lab, then production is a no-go! ERROR:apache_beam.runners.dataflow.dataflow_runner:2026-04-01T09:25:07.346Z: JOB_MESSAGE_ERROR: Startup of the worker pool in us-central1 failed to bring up any of the desired 1 workers. This is likely a quota issue or a Compute Engine stockout. The service will retry. For troubleshooting steps, see https://cloud.google.com/dataflow/docs/guides/common-errors#worker-pool-failure for help troubleshooting. ZONE_RESOURCE_POOL_EXHAUSTED: Instance 'my-pipeline-1775035379230-04010223-mm24-harness-86c2' creation failed: The zone 'projects/qwiklabs-gcp-02-e142b19585b8/zones/us-central1-a' does not have enough resources available to fulfill the request. Try a different zone, or try again later.
Mikael W. · 13일 전에 리뷰됨
Luis Antonio C. · 13일 전에 리뷰됨
In the Python script, allow users to pass alternative machine types, so there is no possibility of being unable to complete the lab due to insufficient resources being available (for N1?)
Felix V. · 13일 전에 리뷰됨
1. There are 2 issues running the lab: - lab 1: apache_beam.io.filesystem.BeamIOError: Match operation failed with exceptions {'gs://qwiklabs-gcp-04-9d1f7241cd59/events.json': RefreshError(TransportError("Failed to retrieve https://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Engine metadata service. Compute Engine Metadata server unavailable. Last exception: HTTPSConnectionPool(host='metadata.google.internal', port=443): Max retries exceeded with url: /computeMetadata/v1/instance/service-accounts/default/?recursive=true (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate - lab 2 Startup of the worker pool in us-central1 failed to bring up any of the desired 1 workers P.S. Not working http.client transport only supports the http scheme, httpswas specified P.S.S. export GCE_METADATA_MTLS_MODE=none resolves an issue with certificate. But still insufficient resources to run the job.
Igor P. · 13일 전에 리뷰됨
1. There are 2 issues running the lab: - lab 1: apache_beam.io.filesystem.BeamIOError: Match operation failed with exceptions {'gs://qwiklabs-gcp-04-9d1f7241cd59/events.json': RefreshError(TransportError("Failed to retrieve https://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Engine metadata service. Compute Engine Metadata server unavailable. Last exception: HTTPSConnectionPool(host='metadata.google.internal', port=443): Max retries exceeded with url: /computeMetadata/v1/instance/service-accounts/default/?recursive=true (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate - lab 2 Startup of the worker pool in us-central1 failed to bring up any of the desired 1 workers P.S. Not working http.client transport only supports the http scheme, httpswas specified
Igor P. · 13일 전에 리뷰됨
Luis Antonio C. · 13일 전에 리뷰됨
Charles F. · 14일 전에 리뷰됨
Too much confuse the Python file and Instructions were not clear to proceed and modify the files. The code is running with errors. Need support.
Venkateswarlu Kuriseti N. · 14일 전에 리뷰됨
konda l. · 14일 전에 리뷰됨
Jorge Alberto M. · 16일 전에 리뷰됨
There is currently an issue with google.auth library (https://github.com/googleapis/google-cloud-python/issues/16090) that prevents me from finishing the lab - took a long time to find the root couse and I run out of time. Additionally, the worker zones and machine types have to be specified because without them the dataflow cannot spawn workers and jobs keep failing.
Przemyslaw S. · 17일 전에 리뷰됨
1. There are 2 issues running the lab: - lab 1: apache_beam.io.filesystem.BeamIOError: Match operation failed with exceptions {'gs://qwiklabs-gcp-04-9d1f7241cd59/events.json': RefreshError(TransportError("Failed to retrieve https://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Engine metadata service. Compute Engine Metadata server unavailable. Last exception: HTTPSConnectionPool(host='metadata.google.internal', port=443): Max retries exceeded with url: /computeMetadata/v1/instance/service-accounts/default/?recursive=true (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate - lab 2 Startup of the worker pool in us-central1 failed to bring up any of the desired 1 workers
Igor P. · 17일 전에 리뷰됨
Bartłomiej B. · 17일 전에 리뷰됨
Alice Wu 吳馨妤 E. · 19일 전에 리뷰됨
not enough recources to complete the lab
Stef v. · 19일 전에 리뷰됨
Sriyansh S. · 19일 전에 리뷰됨
Shaik S. · 20일 전에 리뷰됨
Too abstract
Anirban S. · 21일 전에 리뷰됨
EMIL-CRISTIAN C. · 21일 전에 리뷰됨
Ram krishna Reddy S. · 22일 전에 리뷰됨
Anatolii L. · 22일 전에 리뷰됨
Ram krishna Reddy S. · 23일 전에 리뷰됨
Ram krishna Reddy S. · 23일 전에 리뷰됨
Ram krishna Reddy S. · 23일 전에 리뷰됨
don't have enough resources to create pipeline
Anatolii L. · 24일 전에 리뷰됨
Google은 게시된 리뷰가 제품을 구매 또는 사용한 소비자에 의해 작성되었음을 보증하지 않습니다. 리뷰는 Google의 인증을 거치지 않습니다.