Dataflow によるサーバーレス データ処理 – Dataflow を使用した CI / CD のレビュー

[2023-03-05 15:07:22,501] {taskinstance.py:1152} ERROR - Google Cloud Dataflow job wordcount-run-f228c786 has failed. Traceback (most recent call last): File "/usr/local/lib/airflow/airflow/models/taskinstance.py", line 985, in _run_raw_task result = task_copy.execute(context=context) File "/usr/local/lib/airflow/airflow/contrib/operators/dataflow_operator.py", line 184, in execute hook.start_java_dataflow(self.job_name, dataflow_options, File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_dataflow_hook.py", line 228, in start_java_dataflow self._start_dataflow(variables, name, command_prefix, label_formatter) File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_api_base_hook.py", line 363, in wrapper return func(self, *args, **kwargs) File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_dataflow_hook.py", line 205, in _start_dataflow _DataflowJob(self.get_conn(), variables['project'], name, File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_dataflow_hook.py", line 96, in wait_for_done raise Exception("Google Cloud Dataflow job {} has failed.".format( Exception: Google Cloud Dataflow job wordcount-run-f228c786 has failed. [2023-03-05 15:07:22,505] {taskinstance.py:1189} INFO - Marking task as FAILED. dag_id=test_word_count, task_id=wordcount-run, execution_date=20230305T150602, start_date=20230305T150612, end_date=20230305T150722 [2023-03-05 15:07:27,541] {local_task_job.py:102} INFO - Task exited with return code 1

Ferreira Júnior João Borges · 2年以上前にレビュー済み

lab is broken

ALMOHAIMEED Abdullaziz · 2年以上前にレビュー済み

lab is broken

ALMOHAIMEED Abdullaziz · 2年以上前にレビュー済み

R Nair Dhanun · 2年以上前にレビュー済み

Ateeq Algethami Jameelah · 2年以上前にレビュー済み

Pitt Alexander · 2年以上前にレビュー済み

Carrasco Sánchez Sergio · 2年以上前にレビュー済み

outdated composer guide

Amal Ichsanul · 2年以上前にレビュー済み

outdated composer guide

Amal Ichsanul · 2年以上前にレビュー済み

ERROR ERROR: build step 9 "gcr.io/cloud-builders/gcloud" failed: step exited with non-zero status: 1 Broken DAG (dags/data-pipeline-test.py): Traceback (most recent call last): File "/opt/python3.8/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 408, in apply_defaults result = func(self, **kwargs, default_args=default_args) File "/opt/python3.8/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 756, in __init__ raise AirflowException( airflow.exceptions.AirflowException: Invalid arguments were passed to PubSubPublishMessageOperator (task_id: publish_test_complete). Invalid arguments were: **kwargs: {'project': 'qwiklabs-gcp-01-8be47a6b153b'}

K Rajkumar · 2年以上前にレビュー済み

ERROR ERROR: build step 9 "gcr.io/cloud-builders/gcloud" failed: step exited with non-zero status: 1 Broken DAG (dags/data-pipeline-test.py): Traceback (most recent call last): File "/opt/python3.8/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 408, in apply_defaults result = func(self, **kwargs, default_args=default_args) File "/opt/python3.8/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 756, in __init__ raise AirflowException( airflow.exceptions.AirflowException: Invalid arguments were passed to PubSubPublishMessageOperator (task_id: publish_test_complete). Invalid arguments were: **kwargs: {'project': 'qwiklabs-gcp-01-8be47a6b153b'}

K Rajkumar · 2年以上前にレビュー済み

Torres Angel · 2年以上前にレビュー済み

Torres Angel · 2年以上前にレビュー済み

Gonzalez Juan Pablo · 2年以上前にレビュー済み

Azcona Pedro · 2年以上前にレビュー済み

Barman Sampriti · 2年以上前にレビュー済み

alshagri aljuhara · 2年以上前にレビュー済み

Citra Muhammad Archie · 2年以上前にレビュー済み

Merugu Revanth · 2年以上前にレビュー済み

alshagri aljuhara · 2年以上前にレビュー済み

Alahmadi Mona · 2年以上前にレビュー済み

Ateeq Algethami Jameelah · 2年以上前にレビュー済み

MANEA S ALYAMI HAIFAA · 2年以上前にレビュー済み

Ahmed Ola · 2年以上前にレビュー済み

Quibao Ricardo · 2年以上前にレビュー済み