리뷰 Serverless Data Processing with Dataflow - CI/CD with Dataflow개

Sergio C. · 2년 초과 전에 리뷰됨

Multiple Issues with Airflow versioning, this lab is outdated

Alfonso G. · 2년 초과 전에 리뷰됨

Multiple Issues with Airflow versioning, this lab is outdated

Alfonso G. · 2년 초과 전에 리뷰됨

Ilkin I. · 2년 초과 전에 리뷰됨

Pedro A. · 2년 초과 전에 리뷰됨

Revanth M. · 2년 초과 전에 리뷰됨

Revanth M. · 2년 초과 전에 리뷰됨

정아 원. · 2년 초과 전에 리뷰됨

Multiple Issues with Airflow versioning, this lab is outdated

Alfonso G. · 2년 초과 전에 리뷰됨

[2023-03-05 15:07:22,501] {taskinstance.py:1152} ERROR - Google Cloud Dataflow job wordcount-run-f228c786 has failed. Traceback (most recent call last): File "/usr/local/lib/airflow/airflow/models/taskinstance.py", line 985, in _run_raw_task result = task_copy.execute(context=context) File "/usr/local/lib/airflow/airflow/contrib/operators/dataflow_operator.py", line 184, in execute hook.start_java_dataflow(self.job_name, dataflow_options, File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_dataflow_hook.py", line 228, in start_java_dataflow self._start_dataflow(variables, name, command_prefix, label_formatter) File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_api_base_hook.py", line 363, in wrapper return func(self, *args, **kwargs) File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_dataflow_hook.py", line 205, in _start_dataflow _DataflowJob(self.get_conn(), variables['project'], name, File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_dataflow_hook.py", line 96, in wait_for_done raise Exception("Google Cloud Dataflow job {} has failed.".format( Exception: Google Cloud Dataflow job wordcount-run-f228c786 has failed. [2023-03-05 15:07:22,505] {taskinstance.py:1189} INFO - Marking task as FAILED. dag_id=test_word_count, task_id=wordcount-run, execution_date=20230305T150602, start_date=20230305T150612, end_date=20230305T150722 [2023-03-05 15:07:27,541] {local_task_job.py:102} INFO - Task exited with return code 1

João Borges F. · 2년 초과 전에 리뷰됨

[2023-03-05 15:07:22,501] {taskinstance.py:1152} ERROR - Google Cloud Dataflow job wordcount-run-f228c786 has failed. Traceback (most recent call last): File "/usr/local/lib/airflow/airflow/models/taskinstance.py", line 985, in _run_raw_task result = task_copy.execute(context=context) File "/usr/local/lib/airflow/airflow/contrib/operators/dataflow_operator.py", line 184, in execute hook.start_java_dataflow(self.job_name, dataflow_options, File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_dataflow_hook.py", line 228, in start_java_dataflow self._start_dataflow(variables, name, command_prefix, label_formatter) File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_api_base_hook.py", line 363, in wrapper return func(self, *args, **kwargs) File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_dataflow_hook.py", line 205, in _start_dataflow _DataflowJob(self.get_conn(), variables['project'], name, File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_dataflow_hook.py", line 96, in wait_for_done raise Exception("Google Cloud Dataflow job {} has failed.".format( Exception: Google Cloud Dataflow job wordcount-run-f228c786 has failed. [2023-03-05 15:07:22,505] {taskinstance.py:1189} INFO - Marking task as FAILED. dag_id=test_word_count, task_id=wordcount-run, execution_date=20230305T150602, start_date=20230305T150612, end_date=20230305T150722 [2023-03-05 15:07:27,541] {local_task_job.py:102} INFO - Task exited with return code 1

João Borges F. · 2년 초과 전에 리뷰됨

Noura A. · 2년 초과 전에 리뷰됨

[2023-03-05 15:07:22,501] {taskinstance.py:1152} ERROR - Google Cloud Dataflow job wordcount-run-f228c786 has failed. Traceback (most recent call last): File "/usr/local/lib/airflow/airflow/models/taskinstance.py", line 985, in _run_raw_task result = task_copy.execute(context=context) File "/usr/local/lib/airflow/airflow/contrib/operators/dataflow_operator.py", line 184, in execute hook.start_java_dataflow(self.job_name, dataflow_options, File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_dataflow_hook.py", line 228, in start_java_dataflow self._start_dataflow(variables, name, command_prefix, label_formatter) File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_api_base_hook.py", line 363, in wrapper return func(self, *args, **kwargs) File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_dataflow_hook.py", line 205, in _start_dataflow _DataflowJob(self.get_conn(), variables['project'], name, File "/usr/local/lib/airflow/airflow/contrib/hooks/gcp_dataflow_hook.py", line 96, in wait_for_done raise Exception("Google Cloud Dataflow job {} has failed.".format( Exception: Google Cloud Dataflow job wordcount-run-f228c786 has failed. [2023-03-05 15:07:22,505] {taskinstance.py:1189} INFO - Marking task as FAILED. dag_id=test_word_count, task_id=wordcount-run, execution_date=20230305T150602, start_date=20230305T150612, end_date=20230305T150722 [2023-03-05 15:07:27,541] {local_task_job.py:102} INFO - Task exited with return code 1

João Borges F. · 2년 초과 전에 리뷰됨

Noura A. · 2년 초과 전에 리뷰됨

Angel T. · 2년 초과 전에 리뷰됨

Angel T. · 2년 초과 전에 리뷰됨

Ola A. · 2년 초과 전에 리뷰됨

Task 2: Not defining --image-version resolves to Cloud Composer current default version, which changed from 1 to 2 in a recent release and is subject to further changes in the future. Consider using --image-version=composer-A-airflow-X[.Y[.Z]]. More info at https://cloud.google.com/composer/docs/concepts/versioning/composer-versioning-overview#version-aliases ERROR: (gcloud.composer.environments.create) Cannot specify --python-version with Composer 2.X or greater. GCP get your sh## together please waste my time. and I paid premium sub

Ridhwan B. · 2년 초과 전에 리뷰됨

Mona A. · 2년 초과 전에 리뷰됨

Feng T. · 2년 초과 전에 리뷰됨

chaitanya g. · 2년 초과 전에 리뷰됨

Lakshmi Sahithi P. · 2년 초과 전에 리뷰됨

Task 2: Composer creation failed over Cloud Shell. The command is wrong. Workaround over UI and grant missing permissions on service-account. Fails again in Task 5. DAG gets invaild code... .

Robert J. · 2년 초과 전에 리뷰됨

Anusha P. · 2년 초과 전에 리뷰됨