An Introduction to Cloud Composer 3 Reviews

Alan H. · Reviewed 1 минута ago

default-hostname *** Reading remote logs from Cloud Logging. [2024-09-12, 06:04:41 UTC] {local_task_job_runner.py:120} ▶ Pre task execution logs [2024-09-12, 06:04:41 UTC] {dataproc.py:784} INFO - Creating cluster: composer-hadoop-tutorial-cluster-20240911 [2024-09-12, 06:04:41 UTC] {connection.py:274} WARNING - Connection schemes (type: google_cloud_platform) shall not contain '_' according to RFC3986. [2024-09-12, 06:04:41 UTC] {base.py:84} INFO - Using connection ID 'google_cloud_default' for task execution. [2024-09-12, 06:04:41 UTC] {credentials_provider.py:402} INFO - Getting connection using `google.auth.default()` since no explicit credentials are provided. [2024-09-12, 06:08:47 UTC] {taskinstance.py:441} ▼ Post task execution logs [2024-09-12, 06:08:47 UTC] {taskinstance.py:2907} ERROR - Task failed with exception\nTraceback (most recent call last):\n File "/opt/python3.11/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 76, in error_remapped_callable\n return callable_(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/opt/python3.11/lib/python3.11/site-packages/grpc/_channel.py", line 1181, in __call__\n return _end_unary_response_blocking(state, call, False, None)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/opt/python3.11/lib/python3.11/site-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking\n raise _InactiveRpcError(state) # pytype: disable=not-instantiable\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\ngrpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:\n status = StatusCode.UNAVAILABLE\n details = "failed to connect to all addresses; last error: UNAVAILABLE: ipv4:199.36.153.9:443: Socket closed"\n debug_error_string = "UNKNOWN:Error received from peer {created_time:"2024-09-12T06:08:47.042203565+00:00", grpc_status:14, grpc_message:"failed to connect to all addresses; last error: UNAVAILABLE: ipv4:199.36.153.9:443: Socket closed"}"\n>\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/opt/python3.11/lib/python3.11/site-packages/google/api_core/retry/retry_unary.py", line 144, in retry_target\n result = target()\n ^^^^^^^^\n File "/opt/python3.11/lib/python3.11/site-packages/google/api_core/timeout.py", line 120, in func_with_timeout\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/opt/python3.11/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable\n raise exceptions.from_grpc_error(exc) from exc\ngoogle.api_core.exceptions.ServiceUnavailable: 503 failed to connect to all addresses; last error: UNAVAILABLE: ipv4:199.36.153.9:443: Socket closed\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/opt/python3.11/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 465, in _execute_task\n result = _execute_callable(context=context, **execute_callable_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/opt/python3.11/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 432, in _execute_callable\n return execute_callable(context=context, **execute_callable_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/opt/python3.11/lib/python3.11/site-packages/airflow/models/baseoperator.py", line 400, in wrapper\n return func(self, *args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/google/cloud/operators/dataproc.py", line 800, in execute\n operation = self._create_cluster(hook)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/google/cloud/operators/dataproc.py", line 700, in _create_cluster\n return hook.create_cluster(\n ^^^^^^^^^^^^^^^^^^^^\n File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/google/common/hooks/base_google.py", line 559, in inner_wrapper\n return func(self, *args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/google/cloud/hooks/dataproc.py", line 343, in create_cluster\n result = client.create_cluster(\n ^^^^^^^^^^^^^^^^^^^^^^\n File "/opt/python3.11/lib/python3.11/site-packages/google/cloud/dataproc_v1/services/cluster_controller/client.py", line 872, in create_cluster\n response = rpc(\n ^^^^\n File "/opt/python3.11/lib/python3.11/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__\n return wrapped_func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/opt/python3.11/lib/python3.11/site-packages/google/api_ [2024-09-12, 06:08:47 UTC] {taskinstance.py:1206} INFO - Marking task as UP_FOR_RETRY. dag_id=composer_hadoop_tutorial, task_id=create_dataproc_cluster, run_id=scheduled__2024-09-11T00:00:00+00:00, execution_date=20240911T000000, start_date=20240912T060441, end_date=20240912T060847 [2024-09-12, 06:08:47 UTC] {standard_task_runner.py:110} ERROR - Failed to execute job 6 for task create_dataproc_cluster (Timeout of 300.0s exceeded, last exception: 503 failed to connect to all addresses; last error: UNAVAILABLE: ipv4:199.36.153.9:443: Socket closed; 5755) [2024-09-12, 06:08:47 UTC] {local_task_job_runner.py:240} INFO - Task exited with return code 1 [2024-09-12, 06:08:47 UTC] {warnings.py:110} WARNING - /opt/python3.11/lib/python3.11/site-packages/airflow/models/baseoperator.py:1308: AirflowProviderDeprecationWarning: Call to deprecated class DataprocSubmitHadoopJobOperator. (Please use `DataprocSubmitJobOperator` instead. You can use `generate_job` method to generate dictionary representing your job and use it with the new operator.)\n result = cls.__new__(cls)\n [2024-09-12, 06:08:47 UTC] {taskinstance.py:3501} INFO - 0 downstream tasks scheduled from follow-on schedule check [2024-09-12, 06:08:47 UTC] {local_task_job_runner.py:222} ▲▲▲ Log group end

Soyoung L. · Reviewed около 6 часов ago

Sukumar S. · Reviewed около 8 часов ago

Bryan Q. · Reviewed около 9 часов ago

Oscar Leonardo C. · Reviewed около 10 часов ago

Alexandre C. · Reviewed около 11 часов ago

Steven L. · Reviewed около 11 часов ago

Adam S. · Reviewed около 11 часов ago

Rahul P. · Reviewed около 12 часов ago

Prudhvi Varma P. · Reviewed около 12 часов ago

Angela B. · Reviewed около 12 часов ago

Vikram K. · Reviewed около 13 часов ago

André C. · Reviewed около 20 часов ago

Shiva Krishna M. · Reviewed около 21 часов ago

Diwakar C. · Reviewed около 23 часов ago

Nishit M. · Reviewed 1 день ago

Juan Pablo S. · Reviewed 1 день ago

Vadzim G. · Reviewed 1 день ago

Ahmad H. · Reviewed 1 день ago

Jarosław M. · Reviewed 2 дней ago

Hayden t. · Reviewed 2 дней ago

Kuldip K. · Reviewed 2 дней ago

Mustafa E. · Reviewed 2 дней ago

Maria C. · Reviewed 2 дней ago

Wiriyang (Pup) P. · Reviewed 2 дней ago