Dataflow を使用したサーバーレスのデータ処理 - Apache Beam と Dataflow を使用して ETL パイプラインを作成する（Python） のレビュー

Rachewad Akash · 3年以上前にレビュー済み

Patil Abhishek · 3年以上前にレビュー済み

Runtime error using the provided solution, wasnt able to advance the lab (base) jupyter@theia-20221017-211349:~/project/training-data-analyst/quests/dataflow_python/1_Basic_ETL/lab$ python3 my_pipeline.py --project=${PROJECT_ID} --region=us-central1 --stagingLocation=gs://$PROJECT_ID/staging/ --tempLocation=gs://$PROJECT_ID/temp/ --runner=DataflowRunner /opt/conda/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py:2285: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported is_streaming_pipeline = p.options.view_as(StandardOptions).streaming /opt/conda/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_file_loads.py:1129: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported temp_location = p.options.view_as(GoogleCloudOptions).temp_location INFO:root:Building pipeline ... INFO:apache_beam.runners.portability.stager:Downloading source distribution of the SDK from PyPi INFO:apache_beam.runners.portability.stager:Executing command: ['/opt/conda/bin/python3', '-m', 'pip', 'download', '--dest', '/tmp/tmpw1vdo9yu', 'apache-beam==2.42.0', '--no-deps', '--no-binary', ':all:'] ERROR: Ignored the following versions that require a different python version: 2.10.0 Requires-Python >=2.7,<3.0; 2.3.0 Requires-Python >=2.7,<3.0; 2.4.0 Requires-Python >=2.7,<3.0; 2.5.0 Requires-Python >=2.7,<3.0; 2.6.0 Requires-Python >=2.7,<3.0; 2.7.0 Requires-Python >=2.7,<3.0; 2.8.0 Requires-Python >=2.7,<3.0; 2.9.0 Requires-Python >=2.7,<3.0 ERROR: Could not find a version that satisfies the requirement apache-beam==2.42.0 (from versions: 0.6.0, 2.0.0, 2.1.0, 2.1.1, 2.2.0, 2.11.0, 2.12.0, 2.13.0, 2.14.0, 2.15.0, 2.16.0, 2.17.0, 2.18.0, 2.19.0, 2.20.0, 2.21.0, 2.22.0, 2.23.0, 2.24.0, 2.25.0, 2.26.0, 2.27.0, 2.28.0, 2.29.0rc1, 2.29.0, 2.30.0rc1, 2.30.0, 2.31.0rc1, 2.31.0, 2.32.0rc1, 2.32.0, 2.33.0rc1, 2.33.0rc2, 2.33.0, 2.34.0rc1, 2.34.0rc2, 2.34.0, 2.35.0rc2, 2.35.0rc3, 2.35.0rc5, 2.35.0rc6, 2.35.0rc8, 2.35.0, 2.36.0rc1, 2.36.0rc2, 2.36.0rc3, 2.36.0, 2.37.0rc1, 2.37.0rc2, 2.37.0rc3, 2.37.0, 2.38.0rc1, 2.38.0, 2.39.0rc1, 2.39.0rc2, 2.39.0, 2.40.0rc1, 2.40.0rc2, 2.40.0, 2.41.0rc1, 2.41.0rc2, 2.41.0, 2.42.0rc0, 2.42.0rc1, 2.42.0rc2) ERROR: No matching distribution found for apache-beam==2.42.0 Traceback (most recent call last): File "/opt/conda/lib/python3.7/site-packages/apache_beam/utils/processes.py", line 89, in check_output out = subprocess.check_output(*args, **kwargs) File "/opt/conda/lib/python3.7/subprocess.py", line 411, in check_output **kwargs).stdout File "/opt/conda/lib/python3.7/subprocess.py", line 512, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['/opt/conda/bin/python3', '-m', 'pip', 'download', '--dest', '/tmp/tmpw1vdo9yu', 'apache-beam==2.42.0', '--no-deps', '--no-binary', ':all:']' returned non-zero exit status 1. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "my_pipeline.py", line 106, in <module> run() File "my_pipeline.py", line 103, in run p.run() File "/opt/conda/lib/python3.7/site-packages/apache_beam/pipeline.py", line 550, in run self._options).run(False) File "/opt/conda/lib/python3.7/site-packages/apache_beam/pipeline.py", line 574, in run return self.runner.run_pipeline(self, self._options) File "/opt/conda/lib/python3.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", line 475, in run_pipeline artifacts = environments.python_sdk_dependencies(options) File "/opt/conda/lib/python3.7/site-packages/apache_beam/transforms/environments.py", line 806, in python_sdk_dependencies skip_prestaged_dependencies=skip_prestaged_dependencies) File "/opt/conda/lib/python3.7/site-packages/apache_beam/runners/portability/stager.py", line 310, in create_job_resources Stager._create_beam_sdk(sdk_remote_location, temp_dir)) File "/opt/conda/lib/python3.7/site-packages/apache_beam/runners/portability/stager.py", line 822, in _create_beam_sdk sdk_local_file = Stager._download_pypi_sdk_package(temp_dir) File "/opt/conda/lib/python3.7/site-packages/apache_beam/runners/portability/stager.py", line 936, in _download_pypi_sdk_package processes.check_output(cmd_args) File "/opt/conda/lib/python3.7/site-packages/apache_beam/utils/processes.py", line 97, in check_output .format(traceback.format_exc(), args[0][6], error.output)) RuntimeError: Full traceback: Traceback (most recent call last): File "/opt/conda/lib/python3.7/site-packages/apache_beam/utils/processes.py", line 89, in check_output out = subprocess.check_output(*args, **kwargs) File "/opt/conda/lib/python3.7/subprocess.py", line 411, in check_output **kwargs).stdout File "/opt/conda/lib/python3.7/subprocess.py", line 512, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['/opt/conda/bin/python3', '-m', 'pip', 'download', '--dest', '/tmp/tmpw1vdo9yu', 'apache-beam==2.42.0', '--no-deps', '--no-binary', ':all:']' returned non-zero exit status 1. Pip install failed for package: apache-beam==2.42.0 Output from execution of subprocess: b''

Martini Juan · 3年以上前にレビュー済み

D'Elia John · 3年以上前にレビュー済み

Quibao Ricardo · 3年以上前にレビュー済み

Tan Eric · 3年以上前にレビュー済み

Takahama Marcos · 3年以上前にレビュー済み

Carlock Mark · 3年以上前にレビュー済み

Brisan Andrei · 3年以上前にレビュー済み

Cauzo Antonio · 3年以上前にレビュー済み

Vaitinadin Couganadane · 3年以上前にレビュー済み

SandeepGuptha Gadamsetty · 3年以上前にレビュー済み

Maurer Nikolaus · 3年以上前にレビュー済み

Constructive lab

Mourtada Ali · 3年以上前にレビュー済み

Kayal Kharrat Tamer · 3年以上前にレビュー済み

Sharma Prajjwal · 3年以上前にレビュー済み

Until now, this is the best explain lab I ever did in cloudskillboots. Have time tags before tasks, have at least double of required time if you only follow step by step the lab. Give you the opportunity to try yourself an the solution code is an extra non a step...

Novas Villa Ricardo Miguel · 3年以上前にレビュー済み

Kumar Prashant · 3年以上前にレビュー済み

monassa jairo · 3年以上前にレビュー済み

bohra khadija · 3年以上前にレビュー済み

Sharma Prajjwal · 3年以上前にレビュー済み

Gracida Hernández Noé · 3年以上前にレビュー済み

Bharatkumar Pardhi pratik · 3年以上前にレビュー済み

Chen Deshen · 3年以上前にレビュー済み

Karpenok Kyrylo · 3年以上前にレビュー済み