[DEPRECATED] Serverless Data Processing with Dataflow - Using Dataflow SQL for Batch Analytics (Python) Reviews

dfv

Samvar V. · Reviewed over 2 years ago

nhjm

Tarun P. · Reviewed over 2 years ago

Vlad R. · Reviewed over 2 years ago

Lucio A. · Reviewed over 2 years ago

I can't check my prograss becuse this bug : Traceback (most recent call last): File "batch_minute_traffic_SQL_pipeline.py", line 123, in <module> run() File "batch_minute_traffic_SQL_pipeline.py", line 120, in run p.run() File "/opt/conda/lib/python3.7/site-packages/apache_beam/pipeline.py", line 577, in run return self.runner.run_pipeline(self, self._options) File "/opt/conda/lib/python3.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", line 451, in run_pipeline artifacts = environments.python_sdk_dependencies(options) File "/opt/conda/lib/python3.7/site-packages/apache_beam/transforms/environments.py", line 812, in python_sdk_dependencies skip_prestaged_dependencies=skip_prestaged_dependencies) File "/opt/conda/lib/python3.7/site-packages/apache_beam/runners/portability/stager.py", line 315, in create_job_resources Stager._create_beam_sdk(sdk_remote_location, temp_dir)) File "/opt/conda/lib/python3.7/site-packages/apache_beam/runners/portability/stager.py", line 840, in _create_beam_sdk sdk_local_file = Stager._download_pypi_sdk_package(temp_dir) File "/opt/conda/lib/python3.7/site-packages/apache_beam/runners/portability/stager.py", line 960, in _download_pypi_sdk_package processes.check_output(cmd_args) File "/opt/conda/lib/python3.7/site-packages/apache_beam/utils/processes.py", line 97, in check_output .format(traceback.format_exc(), args[0][6], error.output)) RuntimeError: Full traceback: Traceback (most recent call last): File "/opt/conda/lib/python3.7/site-packages/apache_beam/utils/processes.py", line 89, in check_output out = subprocess.check_output(*args, **kwargs) File "/opt/conda/lib/python3.7/subprocess.py", line 411, in check_output **kwargs).stdout File "/opt/conda/lib/python3.7/subprocess.py", line 512, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['/opt/conda/bin/python3', '-m', 'pip', 'download', '--dest', '/tmp/tmpjcwa9z8k', 'apache-beam==2.48.0', '--no-deps', '--no-binary', ':all:']' returned non-zero exit status 1. Pip install failed for package: apache-beam==2.48.0 Output from execution of subprocess: b"Collecting apache-beam==2.48.0\n Using cached apache-beam-2.48.0.zip (3.1 MB)\n Preparing metadata (setup.py): started\n Preparing metadata (setup.py): finished with status 'error'\n"

Mashael A. · Reviewed over 2 years ago

Payal M. · Reviewed over 2 years ago

Pooja K. · Reviewed over 2 years ago

Shrilekha P. · Reviewed over 2 years ago

excelent working example, very real work scenario

Fernando G. · Reviewed over 2 years ago

Pavan M. · Reviewed over 2 years ago

Haoxuan Z. · Reviewed over 2 years ago

Antoni C. · Reviewed over 2 years ago

Fernando Yudi N. · Reviewed over 2 years ago

Fernando Yudi N. · Reviewed over 2 years ago

Artem B. · Reviewed over 2 years ago

Shubham S. · Reviewed over 2 years ago

Piotr K. · Reviewed over 2 years ago

Piotr K. · Reviewed over 2 years ago

Anurag P. · Reviewed over 2 years ago

Andrei C. · Reviewed over 2 years ago

Guilherme V. · Reviewed over 2 years ago

Anurag P. · Reviewed over 2 years ago

Tuan U. · Reviewed over 2 years ago

bhargav c. · Reviewed over 2 years ago

Oleksii C. · Reviewed over 2 years ago