在 Google Cloud 上使用 Dataflow 和 BigQuery 處理 ETL 作業 (Python) Reviews

could not find the python files

irfan a. · Reviewed over 6 years ago

Durgesh M. · Reviewed over 6 years ago

Diptaparna B. · Reviewed over 6 years ago

Durgesh M. · Reviewed over 6 years ago

TATSUYA K. · Reviewed over 6 years ago

instruction not clear and some path is wrong

Fajri R. · Reviewed over 6 years ago

instruction not clear and some path is wrong

Fajri R. · Reviewed over 6 years ago

TATSUYA K. · Reviewed over 6 years ago

There was an error in the command below "python dataflow_python_examples/data_ingestion.py --project=$PROJECT --runner=DataflowRunner --staging_location=gs://$PROJECT/test --temp_location gs://$PROJECT/test --input gs://$PROJECT/data_files/head_usa_names.csv --save_main_session"

KOJI I. · Reviewed over 6 years ago

Adarsh K. · Reviewed over 6 years ago

Chaitanya K. · Reviewed over 6 years ago

JIN OOK L. · Reviewed over 6 years ago

Ange O. · Reviewed over 6 years ago

Rodrigo M. · Reviewed over 6 years ago

Very poor quality... truly, access errors each time, authentication required even when I provided since the beginning, not valid or unupdated commands... how is it possible to learn or even certificate in your cloud "services"???? honestly

Edgar C. · Reviewed over 6 years ago

Poor quality... truly, access errors each time, authentication required even when I provided since the beginning, not valid or updated commands... how is it possible to learn or even certificate myself in your cloud "services"????

Edgar C. · Reviewed over 6 years ago

There's barely enough time to run all Dataflow pipelines

Ignacio V. · Reviewed over 6 years ago

Ignacio V. · Reviewed over 6 years ago

Bugged.....

Hans K. · Reviewed over 6 years ago

student-04-83e38a25f988@instance-1:~$ gsutil ls gs://python-dataflow-example/data_files/ AccessDeniedException: 403 There is an account problem for the requested project. lab is broken, I cannot get the source CSV files

Andrei D. · Reviewed over 6 years ago

Chawatphon R. · Reviewed over 6 years ago

Broken - missing region option in script. student_04_f5bd29a59316@cloudshell:~/professional-services/examples/dataflow-python-examples (qwiklabs-gcp-04-f1d783f10dd5)$ python dataflow_pytho n_examples/data_ingestion.py --project=$PROJECT --runner=DataflowRunner --staging_location=gs://$PROJECT/test --temp_location gs://$PROJECT/test - -input gs://$PROJECT/data_files/head_usa_names.csv --save_main_session ******************************************************************************** Python 2 is deprecated. Upgrade to Python 3 as soon as possible. See https://cloud.google.com/python/docs/python2-sunset To suppress this warning, create an empty ~/.cloudshell/no-python-warning file. The command will automatically proceed in seconds or on any key. ******************************************************************************** /home/student_04_f5bd29a59316/.local/lib/python2.7/site-packages/apache_beam/__init__.py:82: UserWarning: You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only. 'You are using Apache Beam with Python 2. ' Traceback (most recent call last): File "dataflow_python_examples/data_ingestion.py", line 134, in <module> run() File "dataflow_python_examples/data_ingestion.py", line 100, in run p = beam.Pipeline(options=PipelineOptions(pipeline_args)) File "/home/student_04_f5bd29a59316/.local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 198, in __init__ 'Pipeline has validations errors: \n' + '\n'.join(errors)) ValueError: Pipeline has validations errors: Missing required option: region.

Johannes V. · Reviewed over 6 years ago

Hari S. · Reviewed over 6 years ago

All in all the lab is great. I had a few issues with it: Minor issue: The Python commands to create the Dataflow pipelines are missing the --region flag. This was easy to detect from the error message and fix. Minor issue: Step 3 suggests it's necessary to edit like 80 to remove the usage of a decode function, but this function isn't used in data_transformation.py Issue: Looking at the results in BigQuery after steps 3 & 4, they didn't contain the joined data, which was a bit confusing, and leaves me wondering if the pipelines learned actually work as intended. I didn't check if this is reproducible. Major issue: I wasn't able to complete the lab in the given timeframe. I read the comments of the python file for the next step while the current Dataflow was running, but , but that wasn't quick enough. The last Dataflow pipeline was nearly completed as the lab timer ran out, so I wasn't able to get the final score & the completion of the lab. I had to restart, so I went through the lab as fast as I could, starting the next Dataflow pipeline as soon as the previous one ended, foregoing looking at BigQuery, and refreshing the browser after every step. This took 25 minutes to complete. I suggest adding an extra 15 minutes would make this a lot better.

Nir L. · Reviewed over 6 years ago

The lab is great, but there wasn't enough time given. I read the python file for the next step while the current step was running, and the timer for the lab ended just before the final Dataflow job was completed. I wasn't able to get the score & the completion for this lab.

Nir L. · Reviewed over 6 years ago