Serverless Data Processing with Dataflow - Writing an ETL pipeline using Apache Beam and Dataflow (Java) Reviews

Maryna B. · Reviewed over 4 years ago

i want lab with python

PRUDHVEE R. · Reviewed over 4 years ago

Victor G. · Reviewed over 4 years ago

Great lab!

Cristóbal S. · Reviewed over 4 years ago

solution doesn't work: [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[85,10] cannot find symbol [ERROR] symbol: class ProcessElement [ERROR] location: class com.mypackage.pipeline.MyPipeline.JsonToCommonLog [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[58,27] cannot find symbol [ERROR] symbol: variable PipelineOptionsFactory [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[87,13] cannot find symbol [ERROR] symbol: class Gson [ERROR] location: class com.mypackage.pipeline.MyPipeline.JsonToCommonLog [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[87,29] cannot find symbol [ERROR] symbol: class Gson [ERROR] location: class com.mypackage.pipeline.MyPipeline.JsonToCommonLog [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[105,9] cannot find symbol [ERROR] symbol: class Pipeline [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[105,29] cannot find symbol [ERROR] symbol: variable Pipeline [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[106,16] cannot find symbol [ERROR] symbol: method setJobName(java.lang.String) [ERROR] location: variable options of type com.mypackage.pipeline.MyPipeline.Options [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[125,72] package BigQueryIO.Write does not exist [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[124,71] package BigQueryIO.Write does not exist [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[123,25] cannot find symbol [ERROR] symbol: variable BigQueryIO [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[121,37] cannot find symbol [ERROR] symbol: variable ParDo [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[120,39] cannot find symbol [ERROR] symbol: variable TextIO [ERROR] location: class com.mypackage.pipeline.MyPipeline

Mohammad G. · Reviewed over 4 years ago

Thomas M. · Reviewed over 4 years ago

Kavan D. · Reviewed over 4 years ago

Great Lab!

Rajesh Reddy K. · Reviewed over 4 years ago

Serhii K. · Reviewed over 4 years ago

The environment was not provisioned. The directory were not set up, and I had many authorization issues.

Yannick Q. · Reviewed over 4 years ago

Antonio A. · Reviewed over 4 years ago

Antonio A. · Reviewed over 4 years ago

Could not get the lab resources to work, the IDE was never ok so could not open it.

Mats W. · Reviewed over 4 years ago

Aravind r. · Reviewed over 4 years ago

Rahul H. · Reviewed over 4 years ago

Ionel M. · Reviewed over 4 years ago

Clicking on the console.cloud.google.com/dataflow/ link got me into trouble, because I wound up in a browser window for a different account and project. I didn't notice this until my dataflow parameterized job failed. Also, I followed the instructions for creating a schema.json file in Cloud Storage, but can't get the points for this step. Since I was able to see the file in Cloud Storage, and ultimately able to run the Dataflow job, I'm assuming that I did in fact successfully create the schema.json.

David V. · Reviewed over 4 years ago

Sachin M. · Reviewed over 4 years ago

ran out of time

Tony G. · Reviewed over 4 years ago

Tony M. · Reviewed over 4 years ago

Bharat M. · Reviewed almost 5 years ago

Florian R. · Reviewed almost 5 years ago

terrible lab

Piyush P. · Reviewed almost 5 years ago

getting error [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.0.0:java (default-cli) on project advanced-dataflow-lab-1-lab: An exception occured while executing the Java class. BigQueryIO.Write needs a GCS temp location to store temp files. -

Olga K. · Reviewed almost 5 years ago

I want Python, if available.

Steve V. · Reviewed almost 5 years ago