Processamento de dados sem servidor com o Dataflow: como criar um pipeline ETL usando Apache Beam e Dataflow (Java) avaliações

Yasin A. · Revisado há over 4 years

Balaji M. · Revisado há over 4 years

suryavamshi t. · Revisado há over 4 years

Maryna B. · Revisado há over 4 years

i want lab with python

PRUDHVEE R. · Revisado há over 4 years

Victor G. · Revisado há over 4 years

Great lab!

Cristóbal S. · Revisado há over 4 years

solution doesn't work: [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[85,10] cannot find symbol [ERROR] symbol: class ProcessElement [ERROR] location: class com.mypackage.pipeline.MyPipeline.JsonToCommonLog [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[58,27] cannot find symbol [ERROR] symbol: variable PipelineOptionsFactory [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[87,13] cannot find symbol [ERROR] symbol: class Gson [ERROR] location: class com.mypackage.pipeline.MyPipeline.JsonToCommonLog [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[87,29] cannot find symbol [ERROR] symbol: class Gson [ERROR] location: class com.mypackage.pipeline.MyPipeline.JsonToCommonLog [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[105,9] cannot find symbol [ERROR] symbol: class Pipeline [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[105,29] cannot find symbol [ERROR] symbol: variable Pipeline [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[106,16] cannot find symbol [ERROR] symbol: method setJobName(java.lang.String) [ERROR] location: variable options of type com.mypackage.pipeline.MyPipeline.Options [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[125,72] package BigQueryIO.Write does not exist [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[124,71] package BigQueryIO.Write does not exist [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[123,25] cannot find symbol [ERROR] symbol: variable BigQueryIO [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[121,37] cannot find symbol [ERROR] symbol: variable ParDo [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[120,39] cannot find symbol [ERROR] symbol: variable TextIO [ERROR] location: class com.mypackage.pipeline.MyPipeline

Mohammad G. · Revisado há over 4 years

Thomas M. · Revisado há over 4 years

Kavan D. · Revisado há over 4 years

Great Lab!

Rajesh Reddy K. · Revisado há over 4 years

Serhii K. · Revisado há over 4 years

The environment was not provisioned. The directory were not set up, and I had many authorization issues.

Yannick Q. · Revisado há over 4 years

Antonio A. · Revisado há over 4 years

Antonio A. · Revisado há over 4 years

Could not get the lab resources to work, the IDE was never ok so could not open it.

Mats W. · Revisado há over 4 years

Aravind r. · Revisado há over 4 years

Rahul H. · Revisado há over 4 years

Ionel M. · Revisado há over 4 years

Clicking on the console.cloud.google.com/dataflow/ link got me into trouble, because I wound up in a browser window for a different account and project. I didn't notice this until my dataflow parameterized job failed. Also, I followed the instructions for creating a schema.json file in Cloud Storage, but can't get the points for this step. Since I was able to see the file in Cloud Storage, and ultimately able to run the Dataflow job, I'm assuming that I did in fact successfully create the schema.json.

David V. · Revisado há over 4 years

Sachin M. · Revisado há over 4 years

ran out of time

Tony G. · Revisado há over 4 years

Tony M. · Revisado há over 4 years

Bharat M. · Revisado há over 4 years

Florian R. · Revisado há over 4 years