Serverless Data Processing with Dataflow - Writing an ETL pipeline using Apache Beam and Dataflow (Java) Ulasan

Sriyansh S. · Diulas sekitar 4 jam lalu

Sriyansh S. · Diulas sekitar 4 jam lalu

Aubin C. · Diulas sekitar 8 jam lalu

The first part of the lab was confusing. The lab gives an ide-url. This was the first time, I saw an ide-url. However, the lab didnt suggest how to use it. I had to do the following out of my own understanding: I had to find a VM Instance in the Project. SSH into it. And then do the following command: cd /home/theia-java-dataflow/training-data-analyst/quests/dataflow/ Maven was not installed, so I had to do the following: sudo apt install maven On running, the following command, I got an error ("ERROR: (gcloud) Invalid choice: 'storage'.") source create_batch_sinks.sh I had to run sudo gcloud components update. Even that didnt work so I had to use echo "Creating pipeline sinks" PROJECT_ID=$(gcloud config get-value project) # GCS buckets #TODO: Add try/catch for the first bucket since qwiklabs #gcloud storage buckets create --location=US gs://$PROJECT_ID gsutil mb -l US gs://$PROJECT_ID #gcloud storage buckets create --location=US --default-storage-class="COLDLINE" gs://$PROJECT_ID-coldline gsutil mb -l US -c COLDLINE gs://$PROJECT_ID-coldline But as several files contained the gcloud commands and I could not change all of the files to gsutil commands. So I had to install a local version of gcloud with the following statements: #Step0: cd ~ #Step1: curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-linux-x86_64.tar.gz #Step2: tar -xf google-cloud-cli-linux-x86_64.tar.gz #Step3: sudo ./google-cloud-sdk/install.sh I also gto pip3 errors. So I had to install it with below statements: sudo apt-get update sudo apt-get install python3-pip -y Now I am almost at the endo of the lab. I have executed "bash generate_batch_events.sh" but it is stuck at "Installing Packages". I dont think, I would be able to complete the lab.

Sayed Fawad Ali S. · Diulas sekitar 15 jam lalu

Sriyansh S. · Diulas 1 hari lalu

Daniel A. · Diulas 2 hari lalu

Gustavo L. · Diulas 3 hari lalu

Ferdie O. · Diulas 3 hari lalu

Harsh A. · Diulas 3 hari lalu

IDE URL wouldn't work, restarting the lab.

Ferdie O. · Diulas 3 hari lalu

Allan L. · Diulas 3 hari lalu

IDE URL wouldn't work, restarting the lab.

Ferdie O. · Diulas 3 hari lalu

Emerson C. · Diulas 4 hari lalu

Iván Marcelo U. · Diulas 5 hari lalu

Francisco M. · Diulas 5 hari lalu

Leonardo M. · Diulas 5 hari lalu

IDE URL wouldn't work, restarting the lab.

Ferdie O. · Diulas 5 hari lalu

Selina C. · Diulas 5 hari lalu

the ide didn't work

Souhail L. · Diulas 6 hari lalu

Sriyansh S. · Diulas 6 hari lalu

Luis Antonio C. · Diulas 6 hari lalu

Luis Antonio C. · Diulas 7 hari lalu

Daniela L. · Diulas 8 hari lalu

Choh J. · Diulas 9 hari lalu

he reintentado varias veces, al momento de realizar el lab no hay recursos en la cuenta de estudio quedando pegada en la tarea 6

Gabriela C. · Diulas 10 hari lalu