Creating a Streaming Data Pipeline for a Real-Time Dashboard with Dataflow Reviews

Daniela C. · Reviewed over 1 year ago

Just an opinion. I think in the following command (Task 5) the last line should not have the "timestamp" field, just the "minute" field. WITH streaming_data AS ( SELECT timestamp, TIMESTAMP_TRUNC(timestamp, HOUR, 'UTC') AS hour, TIMESTAMP_TRUNC(timestamp, MINUTE, 'UTC') AS minute, TIMESTAMP_TRUNC(timestamp, SECOND, 'UTC') AS second, ride_id, latitude, longitude, meter_reading, ride_status, passenger_count FROM taxirides.realtime ORDER BY timestamp DESC LIMIT 1000 ) # calculate aggregations on stream for reporting: SELECT ROW_NUMBER() OVER() AS dashboard_sort, minute, COUNT(DISTINCT ride_id) AS total_rides, SUM(meter_reading) AS total_revenue, SUM(passenger_count) AS total_passengers FROM streaming_data GROUP BY minute, timestamp

Igor A. · Reviewed over 1 year ago

JESUS ANTONIO C. · Reviewed over 1 year ago

Uriel F. · Reviewed over 1 year ago

Siddhant D. · Reviewed over 1 year ago

ALLAN F. · Reviewed over 1 year ago

I bit confuse uding the setup panel on looker studio

Raziel M. · Reviewed over 1 year ago

The instructions to fill in the Dataflow details were out of order and confusing. And I spend more time waiting for the Dataflow job to run than I did actually working.

Ben S. · Reviewed over 1 year ago

Fiorella Y. · Reviewed over 1 year ago

Frederik Adolfo S. · Reviewed over 1 year ago

Renatha Amelia M. · Reviewed over 1 year ago

John R. · Reviewed over 1 year ago

Brenno N. · Reviewed over 1 year ago

Roberto P. · Reviewed over 1 year ago

Ana P. · Reviewed over 1 year ago

Great!

Emerson C. · Reviewed over 1 year ago

Alan D. · Reviewed over 1 year ago

Afridha B. · Reviewed over 1 year ago

Thiago P. · Reviewed over 1 year ago

It was really clear! I thought that the Dataflow would end but then realized it was an "on going" or "live" job

Alberto A. · Reviewed over 1 year ago

Ayman S. · Reviewed over 1 year ago

Ernesto Rodolfo C. · Reviewed over 1 year ago

Arrijalul K. · Reviewed over 1 year ago

Rohit D. · Reviewed over 1 year ago

Rizki R. · Reviewed over 1 year ago