Відгуки: Building Realtime Pipelines in Cloud Data Fusion
3136 відгуків
Great lab! Some steps needed are not included, like copying the file to the computer, and give permission to Dataflow's service account
Leopoldo G. · Відгук надано майже 5 років тому
Kurtis M. · Відгук надано майже 5 років тому
Ravi R. · Відгук надано майже 5 років тому
ok
Juan David A. · Відгук надано близько 5 років тому
Niranjan A. · Відгук надано близько 5 років тому
Oscar R. · Відгук надано близько 5 років тому
Chris F. · Відгук надано близько 5 років тому
Thiago d. · Відгук надано близько 5 років тому
João S. · Відгук надано близько 5 років тому
Amauri S. · Відгук надано близько 5 років тому
Randi-Michelle C. · Відгук надано близько 5 років тому
Akira T. · Відгук надано близько 5 років тому
Sonit P. · Відгук надано близько 5 років тому
Hello, Cannot end labs due to error: 2021-03-26 00:09:39,723 - ERROR [Driver:i.c.c.e.v.LoggingFailureCollector@60] - Encountered '1' validation failures: 1. Stage 'Pub/Sub' encountered : Subscription Name does not match naming convention. Unexpected Character. Check Plugin documentation for naming convention. 2021-03-26 00:09:39,759 - WARN [Driver:i.c.c.d.SparkStreamingPipelineDriver@211] - Failed to emit field lineage operations for streaming pipeline org.apache.tephra.TransactionFailureException: Exception raised in transactional execution. Cause: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:82) ~[na:na] at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:68) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:227) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:138) ~[na:na] at io.cdap.cdap.app.runtime.spark.AbstractSparkExecutionContext.execute(AbstractSparkExecutionContext.scala:231) ~[na:na] at io.cdap.cdap.app.runtime.spark.SerializableSparkExecutionContext.execute(SerializableSparkExecutionContext.scala:61) ~[na:na] at io.cdap.cdap.app.runtime.spark.DefaultJavaSparkExecutionContext.execute(DefaultJavaSparkExecutionContext.scala:89) ~[na:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.prepareRun(SubmitterPlugin.java:69) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.common.submit.PipelinePhasePreparer.prepare(PipelinePhasePreparer.java:149) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.spark.AbstractSparkPreparer.prepare(AbstractSparkPreparer.java:87) ~[hydrator-spark-core2_2.11-6.3.0.jar:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.generateOperations(SparkFieldLineageRecorder.java:67) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.record(SparkFieldLineageRecorder.java:55) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:209) [program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:175) [program.jar.expanded.zip/:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper$.main(SparkMainWrapper.scala:87) [na:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper.main(SparkMainWrapper.scala) [na:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_275] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_275] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_275] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_275] at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:721) [spark-yarn_2.11-2.3.4.jar:2.3.4] Caused by: io.cdap.cdap.etl.api.validation.ValidationException: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.etl.validation.DefaultFailureCollector.getOrThrowException(DefaultFailureCollector.java:77) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.validation.LoggingFailureCollector.getOrThrowException(LoggingFailureCollector.java:50) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber$SubscriberConfig.validate(GoogleSubscriber.java:180) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:84) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:55) ~[1616717367995-0/:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.lambda$prepareRun$2(SubmitterPlugin.java:71) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional$2.run(SparkTransactional.java:236) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:208) ~[na:na] ... 18 common frames omitted Change name of subscription in the manual, ex for name "subtest" it works :)
Kacper M. · Відгук надано близько 5 років тому
Hello, Cannot end labs due to error: 2021-03-26 00:09:39,723 - ERROR [Driver:i.c.c.e.v.LoggingFailureCollector@60] - Encountered '1' validation failures: 1. Stage 'Pub/Sub' encountered : Subscription Name does not match naming convention. Unexpected Character. Check Plugin documentation for naming convention. 2021-03-26 00:09:39,759 - WARN [Driver:i.c.c.d.SparkStreamingPipelineDriver@211] - Failed to emit field lineage operations for streaming pipeline org.apache.tephra.TransactionFailureException: Exception raised in transactional execution. Cause: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:82) ~[na:na] at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:68) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:227) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:138) ~[na:na] at io.cdap.cdap.app.runtime.spark.AbstractSparkExecutionContext.execute(AbstractSparkExecutionContext.scala:231) ~[na:na] at io.cdap.cdap.app.runtime.spark.SerializableSparkExecutionContext.execute(SerializableSparkExecutionContext.scala:61) ~[na:na] at io.cdap.cdap.app.runtime.spark.DefaultJavaSparkExecutionContext.execute(DefaultJavaSparkExecutionContext.scala:89) ~[na:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.prepareRun(SubmitterPlugin.java:69) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.common.submit.PipelinePhasePreparer.prepare(PipelinePhasePreparer.java:149) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.spark.AbstractSparkPreparer.prepare(AbstractSparkPreparer.java:87) ~[hydrator-spark-core2_2.11-6.3.0.jar:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.generateOperations(SparkFieldLineageRecorder.java:67) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.record(SparkFieldLineageRecorder.java:55) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:209) [program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:175) [program.jar.expanded.zip/:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper$.main(SparkMainWrapper.scala:87) [na:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper.main(SparkMainWrapper.scala) [na:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_275] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_275] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_275] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_275] at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:721) [spark-yarn_2.11-2.3.4.jar:2.3.4] Caused by: io.cdap.cdap.etl.api.validation.ValidationException: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.etl.validation.DefaultFailureCollector.getOrThrowException(DefaultFailureCollector.java:77) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.validation.LoggingFailureCollector.getOrThrowException(LoggingFailureCollector.java:50) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber$SubscriberConfig.validate(GoogleSubscriber.java:180) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:84) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:55) ~[1616717367995-0/:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.lambda$prepareRun$2(SubmitterPlugin.java:71) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional$2.run(SparkTransactional.java:236) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:208) ~[na:na] ... 18 common frames omitted
Kacper M. · Відгук надано близько 5 років тому
ストリーミングでもデータを取り込めるのねってことが分かる・・・？
Yoichi K. · Відгук надано близько 5 років тому
Adipta M. · Відгук надано близько 5 років тому
Tran V. · Відгук надано близько 5 років тому
Alan B. · Відгук надано близько 5 років тому
I could not add the service account permission.
Aishwarya S. · Відгук надано близько 5 років тому
Bartlomiej J. · Відгук надано близько 5 років тому
Nguyen T. · Відгук надано близько 5 років тому
Nguyen T. · Відгук надано близько 5 років тому
Nguyen T. · Відгук надано близько 5 років тому
Mike Z. · Відгук надано близько 5 років тому
Ми не гарантуємо, що опубліковані відгуки написали клієнти, які придбали продукти чи скористалися ними. Відгуки не перевіряються Google.