Building Realtime Pipelines in Cloud Data Fusion avaliações

Randi-Michelle C. · Revisado há over 4 years

Akira T. · Revisado há over 4 years

Sonit P. · Revisado há over 4 years

Hello, Cannot end labs due to error: 2021-03-26 00:09:39,723 - ERROR [Driver:i.c.c.e.v.LoggingFailureCollector@60] - Encountered '1' validation failures: 1. Stage 'Pub/Sub' encountered : Subscription Name does not match naming convention. Unexpected Character. Check Plugin documentation for naming convention. 2021-03-26 00:09:39,759 - WARN [Driver:i.c.c.d.SparkStreamingPipelineDriver@211] - Failed to emit field lineage operations for streaming pipeline org.apache.tephra.TransactionFailureException: Exception raised in transactional execution. Cause: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:82) ~[na:na] at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:68) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:227) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:138) ~[na:na] at io.cdap.cdap.app.runtime.spark.AbstractSparkExecutionContext.execute(AbstractSparkExecutionContext.scala:231) ~[na:na] at io.cdap.cdap.app.runtime.spark.SerializableSparkExecutionContext.execute(SerializableSparkExecutionContext.scala:61) ~[na:na] at io.cdap.cdap.app.runtime.spark.DefaultJavaSparkExecutionContext.execute(DefaultJavaSparkExecutionContext.scala:89) ~[na:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.prepareRun(SubmitterPlugin.java:69) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.common.submit.PipelinePhasePreparer.prepare(PipelinePhasePreparer.java:149) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.spark.AbstractSparkPreparer.prepare(AbstractSparkPreparer.java:87) ~[hydrator-spark-core2_2.11-6.3.0.jar:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.generateOperations(SparkFieldLineageRecorder.java:67) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.record(SparkFieldLineageRecorder.java:55) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:209) [program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:175) [program.jar.expanded.zip/:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper$.main(SparkMainWrapper.scala:87) [na:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper.main(SparkMainWrapper.scala) [na:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_275] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_275] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_275] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_275] at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:721) [spark-yarn_2.11-2.3.4.jar:2.3.4] Caused by: io.cdap.cdap.etl.api.validation.ValidationException: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.etl.validation.DefaultFailureCollector.getOrThrowException(DefaultFailureCollector.java:77) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.validation.LoggingFailureCollector.getOrThrowException(LoggingFailureCollector.java:50) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber$SubscriberConfig.validate(GoogleSubscriber.java:180) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:84) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:55) ~[1616717367995-0/:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.lambda$prepareRun$2(SubmitterPlugin.java:71) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional$2.run(SparkTransactional.java:236) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:208) ~[na:na] ... 18 common frames omitted Change name of subscription in the manual, ex for name "subtest" it works :)

Kacper M. · Revisado há over 4 years

Hello, Cannot end labs due to error: 2021-03-26 00:09:39,723 - ERROR [Driver:i.c.c.e.v.LoggingFailureCollector@60] - Encountered '1' validation failures: 1. Stage 'Pub/Sub' encountered : Subscription Name does not match naming convention. Unexpected Character. Check Plugin documentation for naming convention. 2021-03-26 00:09:39,759 - WARN [Driver:i.c.c.d.SparkStreamingPipelineDriver@211] - Failed to emit field lineage operations for streaming pipeline org.apache.tephra.TransactionFailureException: Exception raised in transactional execution. Cause: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:82) ~[na:na] at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:68) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:227) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:138) ~[na:na] at io.cdap.cdap.app.runtime.spark.AbstractSparkExecutionContext.execute(AbstractSparkExecutionContext.scala:231) ~[na:na] at io.cdap.cdap.app.runtime.spark.SerializableSparkExecutionContext.execute(SerializableSparkExecutionContext.scala:61) ~[na:na] at io.cdap.cdap.app.runtime.spark.DefaultJavaSparkExecutionContext.execute(DefaultJavaSparkExecutionContext.scala:89) ~[na:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.prepareRun(SubmitterPlugin.java:69) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.common.submit.PipelinePhasePreparer.prepare(PipelinePhasePreparer.java:149) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.spark.AbstractSparkPreparer.prepare(AbstractSparkPreparer.java:87) ~[hydrator-spark-core2_2.11-6.3.0.jar:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.generateOperations(SparkFieldLineageRecorder.java:67) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.record(SparkFieldLineageRecorder.java:55) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:209) [program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:175) [program.jar.expanded.zip/:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper$.main(SparkMainWrapper.scala:87) [na:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper.main(SparkMainWrapper.scala) [na:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_275] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_275] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_275] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_275] at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:721) [spark-yarn_2.11-2.3.4.jar:2.3.4] Caused by: io.cdap.cdap.etl.api.validation.ValidationException: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.etl.validation.DefaultFailureCollector.getOrThrowException(DefaultFailureCollector.java:77) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.validation.LoggingFailureCollector.getOrThrowException(LoggingFailureCollector.java:50) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber$SubscriberConfig.validate(GoogleSubscriber.java:180) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:84) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:55) ~[1616717367995-0/:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.lambda$prepareRun$2(SubmitterPlugin.java:71) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional$2.run(SparkTransactional.java:236) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:208) ~[na:na] ... 18 common frames omitted

Kacper M. · Revisado há over 4 years

ストリーミングでもデータを取り込めるのねってことが分かる・・・？

Yoichi K. · Revisado há over 4 years

Adipta M. · Revisado há over 4 years

Tran V. · Revisado há over 4 years

Alan B. · Revisado há over 4 years

I could not add the service account permission.

Aishwarya S. · Revisado há over 4 years

Bartlomiej J. · Revisado há over 4 years

Nguyen T. · Revisado há over 4 years

Nguyen T. · Revisado há over 4 years

Nguyen T. · Revisado há over 4 years

Mike Z. · Revisado há over 4 years

Juan C. · Revisado há over 4 years

Basma A. · Revisado há over 4 years

Piotr S. · Revisado há over 4 years

Kenneth S. · Revisado há over 4 years

Shivaranjani M. · Revisado há over 4 years

Jefferson V. · Revisado há over 4 years

Jérôme M. · Revisado há over 4 years

Charles S. · Revisado há over 4 years

Vidyadhar H. · Revisado há over 4 years

Dennys R. · Revisado há over 4 years