Building Realtime Pipelines in Cloud Data Fusion のレビュー

Cowin Randi-Michelle · 4年以上前にレビュー済み

Tanaka Akira · 4年以上前にレビュー済み

Panigrahy Sonit · 4年以上前にレビュー済み

Hello, Cannot end labs due to error: 2021-03-26 00:09:39,723 - ERROR [Driver:i.c.c.e.v.LoggingFailureCollector@60] - Encountered '1' validation failures: 1. Stage 'Pub/Sub' encountered : Subscription Name does not match naming convention. Unexpected Character. Check Plugin documentation for naming convention. 2021-03-26 00:09:39,759 - WARN [Driver:i.c.c.d.SparkStreamingPipelineDriver@211] - Failed to emit field lineage operations for streaming pipeline org.apache.tephra.TransactionFailureException: Exception raised in transactional execution. Cause: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:82) ~[na:na] at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:68) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:227) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:138) ~[na:na] at io.cdap.cdap.app.runtime.spark.AbstractSparkExecutionContext.execute(AbstractSparkExecutionContext.scala:231) ~[na:na] at io.cdap.cdap.app.runtime.spark.SerializableSparkExecutionContext.execute(SerializableSparkExecutionContext.scala:61) ~[na:na] at io.cdap.cdap.app.runtime.spark.DefaultJavaSparkExecutionContext.execute(DefaultJavaSparkExecutionContext.scala:89) ~[na:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.prepareRun(SubmitterPlugin.java:69) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.common.submit.PipelinePhasePreparer.prepare(PipelinePhasePreparer.java:149) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.spark.AbstractSparkPreparer.prepare(AbstractSparkPreparer.java:87) ~[hydrator-spark-core2_2.11-6.3.0.jar:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.generateOperations(SparkFieldLineageRecorder.java:67) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.record(SparkFieldLineageRecorder.java:55) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:209) [program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:175) [program.jar.expanded.zip/:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper$.main(SparkMainWrapper.scala:87) [na:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper.main(SparkMainWrapper.scala) [na:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_275] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_275] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_275] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_275] at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:721) [spark-yarn_2.11-2.3.4.jar:2.3.4] Caused by: io.cdap.cdap.etl.api.validation.ValidationException: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.etl.validation.DefaultFailureCollector.getOrThrowException(DefaultFailureCollector.java:77) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.validation.LoggingFailureCollector.getOrThrowException(LoggingFailureCollector.java:50) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber$SubscriberConfig.validate(GoogleSubscriber.java:180) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:84) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:55) ~[1616717367995-0/:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.lambda$prepareRun$2(SubmitterPlugin.java:71) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional$2.run(SparkTransactional.java:236) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:208) ~[na:na] ... 18 common frames omitted Change name of subscription in the manual, ex for name "subtest" it works :)

Mucha Kacper · 4年以上前にレビュー済み

Hello, Cannot end labs due to error: 2021-03-26 00:09:39,723 - ERROR [Driver:i.c.c.e.v.LoggingFailureCollector@60] - Encountered '1' validation failures: 1. Stage 'Pub/Sub' encountered : Subscription Name does not match naming convention. Unexpected Character. Check Plugin documentation for naming convention. 2021-03-26 00:09:39,759 - WARN [Driver:i.c.c.d.SparkStreamingPipelineDriver@211] - Failed to emit field lineage operations for streaming pipeline org.apache.tephra.TransactionFailureException: Exception raised in transactional execution. Cause: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:82) ~[na:na] at io.cdap.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:68) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:227) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:138) ~[na:na] at io.cdap.cdap.app.runtime.spark.AbstractSparkExecutionContext.execute(AbstractSparkExecutionContext.scala:231) ~[na:na] at io.cdap.cdap.app.runtime.spark.SerializableSparkExecutionContext.execute(SerializableSparkExecutionContext.scala:61) ~[na:na] at io.cdap.cdap.app.runtime.spark.DefaultJavaSparkExecutionContext.execute(DefaultJavaSparkExecutionContext.scala:89) ~[na:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.prepareRun(SubmitterPlugin.java:69) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.common.submit.PipelinePhasePreparer.prepare(PipelinePhasePreparer.java:149) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.spark.AbstractSparkPreparer.prepare(AbstractSparkPreparer.java:87) ~[hydrator-spark-core2_2.11-6.3.0.jar:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.generateOperations(SparkFieldLineageRecorder.java:67) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkFieldLineageRecorder.record(SparkFieldLineageRecorder.java:55) ~[program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:209) [program.jar.expanded.zip/:na] at io.cdap.cdap.datastreams.SparkStreamingPipelineDriver.run(SparkStreamingPipelineDriver.java:175) [program.jar.expanded.zip/:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper$.main(SparkMainWrapper.scala:87) [na:na] at io.cdap.cdap.app.runtime.spark.SparkMainWrapper.main(SparkMainWrapper.scala) [na:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_275] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_275] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_275] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_275] at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:721) [spark-yarn_2.11-2.3.4.jar:2.3.4] Caused by: io.cdap.cdap.etl.api.validation.ValidationException: Errors were encountered during validation. Subscription Name does not match naming convention. at io.cdap.cdap.etl.validation.DefaultFailureCollector.getOrThrowException(DefaultFailureCollector.java:77) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.etl.validation.LoggingFailureCollector.getOrThrowException(LoggingFailureCollector.java:50) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber$SubscriberConfig.validate(GoogleSubscriber.java:180) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:84) ~[1616717367995-0/:na] at io.cdap.plugin.gcp.publisher.source.GoogleSubscriber.prepareRun(GoogleSubscriber.java:55) ~[1616717367995-0/:na] at io.cdap.cdap.etl.common.submit.SubmitterPlugin.lambda$prepareRun$2(SubmitterPlugin.java:71) ~[cdap-etl-core-6.3.0.jar:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional$2.run(SparkTransactional.java:236) ~[na:na] at io.cdap.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:208) ~[na:na] ... 18 common frames omitted

Mucha Kacper · 4年以上前にレビュー済み

ストリーミングでもデータを取り込めるのねってことが分かる・・・？

Kojima Yoichi · 4年以上前にレビュー済み

Martulandi Adipta · 4年以上前にレビュー済み

Vu Tran · 4年以上前にレビュー済み

Burye Alan · 4年以上前にレビュー済み

I could not add the service account permission.

Sontakke Aishwarya · 4年以上前にレビュー済み

Jankowski Bartlomiej · 4年以上前にレビュー済み

Truong The Anh Nguyen · 4年以上前にレビュー済み

Zhang Mike · 4年以上前にレビュー済み

Cordero Ruiz Juan · 4年以上前にレビュー済み

Alkerm Basma · 4年以上前にレビュー済み

Stańczak Piotr · 4年以上前にレビュー済み

Schroer Kenneth · 4年以上前にレビュー済み

M Shivaranjani · 4年以上前にレビュー済み

Veríssimo Jefferson · 4年以上前にレビュー済み

MASSOT Jérôme · 4年以上前にレビュー済み

Saroka Charles · 4年以上前にレビュー済み

Harmalkar Vidyadhar · 4年以上前にレビュー済み

Romero Dennys · 4年以上前にレビュー済み