Opiniones sobre Procesamiento de datos sin servidores con Dataflow: CI/CD con Dataflow

4526 opiniones

s.blok1@chello.nl B. · Se revisó hace más de 1 año

levi a. · Se revisó hace más de 1 año

ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Invalid image version: 'composer-1.20.12-airflow-2.3.4' derived based on your input: 'composer-1.20.12-airflow-2.3.4'. Currently suggested new environment image versions are: [composer-2.6.4-airflow-2.6.3, composer-2.6.4-airflow-2.5.3]. Cloud Composer 1 will reach the end of support on March 25, 2024. Creation of new Cloud Composer 1 environments is only possible if the project already contains a Composer 1 environment in the same location. We recommend migration to Cloud Composer 2. ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Configuring node location is not supported for Cloud Composer environments in versions 2.0.0 and newer.

Serhii Z. · Se revisó hace más de 1 año

a lot of bugs

Umang M. · Se revisó hace más de 1 año

Lab out of date: Multiple errors due to composer v1 not v2: cd ~/ci-cd-for-data-processing-workflow/env-setup chmod +x set_composer_variables.sh ./set_composer_variables.sh Welcome to Cloud Shell! Type "help" to get started. Your Cloud Platform project in this session is set to qwiklabs-gcp-03-9b7a5801b2d0. Use “gcloud config set project [PROJECT_ID]” to change to a different project. student_00_ea9e4e2b2fa3@cloudshell:~ (qwiklabs-gcp-03-9b7a5801b2d0)$ git clone https://github.com/GoogleCloudPlatform/ci-cd-for-data-processing-workflow.git Cloning into 'ci-cd-for-data-processing-workflow'... remote: Enumerating objects: 172, done. remote: Counting objects: 100% (100/100), done. remote: Compressing objects: 100% (60/60), done. remote: Total 172 (delta 47), reused 68 (delta 33), pack-reused 72 Receiving objects: 100% (172/172), 55.99 KiB | 2.80 MiB/s, done. Resolving deltas: 100% (66/66), done. student_00_ea9e4e2b2fa3@cloudshell:~ (qwiklabs-gcp-03-9b7a5801b2d0)$ cd ~/ci-cd-for-data-processing-workflow/env-setup source set_env.sh student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME \ --location $COMPOSER_REGION \ --zone $COMPOSER_ZONE_ID \ --python-version 3 \ --image-version composer-1.20.8-airflow-1.10.15 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Invalid image version: 'composer-1.20.8-airflow-1.10.15' derived based on your input: 'composer-1.20.8-airflow-1.10.15'. Currently suggested new environment image versions are: [composer-2.6.5-airflow-2.7.3, composer-2.6.5-airflow-2.6.3, composer-2.6.5-airflow-2.5.3]. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-1.20.8-airflow-1.10.15 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Invalid image version: 'composer-1.20.8-airflow-1.10.15' derived based on your input: 'composer-1.20.8-airflow-1.10.15'. Currently suggested new environment image versions are: [composer-2.6.5-airflow-2.7.3, composer-2.6.5-airflow-2.6.3, composer-2.6.5-airflow-2.5.3]. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-1.20.8-airflow-1.10.15 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Invalid image version: 'composer-1.20.8-airflow-1.10.15' derived based on your input: 'composer-1.20.8-airflow-1.10.15'. Currently suggested new environment image versions are: [composer-2.6.5-airflow-2.7.3, composer-2.6.5-airflow-2.6.3, composer-2.6.5-airflow-2.5.3]. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-2.6.5-airflow-2.7.3 ERROR: (gcloud.composer.environments.create) Cannot specify --python-version with Composer 2.X or greater. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-2.6.5-airflow-2.7.3^C --zone $COMPOSER_ZONE_ID \ --python-version 3 \ --image-version composer-1.20.8-airflow-1.10.15 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Invalid image version: 'composer-1.20.8-airflow-1.10.15' derived based on your input: 'composer-1.20.8-airflow-1.10.15'. Currently suggested new environment image versions are: [composer-2.6.5-airflow-2.7.3, composer-2.6.5-airflow-2.6.3, composer-2.6.5-airflow-2.5.3]. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-1.20.8-airflow-1.10.15 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Invalid image version: 'composer-1.20.8-airflow-1.10.15' derived based on your input: 'composer-1.20.8-airflow-1.10.15'. Currently suggested new environment image versions are: [composer-2.6.5-airflow-2.7.3, composer-2.6.5-airflow-2.6.3, composer-2.6.5-airflow-2.5.3]. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-1.20.8-airflow-1.10.15ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Invalid image version: 'composer-1.20.8-airflow-1.10.15' derived based on your input: 'composer-1.20.8-airflow-1.10.15'. Currently suggested new environment image versions are: [composer-2.6.5-airflow-2.7.3, composer-2.6.5-airflow-2.6.3, composer-2.6.5-airflow-2.5.3]. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-2.6.5-airflow-2.7.3 ERROR: (gcloud.composer.environments.create) Cannot specify --python-version with Composer 2.X or greater. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-2.6.5-airflow-2.7.3^C student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --im age-version composer-2.6.5-airflow-2.7.3 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Configuring node location is not supported for Cloud Composer environments in versions 2.0.0 and newer. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --zone $COMPOSER_ZONE_ID --image-version composer-2.6.5-air flow-2.7.3 ERROR: (gcloud.composer.environments.create) Error parsing [environment]. The [environment] resource is not properly specified. Failed to find attribute [location]. The attribute can be set in the following ways: - provide the argument `environment` on the command line with a fully specified name - provide the argument `--location` on the command line - set the property `composer/location` student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --image-version composer-2.6.5-airflow-2.7.3 ERROR: (gcloud.composer.environments.create) FAILED_PRECONDITION: Composer API Service Agent service account (service-922910890009@cloudcomposer-accounts.iam.gserviceaccount.com) does not have required permissions set. Cloud Composer v2 API Service Agent Extension role might be missing. Please refer to https://cloud.google.com/composer/docs/composer-2/create-environments#grant-permissions and Composer Creation Troubleshooting pages to resolve this issue. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --image-version composer-2.6.5-airflow-2.7.3 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Configuring node location is not supported for Cloud Composer environments in versions 2.0.0 and newer. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID WARNING: Not defining --image-version resolves to Cloud Composer current default version, which is presently Composer 2 and is subject to further changes in the future. Consider using --image-version=composer-A-airflow-X[.Y[.Z]]. More info at https://cloud.google.com/composer/docs/concepts/versioning/composer-versioning-overview#version-aliases ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Configuring node location is not supported for Cloud Composer environments in versions 2.0.0 and newer. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --image-version composer-2.6.5-airflow-2.7.3 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Configuring node location is not supported for Cloud Composer environments in versions 2.0.0 and newer. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$

Mark G. · Se revisó hace más de 1 año

Bryan G. · Se revisó hace más de 1 año

given wrong composer image

JAQUELINE R. · Se revisó hace más de 1 año

Aakruti r. · Se revisó hace más de 1 año

Major issue/bug in this lab, commands are not working in the lab(outdated image versions used). Also, failing at step 9 of building pipeline even running after several times.

Priyank G. · Se revisó hace más de 1 año

Lab not working

Samson O. · Se revisó hace más de 1 año

Unable to create Cloud Composer environment with the steps in task 2. Task 5 commands fail

Brandt D. · Se revisó hace más de 1 año

Naveen S. · Se revisó hace más de 1 año

JaisainadhReddy B. · Se revisó hace más de 1 año

Not able to proceed as the current lab have only permission to create composer 1 but composer v1 ended on march 25th 2024. Please correct the lab.

Arun J. · Se revisó hace más de 1 año

Naveen S. · Se revisó hace más de 1 año

This is one of the worst labs I have ever taken. So many bugs in the code (maybe it used to work with Airflow v1) that make it impossible to finish in time when using Composer with Airflow v2, and that is the only supported version as returned by the gcloud command creating the Composer environment. Please review and update this lab or remove it from the curriculum.

Michal B. · Se revisó hace más de 1 año

Priyank G. · Se revisó hace más de 1 año

Composer version is not supported anymore. lots of error in solutions too.

Hemant K. · Se revisó hace más de 1 año

Unable to create Cloud Composer environment with the steps in task 2. Images are not compatible

Brandt D. · Se revisó hace más de 1 año

Hemant K. · Se revisó hace más de 1 año

EXCELENTE LABORATORIO

Carlos Mauricio C. · Se revisó hace más de 1 año

Michal B. · Se revisó hace más de 1 año

Guan Xin W. · Se revisó hace más de 1 año

No garantizamos que las opiniones publicadas provengan de consumidores que hayan comprado o utilizado los productos. Google no verifica las opiniones.