Dataflow によるサーバーレス データ処理 – Dataflow を使用した CI / CD のレビュー

andrade levi · 1年以上前にレビュー済み

ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Invalid image version: 'composer-1.20.12-airflow-2.3.4' derived based on your input: 'composer-1.20.12-airflow-2.3.4'. Currently suggested new environment image versions are: [composer-2.6.4-airflow-2.6.3, composer-2.6.4-airflow-2.5.3]. Cloud Composer 1 will reach the end of support on March 25, 2024. Creation of new Cloud Composer 1 environments is only possible if the project already contains a Composer 1 environment in the same location. We recommend migration to Cloud Composer 2. ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Configuring node location is not supported for Cloud Composer environments in versions 2.0.0 and newer.

Zenkov Serhii · 1年以上前にレビュー済み

a lot of bugs

Malviya Umang · 1年以上前にレビュー済み

Lab out of date: Multiple errors due to composer v1 not v2: cd ~/ci-cd-for-data-processing-workflow/env-setup chmod +x set_composer_variables.sh ./set_composer_variables.sh Welcome to Cloud Shell! Type "help" to get started. Your Cloud Platform project in this session is set to qwiklabs-gcp-03-9b7a5801b2d0. Use “gcloud config set project [PROJECT_ID]” to change to a different project. student_00_ea9e4e2b2fa3@cloudshell:~ (qwiklabs-gcp-03-9b7a5801b2d0)$ git clone https://github.com/GoogleCloudPlatform/ci-cd-for-data-processing-workflow.git Cloning into 'ci-cd-for-data-processing-workflow'... remote: Enumerating objects: 172, done. remote: Counting objects: 100% (100/100), done. remote: Compressing objects: 100% (60/60), done. remote: Total 172 (delta 47), reused 68 (delta 33), pack-reused 72 Receiving objects: 100% (172/172), 55.99 KiB | 2.80 MiB/s, done. Resolving deltas: 100% (66/66), done. student_00_ea9e4e2b2fa3@cloudshell:~ (qwiklabs-gcp-03-9b7a5801b2d0)$ cd ~/ci-cd-for-data-processing-workflow/env-setup source set_env.sh student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME \ --location $COMPOSER_REGION \ --zone $COMPOSER_ZONE_ID \ --python-version 3 \ --image-version composer-1.20.8-airflow-1.10.15 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Invalid image version: 'composer-1.20.8-airflow-1.10.15' derived based on your input: 'composer-1.20.8-airflow-1.10.15'. Currently suggested new environment image versions are: [composer-2.6.5-airflow-2.7.3, composer-2.6.5-airflow-2.6.3, composer-2.6.5-airflow-2.5.3]. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-1.20.8-airflow-1.10.15 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Invalid image version: 'composer-1.20.8-airflow-1.10.15' derived based on your input: 'composer-1.20.8-airflow-1.10.15'. Currently suggested new environment image versions are: [composer-2.6.5-airflow-2.7.3, composer-2.6.5-airflow-2.6.3, composer-2.6.5-airflow-2.5.3]. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-1.20.8-airflow-1.10.15 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Invalid image version: 'composer-1.20.8-airflow-1.10.15' derived based on your input: 'composer-1.20.8-airflow-1.10.15'. Currently suggested new environment image versions are: [composer-2.6.5-airflow-2.7.3, composer-2.6.5-airflow-2.6.3, composer-2.6.5-airflow-2.5.3]. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-2.6.5-airflow-2.7.3 ERROR: (gcloud.composer.environments.create) Cannot specify --python-version with Composer 2.X or greater. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-2.6.5-airflow-2.7.3^C --zone $COMPOSER_ZONE_ID \ --python-version 3 \ --image-version composer-1.20.8-airflow-1.10.15 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Invalid image version: 'composer-1.20.8-airflow-1.10.15' derived based on your input: 'composer-1.20.8-airflow-1.10.15'. Currently suggested new environment image versions are: [composer-2.6.5-airflow-2.7.3, composer-2.6.5-airflow-2.6.3, composer-2.6.5-airflow-2.5.3]. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-1.20.8-airflow-1.10.15 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Invalid image version: 'composer-1.20.8-airflow-1.10.15' derived based on your input: 'composer-1.20.8-airflow-1.10.15'. Currently suggested new environment image versions are: [composer-2.6.5-airflow-2.7.3, composer-2.6.5-airflow-2.6.3, composer-2.6.5-airflow-2.5.3]. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-1.20.8-airflow-1.10.15ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Invalid image version: 'composer-1.20.8-airflow-1.10.15' derived based on your input: 'composer-1.20.8-airflow-1.10.15'. Currently suggested new environment image versions are: [composer-2.6.5-airflow-2.7.3, composer-2.6.5-airflow-2.6.3, composer-2.6.5-airflow-2.5.3]. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-2.6.5-airflow-2.7.3 ERROR: (gcloud.composer.environments.create) Cannot specify --python-version with Composer 2.X or greater. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --python-version 3 --image-version composer-2.6.5-airflow-2.7.3^C student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --im age-version composer-2.6.5-airflow-2.7.3 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Configuring node location is not supported for Cloud Composer environments in versions 2.0.0 and newer. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --zone $COMPOSER_ZONE_ID --image-version composer-2.6.5-air flow-2.7.3 ERROR: (gcloud.composer.environments.create) Error parsing [environment]. The [environment] resource is not properly specified. Failed to find attribute [location]. The attribute can be set in the following ways: - provide the argument `environment` on the command line with a fully specified name - provide the argument `--location` on the command line - set the property `composer/location` student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --image-version composer-2.6.5-airflow-2.7.3 ERROR: (gcloud.composer.environments.create) FAILED_PRECONDITION: Composer API Service Agent service account (service-922910890009@cloudcomposer-accounts.iam.gserviceaccount.com) does not have required permissions set. Cloud Composer v2 API Service Agent Extension role might be missing. Please refer to https://cloud.google.com/composer/docs/composer-2/create-environments#grant-permissions and Composer Creation Troubleshooting pages to resolve this issue. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --image-version composer-2.6.5-airflow-2.7.3 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Configuring node location is not supported for Cloud Composer environments in versions 2.0.0 and newer. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID WARNING: Not defining --image-version resolves to Cloud Composer current default version, which is presently Composer 2 and is subject to further changes in the future. Consider using --image-version=composer-A-airflow-X[.Y[.Z]]. More info at https://cloud.google.com/composer/docs/concepts/versioning/composer-versioning-overview#version-aliases ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Configuring node location is not supported for Cloud Composer environments in versions 2.0.0 and newer. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$ gcloud composer environments create $COMPOSER_ENV_NAME --location $COMPOSER_REGION --zone $COMPOSER_ZONE_ID --image-version composer-2.6.5-airflow-2.7.3 ERROR: (gcloud.composer.environments.create) INVALID_ARGUMENT: Found 1 problem: 1) Configuring node location is not supported for Cloud Composer environments in versions 2.0.0 and newer. student_00_ea9e4e2b2fa3@cloudshell:~/ci-cd-for-data-processing-workflow/env-setup (qwiklabs-gcp-03-9b7a5801b2d0)$

Glover Mark · 1年以上前にレビュー済み

Galdámez Bryan · 1年以上前にレビュー済み

given wrong composer image

RANDO RIBEIRO DE ALMEIDA JAQUELINE · 1年以上前にレビュー済み

rathore Aakruti · 1年以上前にレビュー済み

Major issue/bug in this lab, commands are not working in the lab(outdated image versions used). Also, failing at step 9 of building pipeline even running after several times.

Ghosh Priyank · 1年以上前にレビュー済み

Lab not working

Okpara Samson · 1年以上前にレビュー済み

Unable to create Cloud Composer environment with the steps in task 2. Task 5 commands fail

Davis Brandt · 1年以上前にレビュー済み

Saboji Naveen · 1年以上前にレビュー済み

Saboji Naveen · 1年以上前にレビュー済み

Bapatu JaisainadhReddy · 1年以上前にレビュー済み

Not able to proceed as the current lab have only permission to create composer 1 but composer v1 ended on march 25th 2024. Please correct the lab.

James Arun · 1年以上前にレビュー済み

Saboji Naveen · 1年以上前にレビュー済み

This is one of the worst labs I have ever taken. So many bugs in the code (maybe it used to work with Airflow v1) that make it impossible to finish in time when using Composer with Airflow v2, and that is the only supported version as returned by the gcloud command creating the Composer environment. Please review and update this lab or remove it from the curriculum.

Bochenek Michal · 1年以上前にレビュー済み

Ghosh Priyank · 1年以上前にレビュー済み

Composer version is not supported anymore. lots of error in solutions too.

Kumar Hemant · 1年以上前にレビュー済み

Kumar Hemant · 1年以上前にレビュー済み

Unable to create Cloud Composer environment with the steps in task 2. Images are not compatible

Davis Brandt · 1年以上前にレビュー済み

Kumar Hemant · 1年以上前にレビュー済み

EXCELENTE LABORATORIO

Cota Vega Carlos Mauricio · 1年以上前にレビュー済み

This is one of the worst labs I have ever taken. So many bugs in the code (maybe it used to work with Airflow v1) that make it impossible to finish in time when using Composer with Airflow v2, and that is the only supported version as returned by the gcloud command creating the Composer environment. Please review this lab or remove it from the curriculum.

Bochenek Michal · 1年以上前にレビュー済み

Wang Guan Xin · 1年以上前にレビュー済み

It shows error at image version composer and airflow versions

Signin LTI · 1年以上前にレビュー済み