[Deprecated] TFX Metadata recensioni

Prasuna M. · Recensione inserita oltre un anno fa

Roman T. · Recensione inserita oltre un anno fa

J H. · Recensione inserita oltre un anno fa

Hadrián P. · Recensione inserita oltre un anno fa

Scot J. · Recensione inserita oltre un anno fa

Ada A. · Recensione inserita oltre un anno fa

Yomna M. · Recensione inserita oltre un anno fa

Ezra J. · Recensione inserita oltre un anno fa

Vlad L. · Recensione inserita oltre un anno fa

2024-02-27 11:25:35.208831: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/lib 2024-02-27 11:25:35.208907: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. INFO:absl:tensorflow_ranking is not available: No module named 'tensorflow_ranking' INFO:absl:tensorflow_text is not available: No module named 'tensorflow_text' INFO:absl:Running driver for Tuner INFO:absl:MetadataStore with gRPC connection initialized INFO:absl:Adding KFP pod name tfx-covertype-lab-04-nxkkw-2692208594 to execution INFO:absl:Running executor for Tuner INFO:absl:Attempting to infer TFX Python dependency for beam INFO:absl:Copying all content from install dir /tfx-src/tfx to temp dir /tmp/tmp4c8gwt61/build/tfx INFO:absl:Generating a temp setup file at /tmp/tmp4c8gwt61/build/tfx/setup.py INFO:absl:Creating temporary sdist package, logs available at /tmp/tmp4c8gwt61/build/tfx/setup.log INFO:absl:Added --extra_package=/tmp/tmp4c8gwt61/build/tfx/dist/tfx_ephemeral-0.25.0.tar.gz to beam args WARNING:absl:workerCount is overridden with 2 INFO:absl:json_inputs='{"examples": [{"artifact": {"id": "21", "type_id": "18", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transformed_examples/21", "properties": {"split_names": {"string_value": "[\"train\", \"eval\"]"}}, "custom_properties": {"name": {"string_value": "transformed_examples"}, "producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}}, "state": "LIVE", "create_time_since_epoch": "1709033025920", "last_update_time_since_epoch": "1709033119881"}, "artifact_type": {"id": "18", "name": "Examples", "properties": {"split_names": "STRING", "span": "INT", "version": "INT"}}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "Examples"}], "transform_graph": [{"artifact": {"id": "20", "type_id": "22", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transform_graph/21", "custom_properties": {"producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}, "name": {"string_value": "transform_graph"}}, "state": "LIVE", "create_time_since_epoch": "1709033025917", "last_update_time_since_epoch": "1709033119878"}, "artifact_type": {"id": "22", "name": "TransformGraph"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "TransformGraph"}]}'. INFO:absl:json_outputs='{"best_hyperparameters": [{"artifact": {"id": "25", "type_id": "28", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Tuner/best_hyperparameters/24", "custom_properties": {"name": {"string_value": "best_hyperparameters"}, "producer_component": {"string_value": "Tuner"}}}, "artifact_type": {"id": "28", "name": "HyperParameters"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "HyperParameters"}]}'. INFO:absl:json_exec_properties='{"custom_config": "{\"ai_platform_training_args\": {\"masterConfig\": {\"imageUri\": \"gcr.io/qwiklabs-gcp-01-272c395e4a02/tfx_covertype_lab_04@sha256:276d6670faaf7a4939cc0b991101cad0e92bb3ad870973a3018474337aaa1647\"}, \"project\": \"qwiklabs-gcp-01-272c395e4a02\", \"region\": \"us-central1\", \"serviceAccount\": \"tfx-tuner-caip-service-account@qwiklabs-gcp-01-272c395e4a02.iam.gserviceaccount.com\"}}", "eval_args": "{\"num_steps\": 500}", "kfp_pod_name": "tfx-covertype-lab-04-nxkkw-2692208594", "module_file": "model.py", "train_args": "{\"num_steps\": 5000}", "tune_args": "{\n \"num_parallel_trials\": 3\n}", "tuner_fn": null}'. WARNING:googleapiclient.discovery_cache:file_cache is unavailable when using oauth2client >= 4.0.0 or google-auth Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/googleapiclient/discovery_cache/__init__.py", line 36, in autodetect from google.appengine.api import memcache ModuleNotFoundError: No module named 'google.appengine' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/googleapiclient/discovery_cache/file_cache.py", line 33, in from oauth2client.contrib.locked_file import LockedFile ModuleNotFoundError: No module named 'oauth2client.contrib.locked_file' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/googleapiclient/discovery_cache/file_cache.py", line 37, in from oauth2client.locked_file import LockedFile ModuleNotFoundError: No module named 'oauth2client.locked_file' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/googleapiclient/discovery_cache/__init__.py", line 42, in autodetect from . import file_cache File "/usr/local/lib/python3.7/dist-packages/googleapiclient/discovery_cache/file_cache.py", line 41, in "file_cache is unavailable when using oauth2client >= 4.0.0 or google-auth" ImportError: file_cache is unavailable when using oauth2client >= 4.0.0 or google-auth INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/discovery/v1/apis/ml/v1/rest INFO:absl:TrainingInput={'masterConfig': {'imageUri': 'gcr.io/qwiklabs-gcp-01-272c395e4a02/tfx_covertype_lab_04@sha256:276d6670faaf7a4939cc0b991101cad0e92bb3ad870973a3018474337aaa1647', 'containerCommand': ['python', '-m', 'tfx.scripts.run_executor', '--executor_class_path', 'tfx.extensions.google_cloud_ai_platform.tuner.executor._WorkerExecutor', '--inputs', '{"examples": [{"artifact": {"id": "21", "type_id": "18", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transformed_examples/21", "properties": {"split_names": {"string_value": "[\\"train\\", \\"eval\\"]"}}, "custom_properties": {"name": {"string_value": "transformed_examples"}, "producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}}, "state": "LIVE", "create_time_since_epoch": "1709033025920", "last_update_time_since_epoch": "1709033119881"}, "artifact_type": {"id": "18", "name": "Examples", "properties": {"split_names": "STRING", "span": "INT", "version": "INT"}}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "Examples"}], "transform_graph": [{"artifact": {"id": "20", "type_id": "22", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transform_graph/21", "custom_properties": {"producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}, "name": {"string_value": "transform_graph"}}, "state": "LIVE", "create_time_since_epoch": "1709033025917", "last_update_time_since_epoch": "1709033119878"}, "artifact_type": {"id": "22", "name": "TransformGraph"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "TransformGraph"}]}', '--outputs', '{"best_hyperparameters": [{"artifact": {"id": "25", "type_id": "28", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Tuner/best_hyperparameters/24", "custom_properties": {"name": {"string_value": "best_hyperparameters"}, "producer_component": {"string_value": "Tuner"}}}, "artifact_type": {"id": "28", "name": "HyperParameters"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "HyperParameters"}]}', '--exec-properties', '{"custom_config": "{\\"ai_platform_training_args\\": {\\"masterConfig\\": {\\"imageUri\\": \\"gcr.io/qwiklabs-gcp-01-272c395e4a02/tfx_covertype_lab_04@sha256:276d6670faaf7a4939cc0b991101cad0e92bb3ad870973a3018474337aaa1647\\"}, \\"project\\": \\"qwiklabs-gcp-01-272c395e4a02\\", \\"region\\": \\"us-central1\\", \\"serviceAccount\\": \\"tfx-tuner-caip-service-account@qwiklabs-gcp-01-272c395e4a02.iam.gserviceaccount.com\\"}}", "eval_args": "{\\"num_steps\\": 500}", "kfp_pod_name": "tfx-covertype-lab-04-nxkkw-2692208594", "module_file": "model.py", "train_args": "{\\"num_steps\\": 5000}", "tune_args": "{\\n \\"num_parallel_trials\\": 3\\n}", "tuner_fn": null}']}, 'region': 'us-central1', 'serviceAccount': 'tfx-tuner-caip-service-account@qwiklabs-gcp-01-272c395e4a02.iam.gserviceaccount.com', 'workerCount': 2, 'scaleTier': 'CUSTOM', 'masterType': 'standard', 'workerType': 'standard'} INFO:absl:Submitting job='tfx_tuner_20240227112545', project='qwiklabs-gcp-01-272c395e4a02' to AI Platform. INFO:googleapiclient.discovery:URL being requested: POST https://ml.googleapis.com/v1/projects/qwiklabs-gcp-01-272c395e4a02/jobs?alt=json INFO:googleapiclient.discovery:URL being requested: GET https://ml.googleapis.com/v1/projects/qwiklabs-gcp-01-272c395e4a02/jobs/tfx_tuner_20240227112545?alt=json ERROR:absl:Job 'projects/qwiklabs-gcp-01-272c395e4a02/jobs/tfx_tuner_20240227112545' did not succeed. Detailed response {'jobId': 'tfx_tuner_20240227112545', 'trainingInput': {'scaleTier': 'CUSTOM', 'masterType': 'standard', 'workerType': 'standard', 'workerCount': '2', 'region': 'us-central1', 'masterConfig': {'imageUri': 'gcr.io/qwiklabs-gcp-01-272c395e4a02/tfx_covertype_lab_04@sha256:276d6670faaf7a4939cc0b991101cad0e92bb3ad870973a3018474337aaa1647', 'containerCommand': ['python', '-m', 'tfx.scripts.run_executor', '--executor_class_path', 'tfx.extensions.google_cloud_ai_platform.tuner.executor._WorkerExecutor', '--inputs', '{"examples": [{"artifact": {"id": "21", "type_id": "18", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transformed_examples/21", "properties": {"split_names": {"string_value": "[\\"train\\", \\"eval\\"]"}}, "custom_properties": {"name": {"string_value": "transformed_examples"}, "producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}}, "state": "LIVE", "create_time_since_epoch": "1709033025920", "last_update_time_since_epoch": "1709033119881"}, "artifact_type": {"id": "18", "name": "Examples", "properties": {"split_names": "STRING", "span": "INT", "version": "INT"}}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "Examples"}], "transform_graph": [{"artifact": {"id": "20", "type_id": "22", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transform_graph/21", "custom_properties": {"producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}, "name": {"string_value": "transform_graph"}}, "state": "LIVE", "create_time_since_epoch": "1709033025917", "last_update_time_since_epoch": "1709033119878"}, "artifact_type": {"id": "22", "name": "TransformGraph"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "TransformGraph"}]}', '--outputs', '{"best_hyperparameters": [{"artifact": {"id": "25", "type_id": "28", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Tuner/best_hyperparameters/24", "custom_properties": {"name": {"string_value": "best_hyperparameters"}, "producer_component": {"string_value": "Tuner"}}}, "artifact_type": {"id": "28", "name": "HyperParameters"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "HyperParameters"}]}', '--exec-properties', '{"custom_config": "{\\"ai_platform_training_args\\": {\\"masterConfig\\": {\\"imageUri\\": \\"gcr.io/qwiklabs-gcp-01-272c395e4a02/tfx_covertype_lab_04@sha256:276d6670faaf7a4939cc0b991101cad0e92bb3ad870973a3018474337aaa1647\\"}, \\"project\\": \\"qwiklabs-gcp-01-272c395e4a02\\", \\"region\\": \\"us-central1\\", \\"serviceAccount\\": \\"tfx-tuner-caip-service-account@qwiklabs-gcp-01-272c395e4a02.iam.gserviceaccount.com\\"}}", "eval_args": "{\\"num_steps\\": 500}", "kfp_pod_name": "tfx-covertype-lab-04-nxkkw-2692208594", "module_file": "model.py", "train_args": "{\\"num_steps\\": 5000}", "tune_args": "{\\n \\"num_parallel_trials\\": 3\\n}", "tuner_fn": null}']}, 'serviceAccount': 'tfx-tuner-caip-service-account@qwiklabs-gcp-01-272c395e4a02.iam.gserviceaccount.com'}, 'createTime': '2024-02-27T11:25:45Z', 'startTime': '2024-02-27T11:26:00Z', 'endTime': '2024-02-27T11:26:32Z', 'state': 'FAILED', 'errorMessage': 'The replica master 0 exited with a non-zero status of 1. Termination reason: Error. The replica worker 0 exited with a non-zero status of 1. Termination reason: Error. The replica worker 1 exited with a non-zero status of 1. Termination reason: Error. To find out more about why your job exited please check the logs: https://console.cloud.google.com/logs/viewer?project=999903194638&resource=ml_job%2Fjob_id%2Ftfx_tuner_20240227112545&advancedFilter=resource.type%3D%22ml_job%22%0Aresource.labels.job_id%3D%22tfx_tuner_20240227112545%22', 'trainingOutput': {'consumedMLUnits': 0.01}, 'labels': {'tfx_executor': 'ensions-google_cloud_ai_platform-tuner-executor-_workerexecutor', 'tfx_py_version': '3-7', 'tfx_runner': 'kfp', 'tfx_version': '0-25-0'}, 'etag': 'e43TZkKQZ4E=', 'jobPosition': '0'}. Traceback (most recent call last): File "/tfx-src/tfx/orchestration/kubeflow/container_entrypoint.py", line 360, in main() File "/tfx-src/tfx/orchestration/kubeflow/container_entrypoint.py", line 353, in main execution_info = launcher.launch() File "/tfx-src/tfx/orchestration/launcher/base_component_launcher.py", line 209, in launch copy.deepcopy(execution_decision.exec_properties)) File "/tfx-src/tfx/orchestration/launcher/in_process_component_launcher.py", line 72, in _run_executor copy.deepcopy(input_dict), output_dict, copy.deepcopy(exec_properties)) File "/tfx-src/tfx/extensions/google_cloud_ai_platform/tuner/executor.py", line 121, in Do job_id) File "/tfx-src/tfx/extensions/google_cloud_ai_platform/runner.py", line 305, in start_aip_training job_labels=job_labels) File "/tfx-src/tfx/extensions/google_cloud_ai_platform/runner.py", line 194, in _launch_aip_training raise RuntimeError(err_msg) RuntimeError: Job 'projects/qwiklabs-gcp-01-272c395e4a02/jobs/tfx_tuner_20240227112545' did not succeed. Detailed response {'jobId': 'tfx_tuner_20240227112545', 'trainingInput': {'scaleTier': 'CUSTOM', 'masterType': 'standard', 'workerType': 'standard', 'workerCount': '2', 'region': 'us-central1', 'masterConfig': {'imageUri': 'gcr.io/qwiklabs-gcp-01-272c395e4a02/tfx_covertype_lab_04@sha256:276d6670faaf7a4939cc0b991101cad0e92bb3ad870973a3018474337aaa1647', 'containerCommand': ['python', '-m', 'tfx.scripts.run_executor', '--executor_class_path', 'tfx.extensions.google_cloud_ai_platform.tuner.executor._WorkerExecutor', '--inputs', '{"examples": [{"artifact": {"id": "21", "type_id": "18", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transformed_examples/21", "properties": {"split_names": {"string_value": "[\\"train\\", \\"eval\\"]"}}, "custom_properties": {"name": {"string_value": "transformed_examples"}, "producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}}, "state": "LIVE", "create_time_since_epoch": "1709033025920", "last_update_time_since_epoch": "1709033119881"}, "artifact_type": {"id": "18", "name": "Examples", "properties": {"split_names": "STRING", "span": "INT", "version": "INT"}}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "Examples"}], "transform_graph": [{"artifact": {"id": "20", "type_id": "22", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transform_graph/21", "custom_properties": {"producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}, "name": {"string_value": "transform_graph"}}, "state": "LIVE", "create_time_since_epoch": "1709033025917", "last_update_time_since_epoch": "1709033119878"}, "artifact_type": {"id": "22", "name": "TransformGraph"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "TransformGraph"}]}', '--outputs', '{"best_hyperparameters": [{"artifact": {"id": "25", "type_id": "28", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Tuner/best_hyperparameters/24", "custom_properties": {"name": {"string_value": "best_hyperparameters"}, "producer_component": {"string_value": "Tuner"}}}, "artifact_type": {"id": "28", "name": "HyperParameters"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "HyperParameters"}]}', '--exec-properties', '{"custom_config": "{\\"ai_platform_training_args\\": {\\"masterConfig\\": {\\"imageUri\\": \\"gcr.io/qwiklabs-gcp-01-272c395e4a02/tfx_covertype_lab_04@sha256:276d6670faaf7a4939cc0b991101cad0e92bb3ad870973a3018474337aaa1647\\"}, \\"project\\": \\"qwiklabs-gcp-01-272c395e4a02\\", \\"region\\": \\"us-central1\\", \\"serviceAccount\\": \\"tfx-tuner-caip-service-account@qwiklabs-gcp-01-272c395e4a02.iam.gserviceaccount.com\\"}}", "eval_args": "{\\"num_steps\\": 500}", "kfp_pod_name": "tfx-covertype-lab-04-nxkkw-2692208594", "module_file": "model.py", "train_args": "{\\"num_steps\\": 5000}", "tune_args": "{\\n \\"num_parallel_trials\\": 3\\n}", "tuner_fn": null}']}, 'serviceAccount': 'tfx-tuner-caip-service-account@qwiklabs-gcp-01-272c395e4a02.iam.gserviceaccount.com'}, 'createTime': '2024-02-27T11:25:45Z', 'startTime': '2024-02-27T11:26:00Z', 'endTime': '2024-02-27T11:26:32Z', 'state': 'FAILED', 'errorMessage': 'The replica master 0 exited with a non-zero status of 1. Termination reason: Error. The replica worker 0 exited with a non-zero status of 1. Termination reason: Error. The replica worker 1 exited with a non-zero status of 1. Termination reason: Error. To find out more about why your job exited please check the logs: https://console.cloud.google.com/logs/viewer?project=999903194638&resource=ml_job%2Fjob_id%2Ftfx_tuner_20240227112545&advancedFilter=resource.type%3D%22ml_job%22%0Aresource.labels.job_id%3D%22tfx_tuner_20240227112545%22', 'trainingOutput': {'consumedMLUnits': 0.01}, 'labels': {'tfx_executor': 'ensions-google_cloud_ai_platform-tuner-executor-_workerexecutor', 'tfx_py_version': '3-7', 'tfx_runner': 'kfp', 'tfx_version': '0-25-0'}, 'etag': 'e43TZkKQZ4E=', 'jobPosition': '0'}.

Dmytro Y. · Recensione inserita oltre un anno fa

siong n. · Recensione inserita oltre un anno fa

VU T. · Recensione inserita oltre un anno fa

Szymon T. · Recensione inserita oltre un anno fa

The AI PLatform no longer exists and the setting up KubeFlow pipeline instance is not an option anymore. This causes confusion while doing the labs in this course, they should be updated to reflect the new flow. It is demotivating and makes the learning experience difficult.

Vipul R. · Recensione inserita quasi 2 anni fa

Rafee M. · Recensione inserita quasi 2 anni fa

雄介 藤. · Recensione inserita quasi 2 anni fa

Job failed - but didnt return an error message

Cameron G. · Recensione inserita quasi 2 anni fa

Shubham A. · Recensione inserita quasi 2 anni fa

Mohammad S. · Recensione inserita quasi 2 anni fa

CHRISTIAN ALEJANDRO S. · Recensione inserita quasi 2 anni fa

Francisco V. · Recensione inserita quasi 2 anni fa

Binaya Kumar P. · Recensione inserita quasi 2 anni fa

Alexander B. · Recensione inserita quasi 2 anni fa

CHRISTIAN ALEJANDRO S. · Recensione inserita quasi 2 anni fa

Christoph C. · Recensione inserita quasi 2 anni fa