关于“[Deprecated] TFX Metadata”的评价

Prasuna M. · 评论over 1 year之前

Roman T. · 评论over 1 year之前

J H. · 评论over 1 year之前

Hadrián P. · 评论over 1 year之前

Scot J. · 评论over 1 year之前

Ada A. · 评论over 1 year之前

Yomna M. · 评论over 1 year之前

Ezra J. · 评论over 1 year之前

Vlad L. · 评论over 1 year之前

2024-02-27 11:25:35.208831: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/lib 2024-02-27 11:25:35.208907: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. INFO:absl:tensorflow_ranking is not available: No module named 'tensorflow_ranking' INFO:absl:tensorflow_text is not available: No module named 'tensorflow_text' INFO:absl:Running driver for Tuner INFO:absl:MetadataStore with gRPC connection initialized INFO:absl:Adding KFP pod name tfx-covertype-lab-04-nxkkw-2692208594 to execution INFO:absl:Running executor for Tuner INFO:absl:Attempting to infer TFX Python dependency for beam INFO:absl:Copying all content from install dir /tfx-src/tfx to temp dir /tmp/tmp4c8gwt61/build/tfx INFO:absl:Generating a temp setup file at /tmp/tmp4c8gwt61/build/tfx/setup.py INFO:absl:Creating temporary sdist package, logs available at /tmp/tmp4c8gwt61/build/tfx/setup.log INFO:absl:Added --extra_package=/tmp/tmp4c8gwt61/build/tfx/dist/tfx_ephemeral-0.25.0.tar.gz to beam args WARNING:absl:workerCount is overridden with 2 INFO:absl:json_inputs='{"examples": [{"artifact": {"id": "21", "type_id": "18", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transformed_examples/21", "properties": {"split_names": {"string_value": "[\"train\", \"eval\"]"}}, "custom_properties": {"name": {"string_value": "transformed_examples"}, "producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}}, "state": "LIVE", "create_time_since_epoch": "1709033025920", "last_update_time_since_epoch": "1709033119881"}, "artifact_type": {"id": "18", "name": "Examples", "properties": {"split_names": "STRING", "span": "INT", "version": "INT"}}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "Examples"}], "transform_graph": [{"artifact": {"id": "20", "type_id": "22", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transform_graph/21", "custom_properties": {"producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}, "name": {"string_value": "transform_graph"}}, "state": "LIVE", "create_time_since_epoch": "1709033025917", "last_update_time_since_epoch": "1709033119878"}, "artifact_type": {"id": "22", "name": "TransformGraph"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "TransformGraph"}]}'. INFO:absl:json_outputs='{"best_hyperparameters": [{"artifact": {"id": "25", "type_id": "28", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Tuner/best_hyperparameters/24", "custom_properties": {"name": {"string_value": "best_hyperparameters"}, "producer_component": {"string_value": "Tuner"}}}, "artifact_type": {"id": "28", "name": "HyperParameters"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "HyperParameters"}]}'. INFO:absl:json_exec_properties='{"custom_config": "{\"ai_platform_training_args\": {\"masterConfig\": {\"imageUri\": \"gcr.io/qwiklabs-gcp-01-272c395e4a02/tfx_covertype_lab_04@sha256:276d6670faaf7a4939cc0b991101cad0e92bb3ad870973a3018474337aaa1647\"}, \"project\": \"qwiklabs-gcp-01-272c395e4a02\", \"region\": \"us-central1\", \"serviceAccount\": \"tfx-tuner-caip-service-account@qwiklabs-gcp-01-272c395e4a02.iam.gserviceaccount.com\"}}", "eval_args": "{\"num_steps\": 500}", "kfp_pod_name": "tfx-covertype-lab-04-nxkkw-2692208594", "module_file": "model.py", "train_args": "{\"num_steps\": 5000}", "tune_args": "{\n \"num_parallel_trials\": 3\n}", "tuner_fn": null}'. WARNING:googleapiclient.discovery_cache:file_cache is unavailable when using oauth2client >= 4.0.0 or google-auth Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/googleapiclient/discovery_cache/__init__.py", line 36, in autodetect from google.appengine.api import memcache ModuleNotFoundError: No module named 'google.appengine' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/googleapiclient/discovery_cache/file_cache.py", line 33, in from oauth2client.contrib.locked_file import LockedFile ModuleNotFoundError: No module named 'oauth2client.contrib.locked_file' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/googleapiclient/discovery_cache/file_cache.py", line 37, in from oauth2client.locked_file import LockedFile ModuleNotFoundError: No module named 'oauth2client.locked_file' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/googleapiclient/discovery_cache/__init__.py", line 42, in autodetect from . import file_cache File "/usr/local/lib/python3.7/dist-packages/googleapiclient/discovery_cache/file_cache.py", line 41, in "file_cache is unavailable when using oauth2client >= 4.0.0 or google-auth" ImportError: file_cache is unavailable when using oauth2client >= 4.0.0 or google-auth INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/discovery/v1/apis/ml/v1/rest INFO:absl:TrainingInput={'masterConfig': {'imageUri': 'gcr.io/qwiklabs-gcp-01-272c395e4a02/tfx_covertype_lab_04@sha256:276d6670faaf7a4939cc0b991101cad0e92bb3ad870973a3018474337aaa1647', 'containerCommand': ['python', '-m', 'tfx.scripts.run_executor', '--executor_class_path', 'tfx.extensions.google_cloud_ai_platform.tuner.executor._WorkerExecutor', '--inputs', '{"examples": [{"artifact": {"id": "21", "type_id": "18", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transformed_examples/21", "properties": {"split_names": {"string_value": "[\\"train\\", \\"eval\\"]"}}, "custom_properties": {"name": {"string_value": "transformed_examples"}, "producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}}, "state": "LIVE", "create_time_since_epoch": "1709033025920", "last_update_time_since_epoch": "1709033119881"}, "artifact_type": {"id": "18", "name": "Examples", "properties": {"split_names": "STRING", "span": "INT", "version": "INT"}}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "Examples"}], "transform_graph": [{"artifact": {"id": "20", "type_id": "22", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transform_graph/21", "custom_properties": {"producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}, "name": {"string_value": "transform_graph"}}, "state": "LIVE", "create_time_since_epoch": "1709033025917", "last_update_time_since_epoch": "1709033119878"}, "artifact_type": {"id": "22", "name": "TransformGraph"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "TransformGraph"}]}', '--outputs', '{"best_hyperparameters": [{"artifact": {"id": "25", "type_id": "28", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Tuner/best_hyperparameters/24", "custom_properties": {"name": {"string_value": "best_hyperparameters"}, "producer_component": {"string_value": "Tuner"}}}, "artifact_type": {"id": "28", "name": "HyperParameters"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "HyperParameters"}]}', '--exec-properties', '{"custom_config": "{\\"ai_platform_training_args\\": {\\"masterConfig\\": {\\"imageUri\\": \\"gcr.io/qwiklabs-gcp-01-272c395e4a02/tfx_covertype_lab_04@sha256:276d6670faaf7a4939cc0b991101cad0e92bb3ad870973a3018474337aaa1647\\"}, \\"project\\": \\"qwiklabs-gcp-01-272c395e4a02\\", \\"region\\": \\"us-central1\\", \\"serviceAccount\\": \\"tfx-tuner-caip-service-account@qwiklabs-gcp-01-272c395e4a02.iam.gserviceaccount.com\\"}}", "eval_args": "{\\"num_steps\\": 500}", "kfp_pod_name": "tfx-covertype-lab-04-nxkkw-2692208594", "module_file": "model.py", "train_args": "{\\"num_steps\\": 5000}", "tune_args": "{\\n \\"num_parallel_trials\\": 3\\n}", "tuner_fn": null}']}, 'region': 'us-central1', 'serviceAccount': 'tfx-tuner-caip-service-account@qwiklabs-gcp-01-272c395e4a02.iam.gserviceaccount.com', 'workerCount': 2, 'scaleTier': 'CUSTOM', 'masterType': 'standard', 'workerType': 'standard'} INFO:absl:Submitting job='tfx_tuner_20240227112545', project='qwiklabs-gcp-01-272c395e4a02' to AI Platform. INFO:googleapiclient.discovery:URL being requested: POST https://ml.googleapis.com/v1/projects/qwiklabs-gcp-01-272c395e4a02/jobs?alt=json INFO:googleapiclient.discovery:URL being requested: GET https://ml.googleapis.com/v1/projects/qwiklabs-gcp-01-272c395e4a02/jobs/tfx_tuner_20240227112545?alt=json ERROR:absl:Job 'projects/qwiklabs-gcp-01-272c395e4a02/jobs/tfx_tuner_20240227112545' did not succeed. Detailed response {'jobId': 'tfx_tuner_20240227112545', 'trainingInput': {'scaleTier': 'CUSTOM', 'masterType': 'standard', 'workerType': 'standard', 'workerCount': '2', 'region': 'us-central1', 'masterConfig': {'imageUri': 'gcr.io/qwiklabs-gcp-01-272c395e4a02/tfx_covertype_lab_04@sha256:276d6670faaf7a4939cc0b991101cad0e92bb3ad870973a3018474337aaa1647', 'containerCommand': ['python', '-m', 'tfx.scripts.run_executor', '--executor_class_path', 'tfx.extensions.google_cloud_ai_platform.tuner.executor._WorkerExecutor', '--inputs', '{"examples": [{"artifact": {"id": "21", "type_id": "18", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transformed_examples/21", "properties": {"split_names": {"string_value": "[\\"train\\", \\"eval\\"]"}}, "custom_properties": {"name": {"string_value": "transformed_examples"}, "producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}}, "state": "LIVE", "create_time_since_epoch": "1709033025920", "last_update_time_since_epoch": "1709033119881"}, "artifact_type": {"id": "18", "name": "Examples", "properties": {"split_names": "STRING", "span": "INT", "version": "INT"}}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "Examples"}], "transform_graph": [{"artifact": {"id": "20", "type_id": "22", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transform_graph/21", "custom_properties": {"producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}, "name": {"string_value": "transform_graph"}}, "state": "LIVE", "create_time_since_epoch": "1709033025917", "last_update_time_since_epoch": "1709033119878"}, "artifact_type": {"id": "22", "name": "TransformGraph"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "TransformGraph"}]}', '--outputs', '{"best_hyperparameters": [{"artifact": {"id": "25", "type_id": "28", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Tuner/best_hyperparameters/24", "custom_properties": {"name": {"string_value": "best_hyperparameters"}, "producer_component": {"string_value": "Tuner"}}}, "artifact_type": {"id": "28", "name": "HyperParameters"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "HyperParameters"}]}', '--exec-properties', '{"custom_config": "{\\"ai_platform_training_args\\": {\\"masterConfig\\": {\\"imageUri\\": \\"gcr.io/qwiklabs-gcp-01-272c395e4a02/tfx_covertype_lab_04@sha256:276d6670faaf7a4939cc0b991101cad0e92bb3ad870973a3018474337aaa1647\\"}, \\"project\\": \\"qwiklabs-gcp-01-272c395e4a02\\", \\"region\\": \\"us-central1\\", \\"serviceAccount\\": \\"tfx-tuner-caip-service-account@qwiklabs-gcp-01-272c395e4a02.iam.gserviceaccount.com\\"}}", "eval_args": "{\\"num_steps\\": 500}", "kfp_pod_name": "tfx-covertype-lab-04-nxkkw-2692208594", "module_file": "model.py", "train_args": "{\\"num_steps\\": 5000}", "tune_args": "{\\n \\"num_parallel_trials\\": 3\\n}", "tuner_fn": null}']}, 'serviceAccount': 'tfx-tuner-caip-service-account@qwiklabs-gcp-01-272c395e4a02.iam.gserviceaccount.com'}, 'createTime': '2024-02-27T11:25:45Z', 'startTime': '2024-02-27T11:26:00Z', 'endTime': '2024-02-27T11:26:32Z', 'state': 'FAILED', 'errorMessage': 'The replica master 0 exited with a non-zero status of 1. Termination reason: Error. The replica worker 0 exited with a non-zero status of 1. Termination reason: Error. The replica worker 1 exited with a non-zero status of 1. Termination reason: Error. To find out more about why your job exited please check the logs: https://console.cloud.google.com/logs/viewer?project=999903194638&resource=ml_job%2Fjob_id%2Ftfx_tuner_20240227112545&advancedFilter=resource.type%3D%22ml_job%22%0Aresource.labels.job_id%3D%22tfx_tuner_20240227112545%22', 'trainingOutput': {'consumedMLUnits': 0.01}, 'labels': {'tfx_executor': 'ensions-google_cloud_ai_platform-tuner-executor-_workerexecutor', 'tfx_py_version': '3-7', 'tfx_runner': 'kfp', 'tfx_version': '0-25-0'}, 'etag': 'e43TZkKQZ4E=', 'jobPosition': '0'}. Traceback (most recent call last): File "/tfx-src/tfx/orchestration/kubeflow/container_entrypoint.py", line 360, in main() File "/tfx-src/tfx/orchestration/kubeflow/container_entrypoint.py", line 353, in main execution_info = launcher.launch() File "/tfx-src/tfx/orchestration/launcher/base_component_launcher.py", line 209, in launch copy.deepcopy(execution_decision.exec_properties)) File "/tfx-src/tfx/orchestration/launcher/in_process_component_launcher.py", line 72, in _run_executor copy.deepcopy(input_dict), output_dict, copy.deepcopy(exec_properties)) File "/tfx-src/tfx/extensions/google_cloud_ai_platform/tuner/executor.py", line 121, in Do job_id) File "/tfx-src/tfx/extensions/google_cloud_ai_platform/runner.py", line 305, in start_aip_training job_labels=job_labels) File "/tfx-src/tfx/extensions/google_cloud_ai_platform/runner.py", line 194, in _launch_aip_training raise RuntimeError(err_msg) RuntimeError: Job 'projects/qwiklabs-gcp-01-272c395e4a02/jobs/tfx_tuner_20240227112545' did not succeed. Detailed response {'jobId': 'tfx_tuner_20240227112545', 'trainingInput': {'scaleTier': 'CUSTOM', 'masterType': 'standard', 'workerType': 'standard', 'workerCount': '2', 'region': 'us-central1', 'masterConfig': {'imageUri': 'gcr.io/qwiklabs-gcp-01-272c395e4a02/tfx_covertype_lab_04@sha256:276d6670faaf7a4939cc0b991101cad0e92bb3ad870973a3018474337aaa1647', 'containerCommand': ['python', '-m', 'tfx.scripts.run_executor', '--executor_class_path', 'tfx.extensions.google_cloud_ai_platform.tuner.executor._WorkerExecutor', '--inputs', '{"examples": [{"artifact": {"id": "21", "type_id": "18", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transformed_examples/21", "properties": {"split_names": {"string_value": "[\\"train\\", \\"eval\\"]"}}, "custom_properties": {"name": {"string_value": "transformed_examples"}, "producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}}, "state": "LIVE", "create_time_since_epoch": "1709033025920", "last_update_time_since_epoch": "1709033119881"}, "artifact_type": {"id": "18", "name": "Examples", "properties": {"split_names": "STRING", "span": "INT", "version": "INT"}}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "Examples"}], "transform_graph": [{"artifact": {"id": "20", "type_id": "22", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Transform/transform_graph/21", "custom_properties": {"producer_component": {"string_value": "Transform"}, "state": {"string_value": "published"}, "name": {"string_value": "transform_graph"}}, "state": "LIVE", "create_time_since_epoch": "1709033025917", "last_update_time_since_epoch": "1709033119878"}, "artifact_type": {"id": "22", "name": "TransformGraph"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "TransformGraph"}]}', '--outputs', '{"best_hyperparameters": [{"artifact": {"id": "25", "type_id": "28", "uri": "gs://qwiklabs-gcp-01-272c395e4a02-kubeflowpipelines-default/tfx_covertype_lab_04/b2da8988-ae20-4336-b3c8-6b91ae1315af/Tuner/best_hyperparameters/24", "custom_properties": {"name": {"string_value": "best_hyperparameters"}, "producer_component": {"string_value": "Tuner"}}}, "artifact_type": {"id": "28", "name": "HyperParameters"}, "__artifact_class_module__": "tfx.types.standard_artifacts", "__artifact_class_name__": "HyperParameters"}]}', '--exec-properties', '{"custom_config": "{\\"ai_platform_training_args\\": {\\"masterConfig\\": {\\"imageUri\\": \\"gcr.io/qwiklabs-gcp-01-272c395e4a02/tfx_covertype_lab_04@sha256:276d6670faaf7a4939cc0b991101cad0e92bb3ad870973a3018474337aaa1647\\"}, \\"project\\": \\"qwiklabs-gcp-01-272c395e4a02\\", \\"region\\": \\"us-central1\\", \\"serviceAccount\\": \\"tfx-tuner-caip-service-account@qwiklabs-gcp-01-272c395e4a02.iam.gserviceaccount.com\\"}}", "eval_args": "{\\"num_steps\\": 500}", "kfp_pod_name": "tfx-covertype-lab-04-nxkkw-2692208594", "module_file": "model.py", "train_args": "{\\"num_steps\\": 5000}", "tune_args": "{\\n \\"num_parallel_trials\\": 3\\n}", "tuner_fn": null}']}, 'serviceAccount': 'tfx-tuner-caip-service-account@qwiklabs-gcp-01-272c395e4a02.iam.gserviceaccount.com'}, 'createTime': '2024-02-27T11:25:45Z', 'startTime': '2024-02-27T11:26:00Z', 'endTime': '2024-02-27T11:26:32Z', 'state': 'FAILED', 'errorMessage': 'The replica master 0 exited with a non-zero status of 1. Termination reason: Error. The replica worker 0 exited with a non-zero status of 1. Termination reason: Error. The replica worker 1 exited with a non-zero status of 1. Termination reason: Error. To find out more about why your job exited please check the logs: https://console.cloud.google.com/logs/viewer?project=999903194638&resource=ml_job%2Fjob_id%2Ftfx_tuner_20240227112545&advancedFilter=resource.type%3D%22ml_job%22%0Aresource.labels.job_id%3D%22tfx_tuner_20240227112545%22', 'trainingOutput': {'consumedMLUnits': 0.01}, 'labels': {'tfx_executor': 'ensions-google_cloud_ai_platform-tuner-executor-_workerexecutor', 'tfx_py_version': '3-7', 'tfx_runner': 'kfp', 'tfx_version': '0-25-0'}, 'etag': 'e43TZkKQZ4E=', 'jobPosition': '0'}.

Dmytro Y. · 评论over 1 year之前

siong n. · 评论over 1 year之前

VU T. · 评论over 1 year之前

Szymon T. · 评论over 1 year之前

The AI PLatform no longer exists and the setting up KubeFlow pipeline instance is not an option anymore. This causes confusion while doing the labs in this course, they should be updated to reflect the new flow. It is demotivating and makes the learning experience difficult.

Vipul R. · 评论almost 2 years之前

Rafee M. · 评论almost 2 years之前

雄介 藤. · 评论almost 2 years之前

Job failed - but didnt return an error message

Cameron G. · 评论almost 2 years之前

Shubham A. · 评论almost 2 years之前

Mohammad S. · 评论almost 2 years之前

CHRISTIAN ALEJANDRO S. · 评论almost 2 years之前

Francisco V. · 评论almost 2 years之前

Binaya Kumar P. · 评论almost 2 years之前

Alexander B. · 评论almost 2 years之前

CHRISTIAN ALEJANDRO S. · 评论almost 2 years之前

Christoph C. · 评论almost 2 years之前