tfx.v1.types.BaseComponent.DRIVER_CLASS

BaseDriver is the base class of all custom drivers.

tfx.v1.types.BaseComponent.DRIVER_CLASS(
    metadata_handler: metadata.Metadata
)

This can also be used as the default driver of a component if no custom logic is needed.

_metadata_handler An instance of Metadata.

Methods

pre_execution

pre_execution(
    input_dict: Dict[str, tfx.v1.types.BaseChannel],
    output_dict: Dict[str, tfx.v1.dsl.Channel],
    exec_properties: Dict[str, Any],
    driver_args: data_types.DriverArgs,
    pipeline_info: data_types.PipelineInfo,
    component_info: data_types.ComponentInfo
) -> data_types.ExecutionDecision

Handle pre-execution logic.

There are four steps

  1. Fetches input artifacts from metadata and checks whether uri exists.
  2. Registers execution.
  3. Decides whether a new execution is needed. 4a. If (3), prepare output artifacts. 4b. If not (3), fetch cached output artifacts.

Args
input_dict key -> Channel for inputs.
output_dict key -> Channel for outputs. Uris of the outputs are not assigned.
exec_properties Dict of other execution properties.
driver_args An instance of data_types.DriverArgs class.
pipeline_info An instance of data_types.PipelineInfo, holding pipeline related properties including pipeline_name, pipeline_root and run_id
component_info An instance of data_types.ComponentInfo, holding component related properties including component_type and component_id.

Returns
data_types.ExecutionDecision object.

Raises
RuntimeError if any input as an empty uri.

resolve_exec_properties

resolve_exec_properties(
    exec_properties: Dict[str, Any],
    pipeline_info: data_types.PipelineInfo,
    component_info: data_types.ComponentInfo
) -> Dict[str, Any]

Resolve execution properties.

Subclasses might override this function for customized execution properties resolution logic.

Args
exec_properties Original execution properties passed in.
pipeline_info An instance of data_types.PipelineInfo, holding pipeline related properties including pipeline_name, pipeline_root and run_id
component_info An instance of data_types.ComponentInfo, holding component related properties including component_type and component_id.

Returns
Final execution properties that will be used in execution.

resolve_input_artifacts

resolve_input_artifacts(
    input_dict: Dict[str, tfx.v1.types.BaseChannel],
    exec_properties: Dict[str, Any],
    driver_args: data_types.DriverArgs,
    pipeline_info: data_types.PipelineInfo
) -> Dict[str, List[types.Artifact]]

Resolve input artifacts from metadata.

Subclasses might override this function for customized artifact properties resolution logic. However please note that this function is supposed to be called in normal cases (except head of the pipeline) since it handles artifact info passing from upstream components.

Args
input_dict key -> Channel mapping for inputs generated in logical pipeline.
exec_properties Dict of other execution properties, e.g., configs.
driver_args An instance of data_types.DriverArgs with driver configuration properties.
pipeline_info An instance of data_types.PipelineInfo, holding pipeline related properties including component_type and component_id.

Returns
Final artifacts that will be used in execution.

Raises
ValueError if in interactive mode, the given input channels have not been resolved.

verify_input_artifacts

verify_input_artifacts(
    artifacts_dict: Dict[str, List[types.Artifact]]
) -> None

Verify that all artifacts have existing uri.

Args
artifacts_dict key -> types.Artifact for inputs.

Raises
RuntimeError if any input as an empty or non-existing uri.