kempnerforge.metrics.mlflow.publish

Append eval metrics to the MLflow run a checkpoint was trained under.

A checkpoint’s metadata.json records the tracking backends that were active when it was saved. publish_metrics reads the MLflow entry and appends the metrics to that run at the checkpoint’s step in a single log_batch request, linked to the checkpoint’s logged model when the entry names one. Inputs are fully validated before anything is sent, but MLflow’s log_batch permits partial writes, so a request that fails server side may still have stored some points. The publisher never starts, ends, or edits a run, never writes the sidecar, and keeps no state of its own, so several publishers can write to one run at once and the tracking server arbitrates. The call blocks on the server under MLflow’s own HTTP timeout and retry settings, and authentication is whatever the MLflow client finds in the environment. It is an offline tool: call it from one process, never from the training step.

Command line: python -m kempnerforge.metrics.mlflow.publish <checkpoint_dir> --metrics <json>.

Functions

main([argv])	Command-line entry point; returns the exit code: 0 published, 2 refused, 1 failed.
publish_metrics(checkpoint_dir, metrics[, ...])	Append metrics to the MLflow run a checkpoint was trained under.
read_mlflow_target(checkpoint_dir)	Read the MLflow run a checkpoint was trained under from its metadata.json.

Classes

MlflowTarget

The MLflow run a checkpoint was trained under, as its sidecar records it.

Exceptions

PublishRefused

Nothing was published: the sidecar, the metrics, or the environment is unusable.

exception kempnerforge.metrics.mlflow.publish.PublishRefused

Bases: ValueError

Nothing was published: the sidecar, the metrics, or the environment is unusable.

class kempnerforge.metrics.mlflow.publish.MlflowTarget

Bases: object

The MLflow run a checkpoint was trained under, as its sidecar records it.

run_id: str

experiment: str

tracking_uri: str

step: int

model_id: str | None = None

model_name: str | None = None

__init__(run_id, experiment, tracking_uri, step, model_id=None, model_name=None)

Parameters:

• run_id (str)

• experiment (str)

• tracking_uri (str)

• step (int)

• model_id (str | None)

• model_name (str | None)

Return type:

None

kempnerforge.metrics.mlflow.publish.read_mlflow_target(checkpoint_dir)

Read the MLflow run a checkpoint was trained under from its metadata.json.

The sidecar needs a non-negative integer step and a tracking list holding exactly one entry with backend == "mlflow" whose run_id, experiment and tracking_uri are non-empty strings. model_id and model_name are optional, and non-empty strings when present. A key repeated anywhere in the file is refused, since the last value would silently win. For a Databricks URI it also prepares auth in os.environ for the whole process, the way the training backend does, so artifact uploads made later in the same process authenticate like the tracking client.

Raises:

PublishRefused – Naming each missing or malformed item; nothing is inferred.

Parameters:

checkpoint_dir (str | PathLike[str])

Return type:

MlflowTarget

kempnerforge.metrics.mlflow.publish.publish_metrics(checkpoint_dir, metrics, prefix='eval/')

Append metrics to the MLflow run a checkpoint was trained under.

Every metric is logged as <prefix><name> at the checkpoint’s step by one MlflowClient.log_batch request, on a client bound to the sidecar’s tracking URI (the process-wide MLflow tracking URI is left alone). When the installed MLflow can link metrics to a logged model, they are linked to the sidecar entry’s model_id; without one, to the run’s model named by its model_name that find_or_create_checkpoint_model settles on (finishing it if PENDING), which is created (an external model of this checkpoint directory) when the run has none. Otherwise, or when the installed MLflow cannot link, the metrics are published unlinked.

Parameters:

• checkpoint_dir (str | PathLike[str]) – Checkpoint step directory holding metadata.json.

• metrics (Mapping[str, float]) – Metric name to finite number, at most 1000 of them (one request).

• prefix (str) – Prepended verbatim to every name; may be empty.

Returns:

The target the metrics were appended to, with the model_id they were linked to (None when unlinked).

Raises:

PublishRefused – The sidecar or the metrics are missing or malformed, mlflow is not installed, or no model can be linked (the sidecar entry gives none, or the installed MLflow cannot link one) while this process has an active MLflow model (which log_batch would silently link); nothing was sent. Errors raised by MLflow itself (an invalid metric name, an unreachable server) propagate as raised; a request that fails server side may still have stored some of its points.

Return type:

MlflowTarget

kempnerforge.metrics.mlflow.publish.main(argv=None)

Command-line entry point; returns the exit code: 0 published, 2 refused, 1 failed.

Parameters:

argv (Sequence[str] | None)

Return type:

int