Performance Monitoring Plugin¶

Automatic execution time tracking for inference workflows.

Overview¶

Enabled by Default: Tracks execution time for all inference workflows with zero configuration. Provides timing information in logs and can store detailed metrics in result metadata when the API/DataCollector plugin is active.

How It Works¶

The plugin registers hooks at workflow start and completion:

INFERENCE_START_HOOK   # Records start timestamp
INFERENCE_COMPLETE_HOOK # Calculates and stores execution time

Process:

Records timestamp when inference begins
Calculates total execution time at completion
Stores timing data in app.datacollector.data.batch_metadata['performance'] (if app.datacollector exists)
Logs human-readable timing information

Usage¶

Automatic Logging¶

Performance metrics are automatically logged for every inference:

vipr --config config.yaml inference run

Console Output:

[INFO] Inference workflow started at: 2025-11-21 15:30:45.123456
[INFO] Inference execution time: 2.34 seconds

Accessing Timing Data¶

When the API/DataCollector plugin is active, performance data is saved to the result pickle in batch_metadata:

import pickle

with open('storage/results/<result_id>/data.pkl', 'rb') as f:
    result_data = pickle.load(f)
    
performance = result_data['batch_metadata']['performance']
print(f"Execution time: {performance['execution_time_formatted']}")
print(f"Started: {performance['start_datetime']}")

Available Fields¶

The batch_metadata['performance'] dictionary contains:

Field	Type	Description	Example
`execution_time`	`float`	Total time in seconds	`2.34`
`execution_time_formatted`	`str`	Human-readable time	`"2.34 seconds"`
`start_time`	`float`	Unix timestamp (start)	`1700581845.123`
`end_time`	`float`	Unix timestamp (end)	`1700581847.463`
`start_datetime`	`str`	ISO format start	`"2025-11-21T15:30:45.123456"`
`end_datetime`	`str`	ISO format end	`"2025-11-21T15:30:47.463456"`

Measurement Scope¶

Includes all 5 inference steps (Load Data -> Load Model -> Preprocess -> Predict -> Postprocess)
Includes hook/filter execution time
Uses Python’s time.time() (effective resolution depends on platform/runtime)

Configuration¶

The plugin is enabled by default. To disable it, set the plugin flag in a VIPR configuration file that is loaded by the application.

Example:

plugin.performance:
  enabled: false

This overrides the default setting from vipr-core/vipr/config/plugins.yaml.