Inference Plugin Documentation

The Inference Plugin is a core component of the VIPR framework that provides a flexible, step-based workflow system for machine learning inference pipelines.

Overview

The Inference Plugin implements a generic, extensible workflow for ML/AI inference with the following characteristics:

• Domain-agnostic design: Designed to support diverse ML models and data types

• Step-based architecture: Six configurable steps that can be customized

• Hook and filter system: Extensibility points at each step

• Type-safe: Full type checking with beartype

• Configuration-driven: YAML-based configuration for all components

The plugin integrates with the VIPR Dynamic Hooks and Filters Extension, allowing hooks and filters to be configured via YAML files rather than requiring code changes. This provides maximum flexibility for adapting workflows to different use cases.

Data Flow

The inference workflow processes data through a standardized 6-step pipeline. Each step transforms data in a standardized way, ensuring type safety and extensibility.

Step Input/Output Contracts

LoadDataStep:
  Input:  config parameters (dict)
  Output: DataSet(x, y, errors, metadata)

LoadModelStep:  
  Input:  config parameters (dict)
  Output: model instance (Any)

NormalizeStep:
  Input:  DataSet (original)
  Output: DataSet (normalized with transformed errors via filters)
  Note:   Normalization is implemented via INFERENCE_NORMALIZE_PRE_FILTER.
          Filters can transform both data and errors.
          See LogNormalizer, MinMaxNormalizer, ZScoreNormalizer examples.

PreprocessStep:
  Input:  DataSet (model available via self.app.inference.model)
  Output: DataSet (preprocessed with transformed errors via filters)
  Note:   Preprocessing is implemented via INFERENCE_PREPROCESS_PRE_FILTER.
          Filters can transform both data and errors (e.g., reflectorch interpolation).

PredictionStep:
  Input:  DataSet (model available via self.app.inference.model)
  Output: prediction results (dict)

PostprocessStep:
  Input:  prediction results (dict)
  Output: final results (Any)

DataSet Transfer Object

The vipr.plugins.inference.dataset.DataSet class implements the Transfer Object Pattern for immutable data transport between pipeline steps.

Key features:

• Immutable: Arrays are read-only after creation

• Type-safe: Full Pydantic validation with beartype

• Functional updates: Use copy_with_updates() for modifications

See: vipr.plugins.inference.dataset.DataSet for complete API reference and usage examples.

Extension Points Per Step

Each step provides 6 extension points for customization:

AbstractInferenceStep.run():
├── runPrePreFilterHook()    ◄── Hook 1: Before any processing
├── runPreFilter()           ◄── Filter 1: Transform input  
├── runPostPreFilterHook()   ◄── Hook 2: After input filter
├── execute()                ◄── Core step logic
├── runPrePostFilterHook()   ◄── Hook 3: Before output filter
├── runPostFilter()          ◄── Filter 2: Transform output
└── runPostPostFilterHook()  ◄── Hook 4: After all processing

Architecture

Core Components

1. vipr.plugins.inference.base_inference.BaseInference
   Abstract base providing shared functionality for inference implementations

2. vipr.plugins.inference.inference.Inference
   Main orchestrator implementing the 6-step workflow with complete hook/filter system

3. vipr.plugins.inference.abstract_step.AbstractInferenceStep
   Base class for workflow steps with hooks/filters

4. Workflow Steps:

   • vipr.plugins.inference.steps.load_data_step.LoadDataInferenceStep - Loads input data

   • vipr.plugins.inference.steps.load_model_step.LoadModelInferenceStep - Loads ML models

   • vipr.plugins.inference.steps.normalize_step.NormalizeInferenceStep - Normalizes data

   • vipr.plugins.inference.steps.preprocess_step.PreprocessInferenceStep - Preprocesses data

   • vipr.plugins.inference.steps.prediction_step.PredictionInferenceStep - Executes prediction

   • vipr.plugins.inference.steps.postprocess_step.PostprocessInferenceStep - Post-processes results

Extension Points

The inference workflow uses two extension mechanisms:

Handler-Based Steps

Steps that use pluggable handler implementations:

Step	Handler Type	Interface	Purpose
LoadDataStep	data_loader	vipr.interfaces.data_loader.DataLoaderInterface	Load data from various sources
LoadModelStep	model_loader	vipr.interfaces.model_loader.ModelLoaderInterface	Load ML models
PredictionStep	predictor	vipr.interfaces.predictor.PredictorInterface	Execute predictions
PostprocessStep	postprocessor	vipr.interfaces.postprocessor.PostprocessorInterface	Process results

Filter-Based Steps

Steps that use filter transformations (no handlers):

Step	Primary Filter	Purpose
NormalizeStep	INFERENCE_NORMALIZE_PRE_FILTER	Normalize data and transform errors (dy/dx)
PreprocessStep	INFERENCE_PREPROCESS_PRE_FILTER	Preprocess data (e.g., interpolation with error transformation)

Note: Filter-based steps provide DataSet-aware transformation. See normalizer examples (LogNormalizer, MinMaxNormalizer, ZScoreNormalizer) and preprocessing filters (e.g., Reflectorch interpolation).

Handler Implementation

Base classes: vipr.handlers.data_loader.DataLoaderHandler, vipr.handlers.model_loader.ModelLoaderHandler, vipr.handlers.predictor.PredictorHandler

Pattern:

1. Inherit from appropriate base handler

2. Implement interface method (_load_data, _load_model, _predict)

3. Return correct type (vipr.plugins.inference.dataset.DataSet for data loaders)

Complete examples: See Reflectometry Handler Patterns

Hook and Filter System

The Inference Plugin provides a comprehensive system for extending functionality through hooks and filters. These can be registered either programmatically in your plugin code or through configuration using the Dynamic Hooks and Filters Extension.

Available Hooks

Global Workflow Hooks:
See vipr.plugins.inference.base_inference.BaseInference for hook constants:

• INFERENCE_START_HOOK: Triggered at workflow start

• INFERENCE_COMPLETE_HOOK: Triggered at workflow completion

Step-Level Hooks:
See vipr.plugins.inference.abstract_step.AbstractInferenceStep for all step hook constants.

Where <STEP> is: LOAD_DATA, LOAD_MODEL, NORMALIZE, PREPROCESS, PREDICTION, or POSTPROCESS

• INFERENCE_<STEP>_PRE_PRE_FILTER_HOOK - Before PRE_FILTER execution

• INFERENCE_<STEP>_POST_PRE_FILTER_HOOK - After PRE_FILTER, before execute()

• INFERENCE_<STEP>_PRE_POST_FILTER_HOOK - After execute(), before POST_FILTER

• INFERENCE_<STEP>_POST_POST_FILTER_HOOK - After POST_FILTER execution

Execution Order:

1. PRE_PRE_FILTER_HOOK     ← Hook before input transformation
2. PRE_FILTER              ← Filter transforms input
3. POST_PRE_FILTER_HOOK    ← Hook after input transformation
4. execute()               ← Core step logic
5. PRE_POST_FILTER_HOOK    ← Hook before output transformation
6. POST_FILTER             ← Filter transforms output
7. POST_POST_FILTER_HOOK   ← Hook after output transformation

Available Filters

Step-Level Filters (for each step):

• INFERENCE_<STEP>_PRE_FILTER: Transform input data before execute()

• INFERENCE_<STEP>_POST_FILTER: Transform output data after execute()

Registering Hooks and Filters

You can register hooks and filters in two ways:

1. Programmatically (in your plugin)

# In your plugin
def _post_setup(self, app):
    # Register a hook
    app.hook.register('INFERENCE_START_HOOK', self.on_inference_start)
    
    # Register a filter
    app.filter.register('INFERENCE_NORMALIZE_PRE_FILTER', self.custom_normalize)

def on_inference_start(self, app):
    app.log.info("Custom inference starting...")

def custom_normalize(self, data: DataSet, **kwargs) -> DataSet:
    # Custom normalization logic
    return data.copy_with_updates(
        x=normalized_x,
        y=normalized_y,
        metadata={**data.metadata, 'normalized': True}
    )

2. Via Configuration (using Dynamic Hooks and Filters Extension)

The recommended approach for maximum flexibility is to use the Dynamic Hooks and Filters Extension, which allows you to configure hooks and filters in your YAML file.

TODO: Dynamic Hooks and Filters Extension documentation will be migrated soon.

Configuration

Basic Configuration Structure

vipr:
  # Step configurations
  load_data:
    handler: csv_loader          # Handler to use
    parameters:                  # Handler-specific parameters
      file_path: data/input.csv
      delimiter: ','
  
  load_model:
    handler: pytorch_loader
    parameters:
      model_path: models/my_model.pt
      device: cuda
  
  normalize:
    handler: standard_normalizer
    parameters:
      method: zscore
  
  preprocess:
    handler: default_preprocessor
    parameters:
      remove_outliers: true
  
  prediction:
    handler: default_predictor
    parameters:
      batch_size: 32
  
  postprocess:
    handler: default_postprocessor
    parameters:
      format: json
  
  # Dynamic hooks and filters can be configured here
  # See Dynamic Hooks and Filters Extension documentation for details
  hooks: {}
  filters: {}

Dynamic Hook/Filter Configuration

The Inference Plugin integrates with the dynamic_hooks_filters extension for configuration-based hook and filter registration. This allows you to define hooks and filters in your YAML configuration instead of hardcoding them in plugin code.

Note: For detailed information about configuring dynamic hooks and filters, please refer to the Dynamic Hooks and Filters Extension Documentation.

Usage Example

Creating a Controller

from cement import Controller, ex

class MLController(Controller):
    class Meta:
        label = 'ml'
        stacked_on = 'base'
        stacked_type = 'nested'
    
    @ex(help='Run ML inference')
    def predict(self):
        # The inference workflow is already available as self.app.inference
        # (registered by the inference plugin during app setup)
        
        # Run inference workflow
        result = self.app.inference.run()
        
        # Access intermediate results if needed
        self.app.log.info(f"Model used: {self.app.inference.model}")
        self.app.log.info(f"Preprocessed data shape: {self.app.inference.preprocessed_data[0].shape}")
        
        # Return final result
        self.app.log.info(f"Prediction complete")
        return result

Running via CLI

# With configuration file
vipr --config config/ml_config.yaml ml predict