Discovery Plugin Documentation

The Discovery Plugin provides a decorator-based registry system for all VIPR components, enabling automatic discovery and introspection of available handlers, hooks, and filters.

Overview

The Discovery Plugin serves as the central registry for:

• Hooks & Filters: Readonly (hooks) and transformative (filters) callbacks for workflow extensions

• Data Loaders: Components for loading input data

• Model Loaders: Components for loading ML models

• Predictors: Components for executing predictions

• Postprocessors: Components for post-processing results

Key Features:

• Decorator-based registration: Simple @discover_* decorators

• CLI introspection: Query available components via command line

• API-ready commands: Controller methods exposable as FastAPI endpoints via @api decorator

• Plugin filtering: Filter components by plugin name

• Automatic discovery: Components self-register on import

Architecture

Registry System

The plugin maintains separate registries for each component type:

# Central registries (in decorators.py)
hooks_discovery: dict[str, list[dict]]
filters_discovery: dict[str, list[dict]]
data_loaders_discovery: dict[str, dict]
model_loaders_discovery: dict[str, dict]
predictors_discovery: dict[str, dict]
postprocessors_discovery: dict[str, dict]

Registration Flow

1. Plugin loads and imports modules
2. Decorators execute on import (@discover_filter, @discover_data_loader, etc.)
3. Components register in central registries (hooks_discovery, filters_discovery, etc.)
4. Registries ready for introspection via CLI/API and programmatic access

Available Decorators

Hook & Filter Registration

Decorator	Purpose	Registry
@discover_hook()	Register workflow hooks	hooks_discovery
@discover_filter()	Register workflow filters	filters_discovery

Parameters:

• hook_type / filter_type: Hook/filter name (e.g., INFERENCE_NORMALIZE_PRE_FILTER)

• weight: Execution priority (default: 0, lower runs first)

• enabled_in_config: Enable in generated configs (default: True)

• parameters: Optional Pydantic model class (via parameters=MyParams) for validation/defaults

Handler Registration

Decorator	Purpose	Registry
@discover_data_loader()	Register data loaders	data_loaders_discovery
@discover_model_loader()	Register model loaders	model_loaders_discovery
@discover_predictor()	Register predictors	predictors_discovery
@discover_postprocessor()	Register postprocessors	postprocessors_discovery

Parameters:

• handler_name: Handler identifier (e.g., csv_spectrareader)

• parameters: Optional Pydantic model class for validation/default extraction

Usage Examples

Registering Hooks/Filters

Hook and filter callbacks are attached to these extension points via YAML configuration, which is passed to the CLI workflow command (e.g., vipr --config ./config.yaml inference run). See Dynamic Hooks and Filters Extension for details on registering callbacks.

Example of decorator registration:

from vipr.plugins.discovery.decorators import discover_filter

class LogNormalizer:
    @discover_filter('INFERENCE_NORMALIZE_PRE_FILTER', enabled_in_config=False)
    def normalize_filter(self, data, **kwargs):
        return data

Registering Data Loader

Real example from Reflectometry Plugin:

from vipr.handlers.data_loader import DataLoaderHandler
from vipr.plugins.discovery.decorators import discover_data_loader
from vipr.plugins.inference.dataset import DataSet
from pydantic import BaseModel, Field

class CSVSpectraReaderParams(BaseModel):
    data_path: str = Field(..., description="Path to single CSV/DAT/TXT file")
    column_mapping: dict[str, int] = Field(
        default_factory=lambda: {"q": 0, "I": 1},
        description="Column mapping for CSV data",
    )

@discover_data_loader('csv_spectrareader', parameters=CSVSpectraReaderParams)
class CSVSpectraReaderDataLoader(DataLoaderHandler):
    """CSV/DAT/TXT data loader for reflectometry data."""
    
    class Meta:
        label = 'csv_spectrareader'
    
    def _load_data(self, data_path: str, column_mapping: dict = None, **kwargs) -> DataSet:
        # Load CSV using SpectraReader
        # Returns DataSet with batch_size=1
        ...

Registering Model Loader

Real example from Reflectometry Plugin:

from vipr.handlers.model_loader import ModelLoaderHandler
from vipr.plugins.discovery.decorators import discover_model_loader
from typing import Optional
from pydantic import BaseModel, ConfigDict, Field

from vipr_reflectometry.paths import get_model_family_root
from vipr_reflectometry.reflectorch.load_model.timed_inference_model import TimedInferenceModel

class ReflectorchModelLoaderParams(BaseModel):
    """Parameters for the reflectorch model loader."""

    model_config = ConfigDict(extra='forbid')

    config_name: str = Field(default='b_mc_point_xray_conv_standard_L2_InputQ')
    model_name: Optional[str] = Field(default=None)
    root_dir: Optional[str] = Field(default=None)
    weights_format: str = Field(default='safetensors')
    repo_id: str = Field(default='valentinsingularity/reflectivity')
    device: str = Field(default='cpu')

@discover_model_loader('reflectorch', ReflectorchModelLoaderParams)
class ReflectorchModelLoader(ModelLoaderHandler):
    class Meta:
        label = 'reflectorch'
        
    def _load_model(self, **kwargs):
        """
        Loads a reflectorch model.
        
        Args:
            model_type: Type of reflectorch model ('reflectorch' expected)
            **kwargs: Model parameters such as config_name, device, etc.
            
        Returns:
            Loaded reflectorch model
        """
        params = ReflectorchModelLoaderParams.model_validate(kwargs)
        config_name = params.config_name
        device = params.device

        root_dir = get_model_family_root('reflectorch')
        if root_dir is None:
            raise ValueError("REFLECTOMETRY_ROOT_DIR environment variable must be set")
        
        # Load and return the model with timing measurements
        model = TimedInferenceModel(
            config_name=config_name,
            root_dir=str(root_dir),
            device=device
        )
        return model

CLI Commands

Query Components Summary

# All components
vipr discovery plugins

# Filter by plugin
vipr discovery plugins --plugin normalizers

Output:

{
  "components_summary": {
    "model_loaders": 2,
    "data_loaders": 3,
    "hooks": 15,
    "filters": 12,
    "predictors": 1,
    "postprocessors": 0
  }
}

Query Specific Component Types

# List all model loaders
vipr discovery model-loaders

# List data loaders for specific plugin
vipr discovery data-loaders --plugin reflectometry

# List all hooks
vipr discovery hooks

# List hooks of specific type
vipr discovery hooks --hook-type INFERENCE_POSTPROCESS_PRE_PRE_FILTER_HOOK

# List hooks for specific plugin
vipr discovery hooks --plugin reflectometry

# List all filters
vipr discovery filters

# List filters of specific type
vipr discovery filters --filter-type INFERENCE_NORMALIZE_PRE_FILTER

# List predictors
vipr discovery predictors

Query All Components

# Complete registry dump
vipr discovery components

# Filter by plugin
vipr discovery components --plugin reflectometry

Query Available Types

# List all hook types (inference workflow)
vipr discovery hook-types

# List all filter types (inference workflow)
vipr discovery filter-types

API Integration

All discovery commands are available as HTTP endpoints via the API plugin.

Endpoints

Endpoint	Method	Description
/api/discovery/plugins	GET	Components summary
/api/discovery/model-loaders	GET	List model loaders
/api/discovery/data-loaders	GET	List data loaders
/api/discovery/hooks	GET	List hooks
/api/discovery/filters	GET	List filters
/api/discovery/predictors	GET	List predictors
/api/discovery/postprocessors	GET	List postprocessors
/api/discovery/components	GET	All components
/api/discovery/hook-types	GET	Available hook types
/api/discovery/filter-types	GET	Available filter types
/api/discovery/example-configs	GET	List usable packaged example configs
/api/discovery/example-config	GET	Get one packaged example config by id

Example Config Discovery

Discovery also exposes packaged example configurations for plugins.

Unlike handlers, hooks, and filters, example configs are not stored in the decorator registries. Instead, VIPR scans installed plugin packages for packaged YAML files under:

<plugin_package>/<group>/examples/configs/*.yaml

or

<plugin_package>/<group>/examples/configs/*.yml

For each matching file:

• plugin comes from the plugin entry point name

• group comes from the path segment immediately before examples/configs

• id is built as <plugin>.<group>.<file_stem>

Only usable examples are returned. The referenced handlers, predictors, hooks, and filters must be available in the current runtime. Callbacks with enabled: false are ignored during availability checks.

See Developing VIPR Plugins for the packaging convention and requirements.

Example API Call

# Get all model loaders
curl http://localhost:8000/api/discovery/model-loaders

# Get hooks for specific plugin
curl "http://localhost:8000/api/discovery/hooks?plugin=normalizers"

# Get example configs for one plugin
curl "http://localhost:8000/api/discovery/example-configs?plugin=reflectometry"

# Get one packaged example config by id
curl "http://localhost:8000/api/discovery/example-config?id=reflectometry.reflectorch.FePt_DN"

Plugin Filtering

The Discovery system supports filtering by plugin name for both internal and external plugins.

Internal Plugins

Pattern: vipr.plugins.{plugin_name}.

vipr discovery hooks --plugin normalizers
# Finds: vipr.plugins.normalizers.log_normalizer.LogNormalizer

External Plugins

External plugins are discovered via Python entry points. See Plugin Development Guide for details on entry point configuration and naming conventions.

Plugin Filtering Example:

vipr discovery hooks --plugin reflectometry
# Example match: vipr_reflectometry.reflectorch.reflectorch_extension.Reflectorch

How it works:

• Input: --plugin reflectometry (plugin name from entry point)

• Pattern constructed: vipr_reflectometry.* (automatic vipr_ prefix added)

• Matches: All components in the vipr_reflectometry package

This is why the vipr_{plugin_name} naming convention is recommended – it enables automatic filtering by plugin name.

Integration with Dynamic Hooks Extension

The Discovery Plugin provides the registry foundation for the Dynamic Hooks and Filters Extension:

1. Decorator Registration: Components use @discover_hook/@discover_filter

2. Registry Validation: Dynamic Hooks Extension validates YAML configs against registries

3. Security: Only registered components can be loaded from configuration

4. Parameter Validation: Parameter schemas from decorators enforce type safety

Flow:

@discover_filter decorator
    ↓
Register in filters_discovery
    ↓
Dynamic Hooks Extension reads registry
    ↓
Validates YAML config against registry
    ↓
Imports and registers callback

Registry Data Structure

Hook/Filter Entry

{
    'enabled_in_config': False,
    'weight': 0,
    'method': 'normalize_filter',
    'class': 'vipr.plugins.normalizers.log_normalizer.LogNormalizer',
    'component_id': 'vipr.plugins.normalizers.log_normalizer.LogNormalizer.normalize_filter',
    'parameters': {'scale_factor': 1.0},
    'parameters_schema': {
        'type': 'object',
        'properties': {'scale_factor': {'type': 'number', 'default': 1.0}}
    }
}

Handler Entry

{
    'handler': 'csv_loader',
    'class': 'vipr.plugins.data_loader.csv_loader.CSVDataLoader',
    'parameters': {'delimiter': ','},
    'parameters_schema': {
        'type': 'object',
        'properties': {
            'file_path': {'type': 'string'},
            'delimiter': {'type': 'string', 'default': ','}
        },
        'required': ['file_path']
    }
}

Best Practices

1. Always use decorators: Register all components via @discover_* decorators

2. Unique handler names: Use descriptive, unique names for handlers

3. Document parameters: Include parameter schemas in decorators

4. Class-only callbacks: Hooks/filters must be class methods (enforced)

5. Set enabled_in_config appropriately: Use False for optional components

Programmatic Access

Query Registries in Code

from vipr.plugins.discovery.decorators import (
    get_discovered_hooks,
    get_discovered_filters,
    get_discovered_data_loaders,
    get_discovered_model_loaders
)

# Get all hooks
all_hooks = get_discovered_hooks()

# Get hooks of specific type
start_hooks = get_discovered_hooks(hook_type='INFERENCE_START_HOOK')

# Get hooks for plugin
plugin_hooks = get_discovered_hooks(plugin_name='normalizers')

# Get data loaders
loaders = get_discovered_data_loaders()

# Filter by plugin
reflectometry_loaders = get_discovered_data_loaders(plugin_name='reflectometry')

Configuration Generation

The Discovery system can be used to automatically generate YAML/JSON configurations based on registered components. This is particularly useful for:

• Default configurations: Generate ready-to-use starter configs for plugins

• User templates: Provide users with pre-configured, customizable setups

Use Case: Plugin Configuration Templates

Plugins can provide commands to generate standard configurations by querying the Discovery registries for their components.

Example from Reflectometry Plugin:

from vipr.plugins.discovery.decorators import (
    get_discovered_filters, get_discovered_hooks,
    get_discovered_data_loaders, get_discovered_model_loaders,
    get_discovered_predictors
)

def generate_standard_config(self):
    """Builds a standard configuration based on registry components."""
    # Query registered components for this plugin
    all_model_loaders = get_discovered_model_loaders('reflectometry')
    all_data_loaders = get_discovered_data_loaders('reflectometry')
    all_predictors = get_discovered_predictors('reflectometry')
    
    # Find specific handlers by name
    model_loader = self._find_handler(all_model_loaders, 'reflectorch')
    data_loader = self._find_handler(all_data_loaders, 'csv_spectrareader')
    predictor = self._find_handler(all_predictors, 'reflectorch_predictor')
    
    # Get all hooks and filters
    hooks = get_discovered_hooks()
    filters = get_discovered_filters()
    
    # Build configuration with default parameter values
    config = {
        'inference': {
            'load_model': self._component_to_config(model_loader),
            'load_data': self._component_to_config(data_loader),
            'prediction': self._component_to_config(predictor),
            'hooks': self._generate_hooks_config(hooks),
            'filters': self._generate_filters_config(filters)
        }
    }
    
    return config

def _component_to_config(self, component):
    """Converts a component to configuration format."""
    config = {'handler': component.get('handler')}
    
    # 'parameters' already contains default values extracted from the Pydantic model
    parameters = component.get('parameters', {})
    
    if parameters:
        config['parameters'] = parameters
    
    return config

CLI Usage

# Generate standard config (returns JSON)
vipr reflectometry generate-standard-config

# Write config to YAML file
vipr reflectometry write-standard-config-to-yaml --output config/my_config.yaml

See Also

• Dynamic Hooks and Filters Extension - Uses Discovery registries for validation

• Inference Plugin - Uses handlers registered via Discovery

• Normalizers Plugin - Example of filter registration

• vipr-core/vipr/plugins/discovery/decorators.py - Decorator implementations

• vipr-core/vipr/plugins/discovery/controllers.py - CLI commands