# Discovery Plugin Documentation

The Discovery Plugin provides a decorator-based registry system for all VIPR components, enabling automatic discovery and introspection of available handlers, hooks, and filters.

## Overview

The Discovery Plugin serves as the central registry for:
- **Hooks & Filters**: Readonly (hooks) and transformative (filters) callbacks for workflow extensions
- **Data Loaders**: Components for loading input data
- **Model Loaders**: Components for loading ML models
- **Predictors**: Components for executing predictions
- **Postprocessors**: Components for post-processing results

**Key Features:**
- **Decorator-based registration**: Simple `@discover_*` decorators
- **CLI introspection**: Query available components via command line
- **API-ready commands**: Controller methods exposable as FastAPI endpoints via `@api` decorator
- **Plugin filtering**: Filter components by plugin name
- **Automatic discovery**: Components self-register on import

## Architecture

### Registry System

The plugin maintains separate registries for each component type:

```python
# Central registries (in decorators.py)
hooks_discovery: dict[str, list[dict]]
filters_discovery: dict[str, list[dict]]
data_loaders_discovery: dict[str, dict]
model_loaders_discovery: dict[str, dict]
predictors_discovery: dict[str, dict]
postprocessors_discovery: dict[str, dict]
```

### Registration Flow

```
1. Plugin loads and imports modules
2. Decorators execute on import (@discover_filter, @discover_data_loader, etc.)
3. Components register in central registries (hooks_discovery, filters_discovery, etc.)
4. Registries ready for introspection via CLI/API and programmatic access
```

## Available Decorators

### Hook & Filter Registration

| Decorator | Purpose | Registry |
|-----------|---------|----------|
| `@discover_hook()` | Register workflow hooks | `hooks_discovery` |
| `@discover_filter()` | Register workflow filters | `filters_discovery` |

**Parameters:**
- `hook_type` / `filter_type`: Hook/filter name (e.g., `INFERENCE_NORMALIZE_PRE_FILTER`)
- `weight`: Execution priority (default: 0, lower runs first)
- `enabled_in_config`: Enable in generated configs (default: `True`)
- `**parameters`: Parameter schema (for validation)

### Handler Registration

| Decorator | Purpose | Registry |
|-----------|---------|----------|
| `@discover_data_loader()` | Register data loaders | `data_loaders_discovery` |
| `@discover_model_loader()` | Register model loaders | `model_loaders_discovery` |
| `@discover_predictor()` | Register predictors | `predictors_discovery` |
| `@discover_postprocessor()` | Register postprocessors | `postprocessors_discovery` |

**Parameters:**
- `handler_name`: Handler identifier (e.g., `csv_spectrareader`)
- `parameters`: Parameter schema (optional)

## Usage Examples

### Registering Hooks/Filters

Hook and filter callbacks are attached to these extension points via YAML configuration, which is passed to the CLI workflow command (e.g., `vipr --config ./config.yaml inference run`).
See [Dynamic Hooks and Filters Extension](../../extensions/dynamic_hooks_filters.md) for details on registering callbacks.

**Example of decorator registration:**
```python
from vipr.plugins.discovery.decorators import discover_filter

class LogNormalizer:
    @discover_filter('INFERENCE_NORMALIZE_PRE_FILTER', enabled_in_config=False)
    def normalize_filter(self, data, **kwargs):
        return data
```

### Registering Data Loader

Real example from Reflectometry Plugin:

```python
from vipr.handlers.data_loader import DataLoaderHandler
from vipr.plugins.discovery.decorators import discover_data_loader
from vipr.plugins.inference.dataset import DataSet

@discover_data_loader('csv_spectrareader', {
    'data_path': {
        'type': 'str', 
        'required': True,
        'description': 'Path to single CSV/DAT/TXT file'
    },
    'column_mapping': {
        'type': 'dict',
        'required': False,
        'default': {'q': 0, 'I': 1},
        'description': 'Column mapping for CSV data'
    }
})
class CSVSpectraReaderDataLoader(DataLoaderHandler):
    """CSV/DAT/TXT data loader for reflectometry data."""
    
    class Meta:
        label = 'csv_spectrareader'
    
    def _load_data(self, data_path: str, column_mapping: dict = None, **kwargs) -> DataSet:
        # Load CSV using SpectraReader
        # Returns DataSet with batch_size=1
        ...
```

### Registering Model Loader

Real example from Reflectometry Plugin:

```python
from vipr.handlers.model_loader import ModelLoaderHandler
from vipr.plugins.discovery.decorators import discover_model_loader
from reflectorch import EasyInferenceModel

@discover_model_loader('reflectorch', {
    'config_name': {'type': 'str', 'required': True},
    'device': {'type': 'str', 'required': False, 'default': 'cpu'}
})
class ReflectorchModelLoader(ModelLoaderHandler):
    """Reflectorch model loader for reflectometry analysis."""
    
    class Meta:
        label = 'reflectorch'
        
    def _load_model(self, **kwargs):
        config_name = kwargs.get('config_name')
        device = kwargs.get('device', 'cpu')
        
        # Load Reflectorch model
        model = EasyInferenceModel(
            config_name=config_name,
            root_dir=os.getenv('REFLECTORCH_ROOT_DIR'),
            device=device
        )
        return model
```

## CLI Commands

### Query Components Summary

```bash
# All components
vipr discovery plugins

# Filter by plugin
vipr discovery plugins --plugin normalizers
```

**Output:**
```json
{
  "model_loaders": 2,
  "data_loaders": 3,
  "hooks": 15,
  "filters": 12,
  "predictors": 1,
  "postprocessors": 0
}
```

### Query Specific Component Types

```bash
# List all model loaders
vipr discovery model-loaders

# List data loaders for specific plugin
vipr discovery data-loaders --plugin reflectometry

# List all hooks
vipr discovery hooks

# List hooks of specific type
vipr discovery hooks --hook-type INFERENCE_POSTPROCESS_PRE_PRE_FILTER_HOOK

# List hooks for specific plugin
vipr discovery hooks --plugin reflectometry

# List all filters
vipr discovery filters

# List filters of specific type
vipr discovery filters --filter-type INFERENCE_NORMALIZE_PRE_FILTER

# List predictors
vipr discovery predictors

```

### Query All Components

```bash
# Complete registry dump
vipr discovery components

# Filter by plugin
vipr discovery components --plugin reflectorch
```

### Query Available Types

```bash
# List all hook types (inference workflow)
vipr discovery hook-types

# List all filter types (inference workflow)
vipr discovery filter-types
```

## API Integration

All discovery commands are available as HTTP endpoints via the API plugin.

### Endpoints

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/discovery/plugins` | GET | Components summary |
| `/discovery/model-loaders` | GET | List model loaders |
| `/discovery/data-loaders` | GET | List data loaders |
| `/discovery/hooks` | GET | List hooks |
| `/discovery/filters` | GET | List filters |
| `/discovery/predictors` | GET | List predictors |
| `/discovery/postprocessors` | GET | List postprocessors |
| `/discovery/components` | GET | All components |
| `/discovery/hook-types` | GET | Available hook types |
| `/discovery/filter-types` | GET | Available filter types |

### Example API Call

```bash
# Get all model loaders
curl http://localhost:8000/discovery/model-loaders

# Get hooks for specific plugin
curl "http://localhost:8000/discovery/hooks?plugin=normalizers"
```

## Plugin Filtering

The Discovery system supports filtering by plugin name for both internal and external plugins.

### Internal Plugins

Pattern: `vipr.plugins.{plugin_name}.`

```bash
vipr discovery hooks --plugin normalizers
# Finds: vipr.plugins.normalizers.log_normalizer.LogNormalizer
```

### External Plugins

External plugins are discovered via Python entry points. See [Plugin Development Guide](../README.md#naming-convention) for details on entry point configuration and naming conventions.

**Plugin Filtering Example:**
```bash
vipr discovery hooks --plugin reflectometry
# Example match: vipr_reflectometry.reflectorch.reflectorch_extension.Reflectorch
```

**How it works:**
- **Input:** `--plugin reflectometry` (plugin name from entry point)
- **Pattern constructed:** `vipr_reflectometry.*` (automatic `vipr_` prefix added)
- **Matches:** All components in the `vipr_reflectometry` package

This is why the `vipr_{plugin_name}` naming convention is recommended – it enables automatic filtering by plugin name.

## Integration with Dynamic Hooks Extension

The Discovery Plugin provides the registry foundation for the [Dynamic Hooks and Filters Extension](../../extensions/dynamic_hooks_filters.md):

1. **Decorator Registration**: Components use `@discover_hook`/`@discover_filter`
2. **Registry Validation**: Dynamic Hooks Extension validates YAML configs against registries
3. **Security**: Only registered components can be loaded from configuration
4. **Parameter Validation**: Parameter schemas from decorators enforce type safety

**Flow:**
```
@discover_filter decorator
    ↓
Register in filters_discovery
    ↓
Dynamic Hooks Extension reads registry
    ↓
Validates YAML config against registry
    ↓
Imports and registers callback
```

## Registry Data Structure

### Hook/Filter Entry

```python
{
    'enabled_in_config': False,
    'weight': 0,
    'method': 'normalize_filter',
    'class': 'vipr.plugins.normalizers.log_normalizer.LogNormalizer',
    'component_id': 'vipr.plugins.normalizers.log_normalizer.LogNormalizer.normalize_filter',
    'parameters': {
        'scale_factor': {'type': 'float', 'default': 1.0}
    }
}
```

### Handler Entry

```python
{
    'handler': 'csv_loader',
    'class': 'vipr.plugins.data_loader.csv_loader.CSVDataLoader',
    'parameters': {
        'file_path': {'type': 'string', 'required': True},
        'delimiter': {'type': 'string', 'default': ','}
    }
}
```

## Best Practices

1. **Always use decorators**: Register all components via `@discover_*` decorators
2. **Unique handler names**: Use descriptive, unique names for handlers
3. **Document parameters**: Include parameter schemas in decorators
4. **Class-only callbacks**: Hooks/filters must be class methods (enforced)
5. **Set enabled_in_config appropriately**: Use `False` for optional components

## Programmatic Access

### Query Registries in Code

```python
from vipr.plugins.discovery.decorators import (
    get_discovered_hooks,
    get_discovered_filters,
    get_discovered_data_loaders,
    get_discovered_model_loaders
)

# Get all hooks
all_hooks = get_discovered_hooks()

# Get hooks of specific type
start_hooks = get_discovered_hooks(hook_type='INFERENCE_START_HOOK')

# Get hooks for plugin
plugin_hooks = get_discovered_hooks(plugin_name='normalizers')

# Get data loaders
loaders = get_discovered_data_loaders()

# Filter by plugin
reflectorch_loaders = get_discovered_data_loaders(plugin_name='reflectorch')
```

## Configuration Generation

The Discovery system can be used to automatically generate YAML/JSON configurations based on registered components. This is particularly useful for:

- **Default configurations**: Generate ready-to-use starter configs for plugins
- **User templates**: Provide users with pre-configured, customizable setups

### Use Case: Plugin Configuration Templates

Plugins can provide commands to generate standard configurations by querying the Discovery registries for their components.

**Example from Reflectometry Plugin:**

```python
from vipr.plugins.discovery.decorators import (
    get_discovered_filters, get_discovered_hooks,
    get_discovered_data_loaders, get_discovered_model_loaders,
    get_discovered_predictors
)

def generate_standard_config(self):
    """Builds a standard configuration based on registry components."""
    # Query registered components for this plugin
    all_model_loaders = get_discovered_model_loaders('reflectometry')
    all_data_loaders = get_discovered_data_loaders('reflectometry')
    all_predictors = get_discovered_predictors('reflectometry')
    
    # Find specific handlers by name
    model_loader = self._find_handler(all_model_loaders, 'reflectorch')
    data_loader = self._find_handler(all_data_loaders, 'csv_spectrareader')
    predictor = self._find_handler(all_predictors, 'reflectorch_predictor')
    
    # Get all hooks and filters
    hooks = get_discovered_hooks()
    filters = get_discovered_filters()
    
    # Build configuration with default parameter values
    config = {
        'inference': {
            'load_model': self._component_to_config(model_loader),
            'load_data': self._component_to_config(data_loader),
            'prediction': self._component_to_config(predictor),
            'hooks': self._generate_hooks_config(hooks),
            'filters': self._generate_filters_config(filters)
        }
    }
    
    return config

def _component_to_config(self, component):
    """Converts a component to configuration format."""
    config = {'handler': component.get('handler')}
    
    # Extract default parameter values from schema
    parameters = {}
    for param_name, param_info in component.get('parameters', {}).items():
        if 'default' in param_info:
            parameters[param_name] = param_info['default']
    
    if parameters:
        config['parameters'] = parameters
    
    return config
```

### CLI Usage

```bash
# Generate standard config (returns JSON)
vipr reflectometry generate-standard-config

# Write config to YAML file
vipr reflectometry write-standard-config-to-yaml --output config/my_config.yaml
```

## See Also

- [Dynamic Hooks and Filters Extension](../../extensions/dynamic_hooks_filters.md) - Uses Discovery registries for validation
- [Inference Plugin](./inference.md) - Uses handlers registered via Discovery
- [Normalizers Plugin](./normalizers.md) - Example of filter registration
- `vipr-core/vipr/plugins/discovery/decorators.py` - Decorator implementations
- `vipr-core/vipr/plugins/discovery/controllers.py` - CLI commands