# Configuration Examples

Complete workflow configurations demonstrating VIPR framework integration patterns.

## Reflectorch XRR Workflow

### Basic Configuration

```yaml
vipr:
  inference:
    # DataLoader Handler Selection
    load_data:
      handler: csv_spectrareader           # Handler label from registry
      parameters:
        data_path: '@vipr_reflectometry/reflectorch/examples/data/PTCDI-C3.txt'
        column_mapping:                    # Handler-specific parameters
          q: 0                             # via @discover_data_loader
          I: 1
    
    # ModelLoader Handler Selection
    load_model:
      handler: reflectorch                 # Handler label from registry
      parameters:
        config_name: b_mc_point_xray_conv_standard_L2_InputQ
        device: cpu
    
    # Predictor Handler Selection
    prediction:
      handler: reflectorch_predictor       # Handler label from registry
      parameters:
        calc_pred_curve: true              # Predictor-specific parameters
        calc_pred_sld_profile: true        # via @discover_predictor
        clip_prediction: true
        polish_prediction: true
        prior_bounds:
          thicknesses: [[1, 1500]]
          slds: [[1.5, 6.5]]
          roughnesses: [[0, 10]]
          log10_background: [-10, -4]
          r_scale: [0.9, 1.1]
```

**Framework Integration Points**:
- ✅ Handler selection via registry labels
- ✅ Parameter exposure via discovery decorators
- ✅ DataSet flow (CSV → Reflectorch predictor)
- ✅ No explicit filter/hook configuration (default workflow)

## Reflectorch NR Workflow with Data Cleaning

### Configuration with Preprocessing Filters

```yaml
vipr:
  inference:
    # Filter Configuration (explicit activation)
    filters:
      INFERENCE_PREPROCESS_PRE_FILTER:
        # Filter 1: Data cleaning (weight=-10, runs first)
        - class: vipr_reflectometry.shared.preprocessing.neutron_data_cleaner.NeutronDataCleaner
          enabled: true                    # Explicit activation required
          method: clean_experimental_data  # Filter method from @discover_filter
          parameters:                      # Filter parameters (passed as kwargs)
            error_threshold: 0.5           # Relative error threshold (dR/R)
            consecutive_errors: 3          # Truncation trigger
            remove_single_errors: false
          weight: -10                      # Execution order (lowest first)
        
        # Filter 2: Interpolation (weight=0, runs after cleaning)
        - class: vipr_reflectometry.reflectorch.reflectorch_extension.Reflectorch
          enabled: true
          method: _preprocess_interpolate  # Extension method as filter
          parameters: null                 # No parameters for this filter
          weight: 0
    
    # Data loading with error bars
    load_data:
      handler: csv_spectrareader
      parameters:
        data_path: '@vipr_reflectometry/reflectorch/examples/data/D17_SiO.dat'
        column_mapping:
          q: 0
          I: 1
          dI: 2    # Intensity errors (required for cleaning)
          dQ: 3    # Q resolution
    
    # Neutron reflectometry model
    load_model:
      handler: reflectorch
      parameters:
        config_name: e_mc_point_neutron_conv_standard_L1_InputQDq_n256_size1024
        device: cpu
    
    # Prediction with neutron-specific parameters
    prediction:
      handler: reflectorch_predictor
      parameters:
        calc_pred_curve: true
        calc_pred_sld_profile: true
        clip_prediction: true
        q_resolution: 0.1                  # Neutron-specific: Q resolution
        upper_phase_sld: 0                 # Neutron-specific: D2O/air
        use_q_shift: false
        prior_bounds:
          thicknesses: [[1, 1500]]
          slds: [[1.5, 6.5], [1.5, 6.5]]
          roughnesses: [[0, 10], [0, 10]]
          log10_background: [-10, -4]
          r_scale: [0.9, 1.1]
```

**Framework Integration Points**:
- ✅ Filter chain: Cleaning (weight=-10) → Interpolation (weight=0)
- ✅ DataSet transformation: Raw → Cleaned → Interpolated
- ✅ Error propagation through filter chain (dI, dQ)
- ✅ Extension method used as filter (`Reflectorch._preprocess_interpolate`)
- ✅ Configuration-driven filter activation

### Filter Execution Flow

```
LoadDataStep
  → DataSet(x=Q, y=R, dx=dQ, dy=dI, batch_size=1)
     ↓
PreprocessStep:
  → NeutronDataCleaner.clean_experimental_data (weight=-10)
     Input:  DataSet (raw, may have negative R, high dR/R)
     Output: DataSet (cleaned, padded, dR/R < threshold)
     ↓
  → Reflectorch._preprocess_interpolate (weight=0)
     Input:  DataSet (cleaned, variable Q-grid)
     Output: DataSet (interpolated, model Q-grid)
     ↓
PredictionStep
  → ReflectorchPredictor._predict
     Input:  DataSet (preprocessed)
     Output: dict(predictions, pred_curve, sld_profile)
```

## Flow Model Workflow

### NSF XRR with Posterior Sampling

```yaml
vipr:
  inference:
    # Flow-specific preprocessing filter
    filters:
      INFERENCE_PREPROCESS_PRE_FILTER:
        - class: vipr_reflectometry.flow_models.flow_preprocessor.FlowPreprocessor
          enabled: true
          method: _preprocess_flow          # Log-scale transformation
          parameters: null
          weight: 0
    
    # Postprocessing visualization hook
    hooks:
      INFERENCE_POSTPROCESS_PRE_PRE_FILTER_HOOK:
        - class: vipr_reflectometry.flow_models.postprocessors.basic_corner_plot.BasicCornerPlot
          enabled: true
          method: _create_basic_corner_plot  # Hook for visualization
          parameters: null
          weight: 0
    
    # Data loading
    load_data:
      handler: csv_spectrareader
      parameters:
        data_path: '@vipr_reflectometry/reflectorch/examples/data/PTCDI-C3.txt'
        column_mapping:
          q: 0
          I: 1
    
    # Flow model loading
    load_model:
      handler: flow_model_loader           # Custom model loader
      parameters:
        config_name: mc1_nsf                # Neural Spline Flow configuration
        device: cpu
        model_dir: saved_models
        weights_format: pt
    
    # Flow predictor (generates posterior samples)
    prediction:
      handler: flow_predictor              # Custom predictor for flows
      parameters:
        num_samples: 2000                   # Posterior samples to generate
        return_scaled: false                # Return in original parameter space
```

**Framework Integration Points**:
- ✅ Custom preprocessing filter (log-scale transformation)
- ✅ Hook system for visualization (corner plot)
- ✅ Custom model loader (flow architectures)
- ✅ Custom predictor (posterior sampling vs point prediction)
- ✅ Data collector pattern for UI integration

### Flow Model Output

```python
# Predictor returns dictionary (not DataSet)
{
    'posterior_samples': np.ndarray,  # (num_samples, n_parameters)
    'num_samples': 2000,
    'batch_size': 1
}

# Hook adds visualization to data collector
app.flow_dc.add_plot('corner_plot', fig)
```

## Flow Model with Clustering Postprocessing

### Complete Configuration with All Features

```yaml
vipr:
  inference:
    # Preprocessing
    filters:
      INFERENCE_PREPROCESS_PRE_FILTER:
        - class: vipr_reflectometry.flow_models.flow_preprocessor.FlowPreprocessor
          enabled: true
          method: _preprocess_flow
          parameters: null
          weight: 0
    
    # Multiple postprocessing hooks
    hooks:
      INFERENCE_POSTPROCESS_PRE_PRE_FILTER_HOOK:
        # Hook 1: Clustering analysis (weight=0)
        - class: vipr_reflectometry.flow_models.postprocessors.cluster.clustering.hook.ClusterHook
          enabled: true
          method: run_clustering_analysis
          parameters:
            algorithm: kmeans              # Clustering algorithm
            n_clusters: 3                  # Number of clusters
          weight: 0
        
        # Hook 2: Cluster diagnostics (weight=1, runs after clustering)
        - class: vipr_reflectometry.flow_models.postprocessors.cluster.model_selection.hook.ClusterDiagnosticsHook
          enabled: true
          method: generate_diagnostics
          parameters: null
          weight: 1
        
        # Hook 3: Visualizations (weight=2, runs last)
        - class: vipr_reflectometry.flow_models.postprocessors.marginal_distributions.MarginalDistributions
          enabled: true
          method: plot_marginals
          parameters: null
          weight: 2
    
    load_data:
      handler: hdf5_spectrareader          # Batch loading (N spectra)
      parameters:
        data_path: 'data/xrr_dataset.h5'
        group_path: '/spectra'
    
    load_model:
      handler: flow_model_loader
      parameters:
        config_name: mc1_cinn              # Conditional INN
        device: cuda
        model_dir: saved_models
        weights_format: pt
    
    prediction:
      handler: flow_predictor
      parameters:
        num_samples: 5000
        return_scaled: false
```

**Framework Integration Points**:
- ✅ Multiple hooks with weight-based execution order
- ✅ Hook chain: Clustering → Diagnostics → Visualization
- ✅ Batch processing (HDF5 loader, batch_size > 1)
- ✅ Complex postprocessing workflow
- ✅ Multiple data collectors for different visualizations

## PANPE Workflow

### Configuration (Optional Submodule)

```yaml
vipr:
  inference:
    load_data:
      handler: csv_spectrareader
      parameters:
        data_path: '@vipr_reflectometry/panpe/examples/data/xrr.dat'
        column_mapping:
          q: 0
          I: 1
    
    load_model:
      handler: panpe_model_loader          # PANPE-specific loader
      parameters:
        model_name: panpe_xrr_model
        device: cpu
    
    prediction:
      handler: panpe_predictor             # PANPE-specific predictor
      parameters:
        num_samples: 2000
        adaptive_prior: true               # PANPE feature
```

**Note**: PANPE requires separate installation (`pip install -e .[plugin-panpe]`)

## Multi-File Batch Processing

### HDF5 Batch Configuration

```yaml
vipr:
  inference:
    # Preprocessing with cleaning
    filters:
      INFERENCE_PREPROCESS_PRE_FILTER:
        - class: vipr_reflectometry.shared.preprocessing.neutron_data_cleaner.NeutronDataCleaner
          enabled: true
          method: clean_experimental_data
          parameters:
            error_threshold: 0.5
            consecutive_errors: 3
          weight: -10
        
        - class: vipr_reflectometry.reflectorch.reflectorch_extension.Reflectorch
          enabled: true
          method: _preprocess_interpolate
          parameters: null
          weight: 0
    
    # Batch data loading (N spectra)
    load_data:
      handler: hdf5_spectrareader          # Batch-capable loader
      parameters:
        data_path: 'storage/reflectorch/experimental_data.h5'
        group_path: '/measurements'
    
    load_model:
      handler: reflectorch
      parameters:
        config_name: e_mc_point_neutron_conv_standard_L1_InputQDq_n256_size1024
        device: cuda                       # GPU for batch processing
    
    prediction:
      handler: reflectorch_predictor
      parameters:
        calc_pred_curve: true
        calc_pred_sld_profile: true
        clip_prediction: true
        prior_bounds:
          thicknesses: [[1, 1500]]
          slds: [[1.5, 6.5], [1.5, 6.5]]
          roughnesses: [[0, 10], [0, 10]]
```

**Batch Processing Flow**:

```
HDF5SpectraReaderDataLoader
  → DataSet(batch_size=N, n_points=variable)
     ↓
NeutronDataCleaner (processes each spectrum independently)
  → DataSet(batch_size=N, n_points=max_len, padded=True)
     ↓
Reflectorch._preprocess_interpolate (batch operation)
  → DataSet(batch_size=N, n_points=model_grid_size)
     ↓
ReflectorchPredictor (batch prediction)
  → dict(predictions=[N x parameters], pred_curves=[N x n_points])
```

## Key Configuration Patterns

### Handler Selection Pattern

```yaml
step_name:
  handler: label_from_registry    # Handler lookup by label
  parameters:                     # Handler-specific parameters
    param1: value1                # Exposed via @discover_* decorator
    param2: value2
```

### Filter Configuration Pattern

```yaml
filters:
  FILTER_SLOT_NAME:              # e.g., INFERENCE_PREPROCESS_PRE_FILTER
    - class: fully.qualified.ClassName
      enabled: true              # Explicit activation
      method: filter_method      # Method with @discover_filter
      parameters:                # Passed as kwargs to filter
        param1: value1
      weight: 0                  # Execution order (lower first)
```

### Hook Configuration Pattern

```yaml
hooks:
  HOOK_SLOT_NAME:                # e.g., INFERENCE_POSTPROCESS_PRE_PRE_FILTER_HOOK
    - class: fully.qualified.ClassName
      enabled: true              # Explicit activation
      method: hook_method        # Method with @discover_hook
      parameters: null           # Passed as kwargs to hook
      weight: 0                  # Execution order (lower first)
```

## Package Path Resolution

The plugin uses VIPR's path resolution for data files:

```yaml
# Package path (resolved to plugin location)
data_path: '@vipr_reflectometry/reflectorch/examples/data/D17_SiO.dat'

# Alternative syntax
data_path: 'vipr_reflectometry/reflectorch/examples/data/D17_SiO.dat'

# Absolute path
data_path: '/home/user/data/experiment.dat'

# Relative to working directory
data_path: 'data/experiment.dat'
```

**Pattern**: Use `@package_name/path` for plugin-bundled data files.

## CLI Usage

```bash
# Run with configuration file
vipr --config path/to/config.yaml inference run

# Run with package-path configuration
vipr --config '@vipr_reflectometry/reflectorch/examples/configs/D17_SiO.yaml' inference run

# Run with environment override
REFLECTORCH_ROOT_DIR=/custom/path vipr --config config.yaml inference run
```

## Integration Summary

### VIPR Framework Concepts Demonstrated

1. **Handler Registry**: Dynamic handler selection by label
2. **Discovery System**: Parameter exposure via decorators
3. **DataSet Flow**: Immutable data transfer between steps
4. **Filter Chain**: Weight-based preprocessing pipeline
5. **Hook System**: Event-driven workflow customization
6. **Extension Pattern**: Domain-specific utilities via `app.extend()`
7. **Configuration-Driven**: YAML-based workflow definition
8. **Batch Processing**: N spectra processed in single workflow
9. **Error Propagation**: Uncertainties transformed with data
10. **Package Resolution**: Plugin-bundled resource access

### Workflow Integration Points

```
Configuration (YAML)
    ↓
Handler Selection (Registry Lookup)
    ↓
Parameter Injection (Discovery System)
    ↓
Data Loading → DataSet Creation
    ↓
Filter Chain (Weight-Based Order)
    ↓
Model Loading (Environment Integration)
    ↓
Prediction (Workflow State Access)
    ↓
Hook Execution (Visualization, Analysis)
    ↓
Result Output (Data Collectors)
```

All configuration examples demonstrate how the Reflectometry Plugin leverages VIPR's framework patterns to create flexible, domain-specific workflows without modifying core framework code.