API Plugin Documentation

The API Plugin provides three core capabilities for VIPR framework integration: automatic FastAPI endpoint generation from CLI controllers, structured UI data collection, and UUID-based result storage.

Overview

The API Plugin provides three independent functions that can be used separately or combined:

• @api Decorator: Auto-generate FastAPI endpoints from Cement controllers

• DataCollector: Structured collection of UI data (tables, diagrams, images, logs)

• Result Storage: UUID-based storage with automatic extraction to human-readable formats

Key Point: These are independent capabilities - you can use @api without DataCollector, use DataCollector without storage, or combine all three.

Documentation Structure

• README.md (this file) - Overview, Architecture, Core Components

• usage.md - Usage Examples, Best Practices, Integration

• visualization.md - YAML configuration for matplotlib plot output

Architecture

The API Plugin provides three independent functions:

Function 1: @api Decorator - Automatic Endpoint Generation

Purpose: Convert CLI controller methods to FastAPI HTTP endpoints

Flow:

CLI Controller Method
      ↓
@api(path="/...", method="...", request=Model, response=Model)
      ↓
Metadata attached to method
      ↓
build_router() scans all controllers
      ↓
FastAPI Endpoint (auto-generated with validation)
      ↓
OpenAPI schema generation

Independence: Works standalone - doesn’t require DataCollector or storage

Example:

@api("/discovery/components", "GET", response=dict)
@ex(help='List components')
def components(self):
    return {"count": 42}  # Direct return, no DataCollector needed

---

Function 2: DataCollector - Structured Data Collection

Purpose: Collect structured UI data (tables, diagrams, images, logs) during execution

Flow:

Anywhere in code (hooks, filters, controllers)
      ↓
app.datacollector.table('id', 'Title')
app.datacollector.diagram('id', 'Title')
app.datacollector.image('id', 'Title')
      ↓
Data collected in UIData model (in-memory)
      ↓
Optional: save to storage

Independence: Can be used anywhere without @api decorator or storage

Example:

def my_preprocessing_hook(app):
    # Create table with id 'params' and collect data without saving
    app.datacollector.table('params')\
        .add_row(parameter="thickness_layer_1", predicted="123.4567", polished="123.5000")

---

Function 3: Result Storage - UUID-based Persistence

Purpose: Save collected data to disk with UUID identifier and extract to human-readable formats

Flow:

app.datacollector.save_result(uuid)
      ↓
ResultStorage
      ↓
UUID directory structure:
├── data.pkl              # Complete UIData (pickle)
├── config.yaml           # Original config
├── images/               # PNG/SVG files
├── tables/               # CSV files
└── diagrams/             # CSV files (per series)

Independence: Can store any data structure, not just DataCollector output

Example:

# Save collected data
result_id = str(uuid.uuid4())
app.datacollector.save_result(result_id)

# Later: retrieve
storage = ResultStorage()
data = storage.get_result(result_id)

---

Component Architecture

API Plugin (vipr-core/vipr/plugins/api/)
├── decorator.py          # Function 1: @api decorator
├── data_collector.py     # Function 2: DataCollector + Builders
├── models.py            # Function 2: Pydantic models (UIData, etc.)
├── result_storage.py    # Function 3: UUID-based file storage
├── controllers.py       # UIController (uses @api + ResultStorage)
└── fastapi/            # Function 1: Router generation
    └── router_generator/

---

Integration Example: VIPR Inference

How the API Plugin’s functions are used in the inference workflow:

The inference workflow demonstrates how external code (vipr-framework) can use the API Plugin’s components:

What the API Plugin Provides:

1. DataCollector (Function 2) - Used during inference to collect UI data

2. ResultStorage (Function 3) - Used to persist collected data with UUID

3. UIController with @api (Function 1) - Provides GET endpoint to retrieve stored results

How External Code Uses It:

# vipr-framework/tasks.py (NOT part of API Plugin)
# Manual FastAPI router for long-running inference
@router.post("/api/inference/run")
async def run_inference_async(config: dict):
    # Start Celery task (async execution needed - can't use @api)
    task = run_vipr_inference.delay(config, result_id=uuid.uuid4())
    return {"task_id": task.id, "result_id": result_id}

# During Celery task execution:
# 1. DataCollector from API Plugin is used
def my_inference_hook(app):
    # API Plugin provides app.datacollector
    app.datacollector.table('results').add_row(value=123)  
    app.datacollector.diagram('plot')\
        .set_data('q_experimental', q_values)\
        .set_data('reflectivity_experimental', r_values)

# 2. ResultStorage from API Plugin is used automatically
# INFERENCE_COMPLETE_HOOK (registered by API Plugin)
def store_ui_data_directly(app):
    result_id = app.config.get('vipr', 'result_id')
    # API Plugin's DataCollector.save_result() internally uses ResultStorage
    app.datacollector.save_result(result_id)  # ← API Plugin function

# 3. @api decorator from API Plugin provides result endpoint
# vipr-core/vipr/plugins/api/controllers.py (part of API Plugin)
@api("/ui/result", "GET")  # ← API Plugin decorator
def get_result(self):
    storage = ResultStorage()  # ← API Plugin class
    return storage.get_result(result_id)

Complete Flow with Frontend Polling:

1. Frontend → POST /api/inference/run
   Response: {task_id: "abc123", result_id: "uuid-456"}

2. Frontend → Polling GET /api/progress/abc123
   While status != "SUCCESS": wait and poll again

3. Frontend → GET /ui/result?id=uuid-456 (← API Plugin endpoint)
   → ResultStorage.get_result(uuid-456)
   → Returns complete UIData with tables/diagrams/images

4. Frontend displays results

Summary:

Component	Part of API Plugin?	Usage
POST /api/inference/run	❌ No (vipr-framework)	Starts inference, returns task_id + result_id
GET /api/progress/{task_id}	❌ No (vipr-framework)	Polls Celery task status
DataCollector	✅ Yes	Used during inference to collect data
ResultStorage	✅ Yes	Used to save/load results
GET /ui/result	✅ Yes (UIController)	Retrieves stored results after completion

Why inference starter doesn’t use @api: Long-running operations need Celery for async execution. The @api decorator is designed for fast, synchronous operations only.

Result retrieval timing: The frontend polls the task status endpoint until the Celery job is complete, then calls the API Plugin’s GET /ui/result endpoint to retrieve the stored results.

---

Advanced: Using ResultStorage with Arbitrary Data

While ResultStorage is typically used with DataCollector output, it’s a generic storage system that can persist any Python data structure.

Examples:

from vipr.plugins.api.result_storage import ResultStorage
import uuid

storage = ResultStorage()

# 1. Store raw predictions
predictions = model.predict(X)
storage.save_result_with_id(str(uuid.uuid4()), {
    "predictions": predictions.tolist(),
    "model_version": "v2.1",
    "timestamp": datetime.now().isoformat()
})

# 2. Cache intermediate computations
cache_id = str(uuid.uuid4())
storage.save_result_with_id(cache_id, {
    "layer_activations": activations,
    "gradients": grads,
    "loss_history": losses
})

# Retrieve later
cached_data = storage.get_result(cache_id)

Storing Arbitrary Data vs. DataCollector Data:

• storage.save_result_with_id(uuid, data) - Store any data as pickle file only

• datacollector.save_result(uuid) - Store DataCollector UIData as pickle + extract images/tables/diagrams to separate files

Core Components

1. vipr.plugins.api.decorator.api()

Mark controller methods as API endpoints with type-safe request/response models.

See: vipr.plugins.api.decorator for complete decorator documentation.

Decorator signature:

@api(path: str, 
     method: str = "POST",
     request: Optional[type[BaseModel]] = None,
     response: Any = None,
     tags: Optional[list[str]] = None,
     operation_id: Optional[str] = None,
     transform_result: Optional[Callable] = None)

Key features:

• Type-safe request/response validation

• Automatic OpenAPI schema generation

• Dual accessibility (CLI + HTTP)

• Optional result transformation for HTTP

2. vipr.plugins.api.data_collector.DataCollector

Singleton pattern for collecting structured UI data throughout workflow execution.

Access pattern:

# Extended to app instance
app.datacollector.table('id', 'Title')
app.datacollector.diagram('id', 'Title')
app.datacollector.image('id', 'Title')
app.datacollector.log('message', LogLevel.INFO)

Builder interfaces:

• TableBuilder: Row/column operations

• DiagramBuilder: Data series with metadata

• ImageBuilder: Matplotlib/bytes/file support

3. vipr.plugins.api.result_storage.ResultStorage

UUID-based storage system with automatic file extraction.

Storage structure:

storage/results/
└── <uuid>/
    ├── data.pkl              # Complete result (Pydantic → dict → pickle)
    ├── config.yaml           # Original inference config
    ├── images/
    │   └── sld_profile.png
    ├── tables/
    │   └── parameters.csv
    └── diagrams/
        └── reflectivity_Predicted.csv
        └── reflectivity_Experimental.csv

4. UIController

REST endpoints for result retrieval and storage management.

Endpoints:

• GET /ui/result?id=<uuid> - Retrieve stored result

• GET /ui/result/status?id=<uuid> - Check result existence

• GET /ui/storage/info - Storage statistics

• POST /ui/storage/cleanup - Clean old results

Environment Variables

# Result storage directory
export VIPR_RESULTS_DIR=storage/results

# Config file storage directory  
export VIPR_CONFIG_DIR=/tmp

See Also

• usage.md - Usage Examples, Best Practices, Integration

• Inference Plugin - Uses DataCollector for workflow data

• Discovery Plugin - Auto-generates API endpoints from registry

• Dynamic Hooks Extension - Hook system integration

• vipr-core/vipr/plugins/api/ - Plugin source code

• vipr-framework/services/vipr/main.py - FastAPI integration example