# API Plugin Documentation

The API Plugin provides three core capabilities for VIPR framework integration: automatic FastAPI endpoint generation from CLI controllers, structured UI data collection, and UUID-based result storage.

## Overview

The API Plugin provides **three independent functions** that can be used separately or combined:
- **@api Decorator**: Auto-generate FastAPI endpoints from Cement controllers
- **DataCollector**: Structured collection of UI data (tables, diagrams, images, logs)
- **Result Storage**: UUID-based storage with automatic extraction to human-readable formats

**Key Point:** These are independent capabilities - you can use @api without DataCollector, use DataCollector without storage, or combine all three.

## Documentation Structure

- **[README.md](./README.md)** (this file) - Overview, Architecture, Core Components
- **[usage.md](./usage.md)** - Usage Examples, Best Practices, Integration
- **[visualization.md](./visualization.md)** - YAML configuration for matplotlib plot output

## Architecture

The API Plugin provides three independent functions:

### Function 1: @api Decorator - Automatic Endpoint Generation

**Purpose:** Convert CLI controller methods to FastAPI HTTP endpoints

**Flow:**
```
CLI Controller Method
      ↓
@api(path="/...", method="...", request=Model, response=Model)
      ↓
Metadata attached to method
      ↓
build_router() scans all controllers
      ↓
FastAPI Endpoint (auto-generated with validation)
      ↓
OpenAPI schema generation
```

**Independence:** Works standalone - doesn't require DataCollector or storage

**Example:**
```python
@api("/discovery/components", "GET", response=dict)
@ex(help='List components')
def components(self):
    return {"count": 42}  # Direct return, no DataCollector needed
```

---

### Function 2: DataCollector - Structured Data Collection

**Purpose:** Collect structured UI data (tables, diagrams, images, logs) during execution

**Flow:**
```
Anywhere in code (hooks, filters, controllers)
      ↓
app.datacollector.table('id', 'Title')
app.datacollector.diagram('id', 'Title')
app.datacollector.image('id', 'Title')
      ↓
Data collected in UIData model (in-memory)
      ↓
Optional: save to storage
```

**Independence:** Can be used anywhere without @api decorator or storage

**Example:**
```python
def my_preprocessing_hook(app):
    # Create table with id 'params' and collect data without saving
    app.datacollector.table('params')\
        .add_row(parameter="thickness_layer_1", predicted="123.4567", polished="123.5000")
```

---

### Function 3: Result Storage - UUID-based Persistence

**Purpose:** Save collected data to disk with UUID identifier and extract to human-readable formats

**Flow:**
```
app.datacollector.save_result(uuid)
      ↓
ResultStorage
      ↓
UUID directory structure:
├── data.pkl              # Complete UIData (pickle)
├── config.yaml           # Original config
├── images/               # PNG/SVG files
├── tables/               # CSV files
└── diagrams/             # CSV files (per series)
```

**Independence:** Can store any data structure, not just DataCollector output

**Example:**
```python
# Save collected data
result_id = str(uuid.uuid4())
app.datacollector.save_result(result_id)

# Later: retrieve
storage = ResultStorage()
data = storage.get_result(result_id)
```

---

### Component Architecture

```
API Plugin (vipr-core/vipr/plugins/api/)
├── decorator.py          # Function 1: @api decorator
├── data_collector.py     # Function 2: DataCollector + Builders
├── models.py            # Function 2: Pydantic models (UIData, etc.)
├── result_storage.py    # Function 3: UUID-based file storage
├── controllers.py       # UIController (uses @api + ResultStorage)
└── fastapi/            # Function 1: Router generation
    └── router_generator/
```

---

### Integration Example: VIPR Inference

**How the API Plugin's functions are used in the inference workflow:**

The inference workflow demonstrates how external code (vipr-framework) can use the API Plugin's components:

#### What the API Plugin Provides:

1. **DataCollector (Function 2)** - Used during inference to collect UI data
2. **ResultStorage (Function 3)** - Used to persist collected data with UUID
3. **UIController with @api (Function 1)** - Provides GET endpoint to retrieve stored results

#### How External Code Uses It:

```python
# vipr-framework/tasks.py (NOT part of API Plugin)
# Manual FastAPI router for long-running inference
@router.post("/api/inference/run")
async def run_inference_async(config: dict):
    # Start Celery task (async execution needed - can't use @api)
    task = run_vipr_inference.delay(config, result_id=uuid.uuid4())
    return {"task_id": task.id, "result_id": result_id}
```

```python
# During Celery task execution:
# 1. DataCollector from API Plugin is used
def my_inference_hook(app):
    # API Plugin provides app.datacollector
    app.datacollector.table('results').add_row(value=123)  
    app.datacollector.diagram('plot')\
        .set_data('q_experimental', q_values)\
        .set_data('reflectivity_experimental', r_values)

# 2. ResultStorage from API Plugin is used automatically
# INFERENCE_COMPLETE_HOOK (registered by API Plugin)
def store_ui_data_directly(app):
    result_id = app.config.get('vipr', 'result_id')
    # API Plugin's DataCollector.save_result() internally uses ResultStorage
    app.datacollector.save_result(result_id)  # ← API Plugin function
```

```python
# 3. @api decorator from API Plugin provides result endpoint
# vipr-core/vipr/plugins/api/controllers.py (part of API Plugin)
@api("/ui/result", "GET")  # ← API Plugin decorator
def get_result(self):
    storage = ResultStorage()  # ← API Plugin class
    return storage.get_result(result_id)
```

#### Complete Flow with Frontend Polling:

```
1. Frontend → POST /api/inference/run
   Response: {task_id: "abc123", result_id: "uuid-456"}

2. Frontend → Polling GET /api/progress/abc123
   While status != "SUCCESS": wait and poll again

3. Frontend → GET /ui/result?id=uuid-456 (← API Plugin endpoint)
   → ResultStorage.get_result(uuid-456)
   → Returns complete UIData with tables/diagrams/images

4. Frontend displays results
```

#### Summary:

| Component | Part of API Plugin? | Usage |
|-----------|---------------------|-------|
| POST /api/inference/run | ❌ No (vipr-framework) | Starts inference, returns task_id + result_id |
| GET /api/progress/{task_id} | ❌ No (vipr-framework) | Polls Celery task status |
| DataCollector | ✅ Yes | Used during inference to collect data |
| ResultStorage | ✅ Yes | Used to save/load results |
| GET /ui/result | ✅ Yes (UIController) | Retrieves stored results after completion |

**Why inference starter doesn't use @api:** Long-running operations need Celery for async execution. The @api decorator is designed for fast, synchronous operations only.

**Result retrieval timing:** The frontend polls the task status endpoint until the Celery job is complete, then calls the API Plugin's `GET /ui/result` endpoint to retrieve the stored results.

---

### Advanced: Using ResultStorage with Arbitrary Data

While ResultStorage is typically used with DataCollector output, it's a **generic storage system** that can persist any Python data structure.

**Examples:**

```python
from vipr.plugins.api.result_storage import ResultStorage
import uuid

storage = ResultStorage()

# 1. Store raw predictions
predictions = model.predict(X)
storage.save_result_with_id(str(uuid.uuid4()), {
    "predictions": predictions.tolist(),
    "model_version": "v2.1",
    "timestamp": datetime.now().isoformat()
})

# 2. Cache intermediate computations
cache_id = str(uuid.uuid4())
storage.save_result_with_id(cache_id, {
    "layer_activations": activations,
    "gradients": grads,
    "loss_history": losses
})

# Retrieve later
cached_data = storage.get_result(cache_id)
```

**Storing Arbitrary Data vs. DataCollector Data:**
- `storage.save_result_with_id(uuid, data)` - Store any data as pickle file only
- `datacollector.save_result(uuid)` - Store DataCollector UIData as pickle + extract images/tables/diagrams to separate files

## Core Components

### 1. {py:func}`vipr.plugins.api.decorator.api`

Mark controller methods as API endpoints with type-safe request/response models.

**See:** {py:mod}`vipr.plugins.api.decorator` for complete decorator documentation.

**Decorator signature:**
```python
@api(path: str, 
     method: str = "POST",
     request: Optional[type[BaseModel]] = None,
     response: Any = None,
     tags: Optional[list[str]] = None,
     operation_id: Optional[str] = None,
     transform_result: Optional[Callable] = None)
```

**Key features:**
- Type-safe request/response validation
- Automatic OpenAPI schema generation
- Dual accessibility (CLI + HTTP)
- Optional result transformation for HTTP

### 2. {py:class}`vipr.plugins.api.data_collector.DataCollector`

Singleton pattern for collecting structured UI data throughout workflow execution.

**Access pattern:**
```python
# Extended to app instance
app.datacollector.table('id', 'Title')
app.datacollector.diagram('id', 'Title')
app.datacollector.image('id', 'Title')
app.datacollector.log('message', LogLevel.INFO)
```

**Builder interfaces:**
- `TableBuilder`: Row/column operations
- `DiagramBuilder`: Data series with metadata
- `ImageBuilder`: Matplotlib/bytes/file support

### 3. {py:class}`vipr.plugins.api.result_storage.ResultStorage`

UUID-based storage system with automatic file extraction.

**Storage structure:**
```
storage/results/
└── <uuid>/
    ├── data.pkl              # Complete result (Pydantic → dict → pickle)
    ├── config.yaml           # Original inference config
    ├── images/
    │   └── sld_profile.png
    ├── tables/
    │   └── parameters.csv
    └── diagrams/
        └── reflectivity_Predicted.csv
        └── reflectivity_Experimental.csv
```

### 4. UIController

REST endpoints for result retrieval and storage management.

**Endpoints:**
- `GET /ui/result?id=<uuid>` - Retrieve stored result
- `GET /ui/result/status?id=<uuid>` - Check result existence
- `GET /ui/storage/info` - Storage statistics
- `POST /ui/storage/cleanup` - Clean old results

## Environment Variables

```bash
# Result storage directory
export VIPR_RESULTS_DIR=storage/results

# Config file storage directory  
export VIPR_CONFIG_DIR=/tmp
```

## See Also

- **[usage.md](./usage.md)** - Usage Examples, Best Practices, Integration
- [Inference Plugin](../inference.md) - Uses DataCollector for workflow data
- [Discovery Plugin](../discovery.md) - Auto-generates API endpoints from registry
- [Dynamic Hooks Extension](../../extensions/dynamic_hooks_filters.md) - Hook system integration
- `vipr-core/vipr/plugins/api/` - Plugin source code
- `vipr-framework/services/vipr/main.py` - FastAPI integration example