# Registry Plugin Documentation The Registry Plugin provides model package management for VIPR inference. It introduces: - a package format (`manifest.yaml` + artifacts) - versioned model URIs - local and optional HuggingFace-backed registry backends - a generic model loader (`registry_loader`) that dispatches to domain builders ## Overview The registry solves a common reliability problem: weights and config files drift apart over time. With registry packages, inference references one immutable artifact version: ```text vipr:////@ ``` Examples: ```text vipr://local/reflectometry/NR-2layers-basic@v1.1.0 vipr://hf/reflectometry/fxc34ran@v1.0.0 ``` Backend behavior: - `local`: supports versions and aliases (`@production`) - `hf`: supports pinned versions only (no aliases) ## Backend configuration Local backend: - always enabled - storage root from `VIPR_REGISTRY_DIR` - default storage root: `~/.vipr/registry` HuggingFace backend: - enabled when `VIPR_HF_REPO` is set (example: `HW-SC/registry`) - uses HuggingFace cache via `huggingface_hub` - URI scheme: `vipr://hf/...` - resolves versions via `index.yaml` (version must exist there) - fetches snapshots from the repository `main` revision and selects packages by versioned path `///...` - uses `index.yaml` only to declare which package versions are available - if a requested version is missing, VIPR fails early with `Version not found in index` Example: ```bash export VIPR_HF_REPO=HW-SC/registry ``` Private HF repositories (authentication): - Public HF repos work without authentication. - Private HF repos require a valid HuggingFace token. - Interactive setup: ```bash hf auth login ``` - Non-interactive/CI setup (example): ```bash export HF_TOKEN= ``` ## Package contract A package contains: - `manifest.yaml` - artifact files (for example config and weights) Minimal example (`format_version: 1`): ```yaml format_version: 1 name: NR-2layers-basic version: v1.1.0 domain: reflectometry family: reflectorch description: "Reflectorch baseline model for neutron reflectometry" model_builder: handler: reflectorch_builder parameters: config_ref: configs/NR-2layers-basic-v1.yaml weights_ref: saved_models/model_NR-2layers-basic-v1.safetensors artifacts: - role: config path: configs/NR-2layers-basic-v1.yaml sha256: "" - role: weights path: saved_models/model_NR-2layers-basic-v1.safetensors format: safetensors sha256: "" metrics: {} tags: source: bundle-command family: reflectorch ``` Note: - Bundle commands write no `lineage` block by default. - To include absolute source paths for provenance, use `--include-lineage` on the domain bundle command. Optional contract fields can be added to the same manifest: ```yaml default_predictor: handler: reflectorch_predictor parameters: calc_pred_curve: true polish_prediction: true requires: preprocess_filters: - class: vipr_reflectometry.reflectorch.preprocess.interpolation_filter.InterpolationFilter method: preprocess_interpolate hook: INFERENCE_PREPROCESS_PRE_FILTER ``` Mapping to inference YAML: - `default_predictor.handler` -> `vipr.inference.prediction.handler` - `default_predictor.parameters` -> defaults for `vipr.inference.prediction.parameters` - `requires.preprocess_filters[*]` -> required enabled entries in `vipr.inference.filters[hook]` ## Loading flow `registry_loader` performs these steps in order: 1. Parse `load_model.parameters` (`model_uri`, optional `device`, optional `builder_parameters`). 2. Resolve URI via `app.registry.resolve()` to a local package directory. 3. Parse `manifest.yaml` via `ModelPackage.from_dir()`. 4. Validate artifact presence and SHA-256 checksums. 5. Validate that `model_builder.handler` is registered. 6. Resolve device override (load_model first, then prediction fallback). 7. Build an in-memory package copy with merged `builder_parameters` (manifest on disk is unchanged). 8. Apply predictor contract (`default_predictor`). 9. Validate required filters contract (`requires.preprocess_filters`). 10. Call the domain builder: `builder.build(package_dir, model_package_for_build)`. The object returned by `build()` is the executable model used by the predictor. ## `registry_loader` parameter overrides You can override builder behavior from inference YAML without editing `manifest.yaml`: ```yaml load_model: handler: registry_loader parameters: model_uri: vipr://hf/reflectometry/fxc34ran@v1.0.0 device: cpu builder_parameters: some_extra: value ``` Notes: - `builder_parameters` is merged into `model_builder.parameters` in memory. - The manifest file in the package is not modified. - Device precedence is: 1. `load_model.parameters.builder_parameters.device` 2. `load_model.parameters.device` 3. `prediction.parameters.device` ## CLI commands ### Generic registry commands ```bash vipr registry bundle --manifest manifest.yaml --source-dir ./my-package vipr registry list vipr registry show vipr://local/my-domain/my-model@v1.0.0 vipr registry promote vipr://local/my-domain/my-model@v1.0.0 production vipr registry sync vipr registry sync --domain reflectometry --name mc1-nsf ``` `vipr registry sync` prefetches packages from the configured HF backend (`VIPR_HF_REPO`) into local cache. ### Domain convenience commands Domain plugins can add helper commands to generate package layout/manifest. Reflectometry example: ```bash vipr reflectometry registry bundle-reflectorch --config ... --weights ... --name ... --version ... vipr reflectometry registry bundle-flow --config ... --weights ... --name ... --version ... vipr reflectometry registry bundle-panpe --config ... --weights ... --name ... --version ... ``` Implementation note: the reflectometry controller uses internal Cement handler label `reflectometry_registry` with CLI alias `registry`, so the public command path remains `vipr reflectometry registry ...`. ## Extension model A new domain does not need a custom controller for registry basics. Required: 1. implement/register a `ModelBuilderHandler` 2. create package directory + `manifest.yaml` 3. call generic `vipr registry bundle` Optional: - add domain convenience bundle commands ## Current scope Included: - local backend with immutable versions + alias promotion - optional HuggingFace backend (`VIPR_HF_REPO`) - package integrity/runtime checks - generic `registry_loader` - manifest contract checks (`default_predictor`, `requires.preprocess_filters`) - HF prefetch command (`vipr registry sync`) Not included yet: - training workflow - HF `push` / alias promotion via CLI (maintainer workflow is still manual) - runtime filter-chain injection from package metadata ## See also - [Registry Quickstart: Fe_Pt_DN example](registry-quickstart.md) — step-by-step walkthrough including local and HF usage.