MedVFS Runtime Architecture

Real-time Access vs Setup Mode

The setup (materialization) mode and the real-time (serve) mode are not alternatives — they are two patterns with different use cases that coexist within MedVFS.

Criterion	Setup (materialization)	Real-time (Serve)
Primary Use Case	ML Training, regulated studies	Clinical inference, monitoring
Reproducibility	✅ Total — immutable snapshot	❌ Data can change
Data Latency	High initially, zero during training	Low and continuous
Regulatory	✅ ICH E6, 21 CFR Part 11	⚠️ Requires additional audit trail
Network	Only in setup	Permanent
When to use	Training, validation, prospective studies	Production scoring, clinical alerts

Serve Mode Architecture (FUSE)

graph TD
    A[medvfs serve -c vfs.yaml] --> B[Register FileProvider adapters]
    B --> C[Mount FUSE on /data/vfs]
    
    C -->|Read /data/vfs/slides/001.svs| D[BlockManager]
    D --> E{Block in cache?}
    E -->|Yes| F[Immediate Return]
    E -->|No| G[Fetch Remote + Cache + Return]
    
    G --> H[PrefetchEngine]
    H -->|L1| I[Pre-load blocks N+1..N+3]
    H -->|L2| J[Parse TIFF IFD headers]
    H -->|L3| K[Warmup manifest on startup]

Integration with Nextflow

MedVFS and Nextflow are complementary:

• MedVFS orchestrates and materializes multimodal clinical data → CSVs + manifests.
• Nextflow orchestrates bioinformatic workflows on the materialized data.

Sequential Pattern:

1. MedVFS materializes the genomic manifest.
2. Nextflow consumes the generated samplesheet.csv (e.g., for nf-core/sarek).

ColabBio Internal vs External Hospital Systems

MedVFS uses the exact same vfs.yaml configuration for both ColabBio’s simulated components and real hospital systems. The difference lies only in the values (endpoints, secrets), not the structure.

ColabBio Component	Simulates	Port
cb-omop	Hospital OMOP CDM	5433
cb-fhir	Hospital HL7 FHIR R4	8080
cb-omero	Digital Pathology OMERO	4064/4080
cb-orthanc	Hospital PACS DICOM	8042
cb-minio	Genomic NFS/S3	9000

Ingestion from Public Health Systems

Public systems (like PADRIS or SERMAS) do not allow direct sandbox access. The pattern involves a dedicated ingestion pipeline:

graph LR
    A[Public System API] -->|Extract| B[cb-ingestion CronJob]
    B -->|Transform| C[OMOP CDM / FHIR R4]
    C -->|Load| D[(ColabBio Internal DBs)]
    D -->|medvfs setup| E[ML Sandbox]

Dataset Engine Pipeline

The MedVFS dataset engine transforms data scanned by adapters into an optimized ML layout:

1. FHIR/OMOP Bridge: Injects CSVs generated by FHIR/OMOP adapters directly into the VFS.
2. Patient Linker: Consolidates heterogeneous patient IDs into a canonical patient_id.
3. Dataset Splits: Deterministically partitions VFS files into train, val, and test splits.
4. Version Manifest: Generates a .dataset.json file with a deterministic SHA256 hash across all files for reproducibility.