Tenant Onboarding

Welcome to ColabBio. If you are reading this, you are integrating a new research pipeline (e.g., an AI model, a genomics pipeline, or data analysis) into the platform.

Your project will run as a Tenant Project: meaning you have total freedom over your code and dependencies, but you leverage ColabBio’s secure, massive infrastructure to access data (via FHIR, OMOP, and MedVFS).

This technical guide explains step-by-step how to design, develop, and operate your own Tenant Project based on platform best practices.

1. Design Phase: Data Abstraction

The golden rule of ColabBio is: Your Git repository only contains code and configuration. Massive clinical and imaging data (Terabytes) reside in MedVFS or storage systems managed by the platform.

Standard Repository Structure

You must organize your repository as follows:

my-new-tenant/
├── src/                      # Your intact Python/R/C++ code.
├── Dockerfile                # Dependencies (TensorFlow, PyTorch, etc.).
├── datasets/                 # Data infrastructure configuration.
│   ├── medvfs-local.yaml     # (PVC) Request access to massive data.
│   └── manifest.csv          # Patient IDs (Referential metadata).
├── pipelines/                # Execution workflows.
│   └── my-job-k8s.yaml       # Kubernetes Job to train or infer.
└── ingestion/                # (Optional) Scripts to populate FHIR/OMOP.

2. MedVFS Abstraction (FUSE-First)

Your AI should never read data from the network, S3, or API calls. It should always read from standard POSIX local file system paths (e.g., /data/vfs/images).

In ColabBio, we achieve this by injecting our intelligent daemon MedVFS (FUSE) as a sidecar into your training. Your only job is to create a vfs.yaml file in your repository defining where the clinical sources are (FHIR, OMOP, OMERO), and our daemon will build a virtual file system in real time (translating read() calls into API requests).

Legacy Note: If your cluster restricts FUSE mounts, ColabBio natively supports mounting standard Kubernetes PersistentVolumeClaims (PVCs) through the MedVFS setup mode.

The Real Life Cycle (ETL + MedVFS)

MedVFS assumes metadata and images already exist on the servers (FHIR, OMERO). When a hospital gives you chaotic raw data for the first time, your Tenant’s full lifecycle will be 3 phases:

1. Ingestion Phase (ETL Loaders in Nextflow): Your pipeline invokes transformation scripts to read raw CSVs, populate the FHIR database, and register images in OMERO.
2. Materialization Phase (medvfs setup): The MedVFS daemon connects to the now-populated databases, downloads manifests, partitions training (Train/Test), and prepares the virtual structure.
3. Execution Phase (medvfs serve): The AI starts by mounting the daemon. The AI only sees clean local files and directories, unaware of all prior work.

3. Technical Phase: Infrastructure Integration

Requesting Volumes (MedVFS PVC)

To access a massive dataset (e.g., Histological Images) in legacy/setup mode, create a Kubernetes manifest in datasets/ requesting the volume.

datasets/medvfs-pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-tenant-data-pvc
spec:
  accessModes:
    - ReadOnlyMany
  resources:
    requests:
      storage: 100Gi # Size is virtual if hostPath or NFS

Execution Job Creation

Package your code into a Docker image. Then, create a Job in pipelines/ that executes your image and mounts the previous PVC.

pipelines/my-job-k8s.yaml

apiVersion: batch/v1
kind: Job
metadata:
  name: my-tenant-job
spec:
  template:
    spec:
      containers:
        - name: ai-worker
          image: my-image:latest
          volumeMounts:
            - name: data-volume
              mountPath: /mnt/data
              readOnly: true
      volumes:
        - name: data-volume
          persistentVolumeClaim:
            claimName: my-tenant-data-pvc

4. Operation Phase: Traceability and Metadata (FHIR)

If your Tenant project handles data that other hospital systems need to know about (e.g., generating predictions, or bringing an external patient dataset), you must ingest that metadata into HAPI FHIR or OMOP CDM.

Ingestion Script (The FHIR Bridge)

Located in ingestion/, you must have a Python script that takes your patient identifiers (manifest.csv) and uses the cluster’s internal FHIR REST API (http://colabbio-hapi-fhir...) to register:

1. The Patient resource.
2. Associated resources (e.g., DocumentReference linking the WSI file path).

Operational Example:

# Launch ingestion
kubectl apply -f pipelines/ingestion-job.yaml
# Launch training
kubectl apply -f pipelines/my-job-k8s.yaml