Clinical Companion & Medical Tools

The ColabBio Clinical Companion is an agentic AI copilot built with LangGraph4j. It assists pathologists and oncologists during microscopic examination by executing multi-step clinical reasoning without hallucinations.

graph TD
    A["User Query in Pathology Viewer"] --> B["LangGraph4j StateGraph Engine"]
    B --> C{"Decision Node"}
    C -->|Tool Needed| D["Clinical MCP Registry (19 Tools)"]
    D --> E["Execute FHIR / OMERO / OMOP / DICOM"]
    E --> B
    C -->|Response Ready| F["WebSocket Event Stream"]
    F --> G["Viewer Canvas (Heatmaps, ROI Focus, Cards)"]

1. The 19 Deterministic Clinical Tools

The companion executes typed tools across four distinct clinical domains:

Domain	Tool Name	Function
EHR / FHIR	`fhir_search_patient`	Retrieve patient record and active conditions
	`fhir_get_observations`	Query laboratory values and biomarkers
	`fhir_get_diagnostic_reports`	Fetch past pathology and biopsy reports
Pathology WSI	`omero_find_images_by_patient`	Discover whole slide images for patient ID
	`omero_get_image_metadata`	Dimensions, resolution, channel layers, stain type
	`omero_render_roi_heatmap`	Stream model attention maps for ABMIL/HoverNet
OMOP CDM	`omop_get_conditions`	Fetch SNOMED standardized disease records
	`omop_get_drug_exposures`	Query RxNorm medication and chemotherapy history
	`omop_cohort_statistics`	Population-level baseline survivability analysis
Radiology PACS	`dicom_search_studies`	QIDO-RS search across CT, MRI, Mammography
	`dicom_get_series_metadata`	Slice thickness, contrast protocols, dimensions
	`dicom_resolve_wado_uri`	Generate MedVFS virtual streaming path for DICOM

2. Real-Time Viewer Event Streaming

Rather than returning plain text chat answers, the Clinical Companion emits structured CloudEvents over WebSockets:

colabbio.viewer.heatmap.show: Overlays an attention heat map over the gigapixel slide.
colabbio.viewer.roi.focus: Pans and zooms the virtual microscope to high-risk tumor cellular regions.
colabbio.viewer.patient.card: Renders an interactive sidebar card displaying EHR antecedents and genomic mutations.