The rise of real-world evidence alongside controlled clinical trials marks a major evolution in how we evaluate devices in everyday care. Over the past decade real world evidence has moved from concept to practice. FDA catalogued 90 device decisions that used real world evidence in 2012 to 2019, and a recent review found 117 devices included real world evidence in 2020 to July 2024, with 74 approvals supported by it.
Rather than replacing trials, real world data fills gaps on use, long-term performance and diverse populations. It strengthens benefit claims and reimbursement arguments by reflecting routine care.
What is Real World Data and Real World Evidence?
Real world data includes EHR notes, claims and billing data, registries, imaging archives, device logs, pharmacy data and patient generated inputs from wearables or apps. Real world evidence is the clinical evidence you derive from analysing this data to show safety, performance or value. FDA maintains current definitions and guidance.
Regulators and payers are paying closer attention to real world evidence because it reflects routine use, diverse populations and longer follow up. For software as a medical device, continuous learning systems and connected implants, real world evidence is often the only way to track drift, usability and durability over time.
Quick Reference Table: Sources and What You Can Learn
| Data Source | What You Can Learn Fast |
|---|---|
| EHR notes and discharge summaries | Adverse events and outcomes often missed by codes |
| Claims and billing files | Utilisation, readmission, long follow up at scale |
| Registries | Procedure detail, device exposure, longitudinal outcomes |
| Imaging archives | Objective device position and integrity over time |
| Device and sensor logs | Real use patterns and early malfunction signals |
| Patient reported outcomes | Function, pain, satisfaction between clinic visits |
Regulatory note: EU templates for PMCF plans and software clinical evaluation explicitly reference real world performance data and planned real world evidence analyses. Use these to anchor your approach in MDR language.
What People Worry About When They Hear "AI on Real World Data"
| Concern | Plain Language | How to Address | Where Milo Helps |
|---|---|---|---|
| Accuracy and bias | Will the model read notes and images correctly for every group | Validate against labelled samples and monitor subgroup performance | Data standards at capture, reviewer workflows, drift checks |
| Data quality | Are records complete and consistent | Profiling, missingness rules, reconciliation | Built-in QC at eCRF and ingestion, audit trails |
| Privacy and consent | Is secondary use lawful and traceable | Consent tracking, de-identification, DUA controls | GDPR and HIPAA features with consent logs |
| Security | Can data or models be tampered with | Role controls, encryption, immutable logs | Fine-grained access and activity trails |
| Explainability | Can we show why an alert was raised | Keep inputs and outputs and show logic | Separate AI workspace with originals and processed files saved together |
| Generalisation | Will it work in a new hospital | Site level validation and calibration | Site dashboards and performance monitors |
| Linkage errors | Are records matched to the right person | Conservative match rules and clerical review | Linkage tooling with review queue |
| Model drift | Will performance fade over time | Continuous tests and thresholds | Scheduled drift reports |
| Regulatory acceptance | Will this stand up in audits and PSUR | Traceability, relevance and reliability | Direct links to CER, risk registers and PSUR fields |
| Correlation vs causation | Are we overstating claims | Causal methods and clear language | Templates that distinguish signal finding from claims |
Where AI Adds Real Value on Real World Data
Unstructured Text Extraction
Find adverse events and outcomes hidden in free text like op notes and discharge summaries.
Computer Vision on Existing Images
Detect subtle changes around an implant without ordering new scans.
Time Series Analysis
Watch signals from wearables and connected devices to catch issues early.
Record Linkage
Bring hospital, pharmacy and payer data together to see the full picture.
Data Completeness
Use principled imputation and sensitivity checks to reduce bias from missing fields.
What People Worry About When They Hear "AI on Real World Data"
| Concern | Plain Language | How to Address | Where Milo Helps |
|---|---|---|---|
| Accuracy and bias | Will the model read notes and images correctly for every group | Validate against labelled samples and monitor subgroup performance | Data standards at capture, reviewer workflows, drift checks |
| Data quality | Are records complete and consistent | Profiling, missingness rules, reconciliation | Built-in QC at eCRF and ingestion, audit trails |
| Privacy and consent | Is secondary use lawful and traceable | Consent tracking, de-identification, DUA controls | GDPR and HIPAA features with consent logs |
| Security | Can data or models be tampered with | Role controls, encryption, immutable logs | Fine-grained access and activity trails |
| Explainability | Can we show why an alert was raised | Keep inputs and outputs and show logic | Separate AI workspace with originals and processed files saved together |
| Generalisation | Will it work in a new hospital | Site level validation and calibration | Site dashboards and performance monitors |
| Linkage errors | Are records matched to the right person | Conservative match rules and clerical review | Linkage tooling with review queue |
| Model drift | Will performance fade over time | Continuous tests and thresholds | Scheduled drift reports |
| Regulatory acceptance | Will this stand up in audits and PSUR | Traceability, relevance and reliability | Direct links to CER, risk registers and PSUR fields |
| Correlation vs causation | Are we overstating claims | Causal methods and clear language | Templates that distinguish signal finding from claims |
Where AI Adds Real Value on Real World Data
Unstructured Text Extraction
Find adverse events and outcomes hidden in free text like op notes and discharge summaries.
Computer Vision on Existing Images
Detect subtle changes around an implant without ordering new scans.
Time Series Analysis
Watch signals from wearables and connected devices to catch issues early.
Record Linkage
Bring hospital, pharmacy and payer data together to see the full picture.
Data Completeness
Use principled imputation and sensitivity checks to reduce bias from missing fields.
Milo Makes Real World Data Practical
Start with Lawful and Secure Use
Milo supports secondary use under the right lawful bases with consent tracking and de-identification that align to GDPR and HIPAA expectations.
Capture Clean Data at the Source
Structured eCRF and ePRO, device logs and imaging ingestion create analysis-ready feeds for your models.
Explore Before You Build Models
Use Milo's built-in pattern detection and signal exploration to ask useful questions early. This often surfaces safety trends and informs PMCF focus areas even if you never deploy a predictive model.
Keep AI Processing Separate from the Study Dataset
Work on notes and images in a secure workspace. Milo stores originals and processed outputs together so you can show exactly how findings were produced and moved into evidence reporting.
Prove Traceability into Your QMS
Milo links extracted insights and flagged issues directly to risk registers, PSUR templates and safety reports. This supports MDR Annex XIV and PMCF documentation practices.
U.S. Food and Drug Administration
A Simple Checklist You Can Steal
Define the regulatory question and the decision you want to support.
Map the real world data sources and assess quality and completeness.
Choose fit-for-purpose AI methods for text, images and time series.
Set accuracy, bias, and explainability targets and how you will test them.
Visual Framework: From Data to Evidence
Real World Data → Real World Evidence Flow
Real World Data
EHR, claims, registries, device logs, imaging, patient inputs
AI Analysis
NLP, computer vision, time series, linkage, imputation
Real World Evidence
Safety, performance, value for regulatory & reimbursement
AI Value Chain for Medical Device RWD
Text Extraction
Adverse events from notes
Computer Vision
Implant changes on scans
Time Series
Wearable signals
Record Linkage
Multi-source integration
Quality Control
Completeness & bias checks
Milo Healthcare Platform Ecosystem
Platform
eCRF & ePRO
Structured data capture at source
AI Workspace
Secure processing with full provenance
Consent Tracking
GDPR & HIPAA compliance
Quality Control
Built-in QC and audit trails
Drift Monitoring
Continuous performance checks
QMS Integration
Links to CER, PSUR, risk registers