Keiji AI LogoKeiji AI

Real-World Evidence guide

AI for real-world evidence: feasibility, cohorts and studies on claims and EHR data

How AI agents speed up real-world data feasibility and RWE studies: plain-language cohorts, OMOP, attrition tables, generated code and FDA guidance.

AI agents speed up a real-world data feasibility study by turning a plain-language question into a cohort definition, an attrition table and the SQL, Python or R code that produced it, on claims or EHR data you already hold. The time saved is the data engineering cycle between the epidemiologist's question and the first count. The work is only usable if every criterion, code list and step is visible and reproducible, which is also what FDA's real-world evidence guidance asks of sponsors. Buyers should check whose data the tool runs on, where the data stays, and whether they get the code.

What an AI agent changes in a real-world evidence study

A real-world evidence (RWE) study starts with a question from an epidemiologist, a medical affairs lead or an investigator. Traditionally that question goes to a data engineer or programmer, who writes the cohort logic, returns a count, and waits for the next round of changes. Each round can take days.

An AI agent sits in that gap. The researcher describes the population, exposure, outcome and time window in plain language. The agent proposes a plan, maps the criteria to codes in the data, writes the query, and returns the count with an attrition table. The researcher reviews and edits, and the agent reruns.

FDA defines real-world data as data relating to patient health status or the delivery of health care, routinely collected from sources such as EHRs, claims and registries. Real-world evidence is the clinical evidence derived from analyzing that data (FDA, Real-World Evidence). AI does not change those definitions. It changes how quickly a team can move from question to a defensible analysis.

Three kinds of products compete for this work, and they are easy to confuse:

  • Analytics software runs studies on data you bring. TrialMind is in this group.
  • Federated data networks supply patients from many institutions and a query tool, for example TriNetX, the Atropos Evidence Network and Truveta.
  • Data providers license curated datasets with analytics on top, for example Flatiron Health in oncology and IQVIA.

Feasibility: from question to attrition table

Feasibility asks one thing: how many eligible patients exist in a given data source? The answer that matters is not the final number. It is the attrition table that shows how each inclusion and exclusion criterion changed the count.

An AI-assisted feasibility run typically follows these steps:

  1. State the question: indication, line of therapy, biomarker, time window, data source.
  2. Review the proposed criteria and code lists before anything runs.
  3. Run the query and read the attrition table, one row per criterion.
  4. Adjust a criterion and rerun to see its effect on the count.
  5. Keep the final definition and code for the protocol.

FDA treats this step as part of study design. Its 2023 guidance says evaluating data sources for feasibility helps assess whether a source is fit for use and estimate statistical precision, without evaluating outcomes for treatment arms. It also asks sponsors to describe the data sources evaluated, including feasibility results, in the protocol or an appendix (Considerations for the Use of Real-World Data and Real-World Evidence, August 2023).

That has a practical consequence. Feasibility output should be saved and versioned, not thrown away after a quick count.

Plain-language cohort definition on claims, EHR and OMOP

Claims and EHR data record care through codes: ICD-10 diagnoses, procedure codes, drug codes and lab results. A cohort definition in plain language has to become a code-based algorithm before it can run.

FDA's EHR and claims guidance names this step. Studies should include a conceptual definition of each key variable, reflecting current medical thinking, and an operational definition that extracts it from the data. In claims and EHR studies the operational definition is often a code-based electronic algorithm (Real-World Data: Assessing Electronic Health Records and Medical Claims Data, final, July 2024).

A useful AI agent keeps both visible. You should be able to read the plain-language criterion, the code list it was mapped to, and the query that applies it.

Why OMOP helps

The OMOP Common Data Model, maintained by the OHDSI community, standardizes the structure and content of observational data, with standardized vocabularies that let analyses define exposures and outcomes consistently across claims and EHR sources (OHDSI, Data Standardization). For an AI agent, OMOP means one set of tables and concepts to reason over. FDA's guidance notes that common data models let an identical query run on multiple datasets in a distributed network.

OMOP is helpful but not required. Many organizations keep data in vendor or in-house schemas, and a tool should work with those too.

Unstructured data and AI-derived variables

If a variable is extracted from clinical notes with AI or NLP, FDA asks that the protocol specify the algorithm's assumptions and parameters, the data used to build it, whether it was supervised, and its validation metrics (FDA EHR and claims guidance).

Generated SQL, Python and R, and the study types they cover

Generated code is what separates an answer you can defend from an answer you have to trust. Ask for the SQL that built the cohort and the Python or R that ran the analysis. Your programmers should be able to read it, rerun it, and move it into a validated environment.

The same cohort definition usually feeds several study types:

Study typeTypical questionTypical output
FeasibilityHow many eligible patients exist?Count and attrition table
Comparative effectivenessHow does treatment A compare with B, or with an external control?Target trial emulation, covariate balance, sensitivity analyses
Treatment patternsWhich therapies, in what order?Lines of therapy, switching, adherence
Burden of illnessHow common and costly is the disease?Incidence, prevalence, outcomes, healthcare use
Safety signalsIs an adverse event associated with exposure?Disproportionality and longitudinal analyses
Eligibility criteria testingWhich trial criterion excludes the most patients?Per-criterion impact, comparison with the real-world population
Recurring cohort updatesWhat changed in the new data release?Same definition rerun, with differences by step

These are the seven study types TrialMind supports, each with an audit trail (TrialMind for Real-World Evidence).

Validation, reproducibility and FDA's real-world evidence guidance

FDA set up its Real-World Evidence Program under a 2018 framework that followed the 21st Century Cures Act. It now lists a set of guidances on EHR and claims data, registries, data standards, non-interventional studies and externally controlled trials (FDA, Real-World Evidence).

Several recommendations bear directly on how an AI tool should work:

  • Pre-specify before you analyze. Sponsors should share draft protocols and statistical analysis plans with FDA before conducting study analyses, and date-stamp any protocol revisions with a rationale (FDA, 2023). A tool that runs analyses instantly should still separate feasibility from outcome analysis.
  • Keep audit trails. Sponsors should maintain audit trails from extraction of real-world data through dataset retention, tracking user access, data changes, protocol changes and analyses performed (guidance text).
  • Be able to share patient-level data. Sponsors must be able to submit patient-level data for real-world data analyzed in a marketing application, and need agreements with data owners for that (guidance text).
  • Validate key variables. Operational definitions are imperfect and can misclassify patients, so sponsors should decide which variables need validation and how (EHR and claims guidance text).

FDA's draft guidance on non-interventional studies (March 2024) adds recommendations for sponsors using observational studies to support effectiveness or safety (FDA draft guidance).

In practice, ask any AI vendor to show three things for one study: the record of every step, the exact code, and a rerun on the same data that reproduces the same numbers.

Academic medical centers: studies on institutional data without code

Academic investigators face a familiar bottleneck. The institution has an EHR research warehouse, often in OMOP or i2b2, and a small data team serving many investigators. A principal investigator who wants a feasibility count before writing a protocol joins a queue.

Two kinds of tools help, and they solve different problems:

  • Federated networks such as TriNetX give investigators counts and analytics across many institutions. TriNetX describes an academic offering for research, teaching and grants, and says its data is the identified source for over $20M in NIH grants (TriNetX).
  • Analysis agents on institutional data let investigators run full studies on their own institution's records, inside the institution's environment.

TrialMind is the second kind. Investigators define cohorts and run analyses in plain language, with no code. Inputs include OMOP, i2b2, REDCap, EHR extracts, published literature and trial registries. It runs under the institution's existing IRB approval and records every definition and step (TrialMind for academic institutions).

For example, TrialMind answered an investigator question on bleeding readmissions with direct oral anticoagulants versus warfarin in atrial fibrillation, narrowing 34,211 patients to 9,983 and reporting an adjusted hazard ratio of 0.82 (95% CI 0.55–1.23). That run used the de-identified MIMIC hospital dataset, not a customer's data.

Licensing matters for academic buyers. TrialMind is licensed per research group, department or institution, with pooled seats, and can be paid from departmental funds, CTSA awards, training grants or a core-facility recharge model.

How TrialMind does it

TrialMind is Keiji AI's platform of AI agents for clinical research. For real-world evidence, it builds cohorts, runs the analysis inside your environment, and returns each result with its sources, methods and code.

  • Data in: claims, EHR, registry and genomic data the customer already licenses, in OMOP or the customer's own schemas. No data purchase is required.
  • Outputs: analysis plan, attrition funnel, tables and figures, and SQL, Python or R code.
  • Deployment: inside the customer's environment. The AI components work on metadata, and raw patient records are not moved.
  • Existing work: SQL, R, Python and cohort definitions a team has already validated import as they are.
  • Controls: SOC 2 Type I and Type II, ISO 27001, HIPAA with BAAs available. The Data Science Assistant v1.0.0 is qualified as a GxP computerized system with 21 CFR Part 11 and EU Annex 11 controls (Compliance). Formal submission analyses still run in the sponsor's validated statistics environment.

Case: a precision oncology diagnostics company

Twenty-plus data scientists at a precision oncology diagnostics company run oncology feasibility studies on their own database. They upload the attrition table they already use; TrialMind writes the SQL, Python or R against their database and returns the answer. The study runs in their environment and the data is not moved. Feasibility studies went from weeks to minutes (TrialMind for Real-World Evidence).

Published methods

The agent methods are published in DSWizard, on reliable multi-step data-science agents for biomedical research (Nature Biomedical Engineering, 2026).

Buyer checklist for AI real-world evidence tools

Use these questions with any vendor. Ask for a live run on a study you have already completed, so you can check the numbers.

  1. Do we need data, analysis, or both? If you already license data, an analytics layer may be enough. If not, compare networks and providers first.
  2. Whose data does it run on? Your own licensed data, the vendor's network, or a mix. Check your data license terms.
  3. Where does the data stay? Inside your environment, in a federated node, or in the vendor's research environment.
  4. Can I see the criteria-to-code mapping? Every plain-language criterion should show the code list and query behind it.
  5. Do I get an attrition table? One row per criterion, with counts.
  6. Do I get the code? SQL, Python or R that your programmers can read and rerun.
  7. Is it reproducible? The same definition on the same data release should give the same numbers, with a record of each step.
  8. How does it fit FDA expectations? Audit trails, pre-specified analysis plans, and access to patient-level data for submissions.
  9. Which data models does it support? OMOP, i2b2, vendor schemas, or your warehouse as it is.
  10. How is it licensed? Per seat, per study, or per group or institution, and whether data costs are separate.

Frequently asked questions

How can AI speed up a real-world data feasibility study?
By turning a plain-language question into criteria, code lists, a query and an attrition table in one pass, so the researcher can iterate without waiting on a data engineer. At a precision oncology diagnostics company, TrialMind took feasibility studies from weeks to minutes.
Which platforms let epidemiologists define cohorts in natural language on claims and EHR data?
TrialMind does this on the claims and EHR data you already license. Flatiron Telescope describes plain-language cohort building on Flatiron's oncology data, and Atropos Health's ChatRWD takes questions in a chat interface.
Do I need OMOP to use an AI agent for RWE?
No, but it helps. OMOP standardizes tables and vocabularies so the same query works across sources; TrialMind works with OMOP or your own schemas.
Can AI-generated RWE support a regulatory submission?
The tool must support what FDA asks for: pre-specified protocols and analysis plans, audit trails, validated variable definitions and access to patient-level data. Formal submission analyses usually still run in the sponsor's validated statistics environment.
What AI tools help academic researchers analyze EHR data without coding?
TrialMind lets investigators define cohorts and run analyses in plain language on their institution's OMOP, i2b2, REDCap or EHR extract data, inside the institution's environment and under its IRB approval. Federated networks such as TriNetX offer no-code querying across many institutions.
What is the difference between TrialMind and a data network such as TriNetX?
TriNetX supplies de-identified patient data from a network of clinical sites with a query platform. TrialMind supplies no data; it runs studies on data you already hold.

See TrialMind on your own question

A 30-minute walkthrough on a study like yours, with sources and code you can check.