Keiji AI LogoKeiji AI

TrialPanorama · npj Digital Medicine 2025

TrialPanorama: a clinical trial dataset and benchmark

A structured database of global clinical trial records, a benchmark of eight clinical research tasks, and an 8-billion-parameter (8B) model trained on the data that outperforms general-purpose models of up to 70 billion parameters (70B) on all eight.

Read the paperResults
Global trial registries and sources
15
Global trial registries and sources
Clinical research tasks
8
Clinical research tasks
Tasks where the 8B model outperforms 70B models
8 of 8
Tasks where the 8B model outperforms 70B models

TrialPanorama in the product: semaglutide, tirzepatide and the SURMOUNT-5 head-to-head trial. More TrialPanorama demos ›

Zifeng Wang1,*,†, Jiacheng Lin2,*, Qiao Jin3,*, Junyi Gao4,5, Jathurshan Pradeepkumar2, Pengcheng Jiang2, Zhiyong Lu3, Jimeng Sun1,2,†

1Keiji AI Inc. 2University of Illinois Urbana-Champaign 3National Institutes of Health 4University of Edinburgh 5Health Data Research UK
*Equal contribution †Correspondence: zifeng@keiji.ai, jimeng@keiji.ai

Paper ›Code ›Database ›Benchmark ›

What TrialPanorama contains

Records from fifteen sources, normalized and linked to biomedical ontologies and the literature.

TrialPanorama overview and end-to-end processing workflow
Figure 1. a, Data collected from fifteen global sources, including PubMed, ClinicalTrials.gov, and international registries; LLM-based extraction, hybrid rules, and ontology-grounded normalization derive unified study attributes, matched to DrugBank, MedDRA, RxNorm, MeSH and others, and stored as a relational schema and a knowledge graph. b, Distribution of data sources, with a country-level breakdown of non-US registries. c, Most frequent drug entities. d, Most frequent disease and condition entities after harmonization.
Abstract

Developing artificial intelligence (AI) for clinical research requires a comprehensive data foundation that supports model training and rigorous evaluation. Here, we introduce TrialPanorama, a large-scale structured resource that aggregates 1.6M clinical trial records from fifteen global registries and links them with biomedical ontologies and associated literature. To demonstrate its utility, we build a pipeline that constructs 152K training and testing samples for eight key clinical research tasks. Three tasks support systematic review workflows, including study search, study screening, and evidence summarization. Five tasks focus on trial design and optimization, including arm design, eligibility criteria design, endpoint selection, sample size estimation, and trial completion assessment and rationalization. Benchmarking cutting-edge large language models (LLMs) reveals that generic LLMs have limited capability in clinical reasoning. In contrast, an 8B LLM we developed on TrialPanorama using supervised finetuning and reinforcement learning wins over the 70B generic counterparts in all eight tasks, with a relative improvement of 73.7%, 67.6%, 38.4%, 37.8%, 26.5%, 20.7%, 20.0%, 18.1%, and 5.2%, respectively. We envision TrialPanorama provides a solid foundation for future scaling of AI for clinical research.

Database structure: ten tables in four groups

Studies, conditions, drugs, endpoints, disposition, relations, biomarkers, outcomes, adverse events, and results.

Trial attributes
Core metadata: title, brief summary, sponsor, start year, recruitment status. Table: studies.
Trial protocols
Setup and design: tested drugs, targeted conditions, patient group allocations. Tables: conditions, drugs, endpoints, disposition, biomarkers.
Trial results
Reported findings: outcomes, adverse events, efficacy results. Tables: outcomes, adverse events, results.
Study-level links
Registry records mapped to publications, and reviews mapped to the studies they aggregate. Table: relations.
Browse the database

Benchmark tasks

Eight tasks across two workflows, systematic review and trial design, built from real trial records and review corpora to test clinical reasoning rather than general language ability.

Systematic review

  • Study search

    Retrieve relevant clinical trials for a research question.

  • Study screening

    Decide whether a study meets specific inclusion criteria.

  • Evidence summarization

    Synthesize findings across related studies.

Trial design

  • Arm design

    Intervention and control arms with appropriate dosages.

  • Eligibility criteria

    Inclusion and exclusion criteria for participants.

  • Endpoint selection

    Appropriate primary and secondary endpoints.

  • Sample size estimation

    Participant numbers required for statistical power.

  • Trial completion

    Likelihood of successful completion, with a rationale.

Workflow turning clinical trial records into supervised training examples
Figure 2. Structured and semi-structured trial records become standardized instruction–input–answer triples for tasks such as sample size estimation, criteria design, and endpoint design.
Browse the benchmark

Results

General-purpose models of up to 70 billion parameters struggle on clinically grounded tasks. An 8-billion-parameter model adapted on TrialPanorama outperforms them on all eight.

Benchmark overview and evaluation results across review and trial design tasks
Figure 3. b, Systematic review tasks and dataset scale. c, Overall performance of representative LLMs across review and trial design benchmarks. d, Trial design tasks with training and evaluation set sizes.
What the results show

Across all eight tasks, generic large language models show limited clinical reasoning. Even models up to 70B parameters struggle with tasks that need domain knowledge, structured trial reasoning, or several clinical constraints at once — scale alone is not enough.

Qwen3-8B-TP*, adapted on TrialPanorama with supervised fine-tuning and reinforcement learning with verifiable reward, outperforms every generic counterpart on every task, with relative improvements of 73.7%, 67.6%, 38.4%, 37.8%, 26.5%, 20.7%, 20.0%, 18.1%, and 5.2%.

Domain-aligned post-training on high-quality, task-aligned trial data matters more than brute-force scaling for clinical research.

Cite this work
@article{wang2025trialpanorama,
  title     = {Developing Large Language Models for Clinical Research Using One Million Clinical Trials},
  author    = {Wang, Zifeng and Lin, Jiacheng and Jin, Qiao and Gao, Junyi and Pradeepkumar, Jathurshan and Jiang, Pengcheng and Lu, Zhiyong and Sun, Jimeng},
  year      = {2025},
}

Paper, data, and code

Database, benchmark, and code are public.

Read the paperAll publications