J Med Internet Res. 2026 Aug 19;28:e96347. doi: 10.2196/96347.
ABSTRACT
BACKGROUND: The Registry of Stroke Care Quality (RES-Q) is a health care quality improvement platform used globally. RES-Q collects structured quality-of-care data for patients with stroke, requiring clinicians to manually extract information from electronic health records or documents such as discharge summaries. This process is essential but time-consuming, particularly given the variability, length, and semistructured nature of clinical reports.
OBJECTIVE: This study aimed to develop and evaluate a multilingual Evidence-Based Question-Answering framework that identifies supporting text spans in clinical reports of patients with stroke and proposes answer suggestions for structured clinical forms, with the goal of reducing clinician workload while preserving full human oversight.
METHODS: We conduct a multilingual study using more than 1500 pseudonymized stroke discharge summaries in 5 languages, annotated with question-evidence-answer triplets. Encoder-based language models are used to extract evidence spans from the reports, while generative language models are used to predict normalized form answers based on the extracted evidences. We compare multiple training strategies, such as (1) models trained on reports in a single target language, (2) models trained jointly on reports in different languages, and (3) models trained on original reports combined with cross-lingual data augmentations. We evaluate performance on Evidence Extraction, Answer Prediction, and end-to-end Evidence-Based Question Answering across the 5 languages.
RESULTS: The presented Evidence-Based Question-Answering system achieves 88% end-to-end accuracy in form filling across 5 languages (77% for patient-specific questions and 95% for default or unverifiable items). Evidence Extraction is the primary bottleneck, reaching 85% F and 79% exact match, whereas Answer Prediction based on extracted evidences is more stable, achieving 95% accuracy. The performance varies by question type, and cross-lingual training generally reduces Evidence Extraction performance but has little effect on Answer Prediction. Model performance is influenced more by reporting practices and dataset characteristics than by language itself.
CONCLUSIONS: Evidence-Based Question Answering over multilingual stroke discharge summaries enables human-in-the-loop validation and effective answer prediction with moderate computational resources. Evidence Extraction is the main bottleneck, while Answer Prediction is robust across languages and model sizes. The approach supports structured data collection, although generalization to new languages requires target-language training data.
PMID:42617111 | DOI:10.2196/96347