La prochaine rencontre, qui aura lieu le 10 novembre à Mila, permettra d'explorer comment pouvons-nous collectivement développer, encadrer et déployer des systèmes agentiques performants, fiables et sécuritaires en connectant chercheur·euse·s académiques, expert·e·s industriel·le·s et praticien·ne·s.
Les inscriptions sont ouvertes pour notre programme de 4 mois à temps plein, à Mila, conçu spécifiquement pour les fondateurs et fondatrices de la deep tech issus de parcours d'élite en STIM. Vous avez jusqu'au 21 novembre 2026 pour appliquer!
Le processus de demandes de supervision à Mila est maintenant ouvert! Vous pouvez appliquer dès maintenant à la maitrise de recherche ou Doctorat et joindre notre grande communauté de chercheur·euse·s à Mila pour l’automne 2027. Date limite : 1er décembre 2026
Nous utilisons des témoins pour analyser le trafic et l’utilisation de notre site web, afin de personnaliser votre expérience. Vous pouvez désactiver ces technologies à tout moment, mais cela peut restreindre certaines fonctionnalités du site. Consultez notre Politique de protection de la vie privée pour en savoir plus.
Paramètre des cookies
Vous pouvez activer et désactiver les types de cookies que vous souhaitez accepter. Cependant certains choix que vous ferez pourraient affecter les services proposés sur nos sites (ex : suggestions, annonces personnalisées, etc.).
Cookies essentiels
Ces cookies sont nécessaires au fonctionnement du site et ne peuvent être désactivés. (Toujours actif)
Cookies analyse
Acceptez-vous l'utilisation de cookies pour mesurer l'audience de nos sites ?
Lecteur Multimédia
Acceptez-vous l'utilisation de cookies pour afficher et vous permettre de regarder les contenus vidéo hébergés par nos partenaires (YouTube, etc.) ?
Background: Large language models (LLMs) could assist in streamlining health care instrument validation. This is particularly relevant as tr… (voir plus)aditional instrument development is time-intensive and resource-intensive. Currently, there is a significant gap in specifically developed and validated trust instruments for pediatric emergency and surgical contexts. Such tools are key because trust is foundational to the patient family-physician relationship and is associated with improved care-seeking and treatment adherence. Objective: This study aimed (1) to develop new trust instruments and conduct their preliminary content validation confirming item relevance through a Synthetic Instrument Validation approach integrating human and LLM capabilities and (2) to evaluate appropriate use cases for LLMs in psychometric assessment. Methods: Two trust instruments were developed (patient family and physician versions). In phase 1, instruments underwent a 2-stage content validation process using parallel synthetic and human expert panels (5 patient family members, 3 physician synthetic experts, and 7 unique human experts). Synthetic panels consisted of 3 persona-prompted LLMs (Claude Sonnet 4, GPT-5, and Grok 4), with human panels serving as comparators. The scale-content validity index (S-CVI) threshold was set at ≥0.80, percent agreement on item relevancy was used to compare panels. In phase 2, LLM performance in quantitative research tasks was evaluated. The patient family instrument underwent Flesch-Kincaid assessment; both instruments underwent cosine similarity analyses using algorithmic and LLM-based methods. Results: In phase 1, percent agreement between panels for the patient family and physician instruments was 90.9% and 88.9% for dimensional validation and 90.9% and 100% for contextual validation. Fleiss κ for within-panel agreement ranged from poor to almost perfect; 0.20 (95% CI -0.23 to 0.63) to 1.00 (95% CI 1.00-1.00). All instrument subsections met or exceeded S-CVI thresholds across both rounds and panels. Phase 2 Flesch-Kincaid metrics converged across all 3 methods to target benchmark (mean Grade Level 8.1, SD 1.1; mean Reading Ease 60.1, SD 5.6), while reference readability was near target, showing convergence across implementations of the same formula. In contrast, cosine similarity analyses revealed significant LLM performance limitations; cosine similarity analyses were, therefore, abandoned in favor of the algorithmic method alone, which achieved a maximum cosine similarity of 0.83, indicating acceptable item distinctiveness overall. Conclusions: Persona-prompted LLMs effectively assisted preliminary content validation and may help reduce timelines from months to weeks, but showed limitations in quantitative computations. This suggests that, in this study, persona-prompted LLMs performed better in subjective rating tasks than in deterministic cosine similarity computations. These findings help illustrate task-dependent boundaries for LLM integration in psychometric research, necessitating selective human-LLM collaboration. While these instruments achieved preliminary content validity, they require full psychometric testing before they can be considered validated for clinical use. Synthetic Instrument Validation offers a feasible approach to preliminary health care instrument development while preserving key elements of content validation.
BACKGROUND: Canadian MD-PhD and Clinician Investigator Program (CIP) pathways both train physician-scientists, yet longitudinal data on grad… (voir plus)uate research productivity are limited. METHODS: We conducted a longitudinal study of McGill University MD-PhD and CIP graduates who completed these programs between 2000 and 2017, using sex- and year-matched MD-only graduates as controls. Outcomes were collected for publications indexed through to December 31, 2022, and included H-index, total publications, total citations, per-paper journal impact factor, per-paper citations, and authorship position (first, second, senior). Productivity was assessed across four time periods: (1) pre-medical/medical training, (2) residency, (3) post-residency/fellowship, and (4) independent practice. RESULTS: Among 549 graduates (MD-PhD = 31; CIP = 97; MD-only = 421), research productivity, research impact, and active research involvement (defined as ≥3 first- or senior-author papers in the prior 5 years) were similar between MD-PhD and CIP graduates; both exceeded MD-only graduates across all metrics. Within-group sex differences were not significant. MD-PhD graduates were more productive during pre-medical/medical training, whereas CIP graduates were comparatively more productive during residency. Sustained productivity in independent practice correlated with research engagement during medical school, residency, and fellowship. In both programs, graduates with active research involvement had higher authorship counts in the 10 years immediately following graduation from medical school. DISCUSSION: Research productivity and impact were comparable between MD-PhD and CIP graduates, and both groups exceeded MD-only peers across measured research metrics. Across both programs, graduates with active research involvement had higher early-career authorship counts, particularly during residency and post-residency/fellowship time periods. These findings provide descriptive benchmarking data for future studies of physician-scientist training pathways in Canada.
Trust is foundational to patient-physician relationships and is associated with improved care-seeking and adherence in primary care. However… (voir plus), validated trust instruments for pediatric emergency and surgical contexts are lacking, and traditional instrument development is slow and resource-intensive. Large language models (LLMs) could streamline the validation process by serving as scalable, systematic expert panel surrogates.
We developed four new trust assessment instruments: two for patient-families and two for physicians. Two-phase content validation was conducted using two parallel synthetic and human expert panels. Synthetic panels consisted of three persona-prompted LLMs (Claude Sonnet 4, GPT-5, Grok4). Human panels served as traditional comparators. Scale-Content Validity Index (S-CVI) and Fleiss’ kappa (k) acceptance thresholds were set at ≥0.80.
Combined human–synthetic expert panels revealed substantial inter-rater reliability across all instruments. Fleiss’ kvalues for dimensional validation were: patient-family = 0.84 (95% CI [0.72, 0.96]), physician = 0.87 (95% CI [0.72, 1.00]);contextual validation: patient-family = 0.83 (95% CI [0.73, 0.93]), physician = 0.88 (95% CI [0.80, 0.96]). All instruments exceeded S-CVI ≥0.80 thresholds across both validation phases.
Persona-prompted LLMs demonstrated comparable validity outcomes to human experts while accelerating validation timelines from months to weeks. Future research needs to evaluate this approach across psychometric testing phases.
This synthetic instrument validation methodology offers a scalable blueprint for healthcare measurement development, enabling faster creation of validated tools to support evidence-based patient care.