Portrait de Ian Arawjo

Ian Arawjo

Membre académique associé
Professeur adjoint, Université de Montréal, Département d'informatique et de recherche opérationnelle

Biographie

Ian Arawjo est professeur adjoint au Département d'informatique et de recherche opérationnelle (DIRO) de l'Université de Montréal. Il détient un doctorat en sciences de l'information de l'Université Cornell, réalisé sous la supervision du professeur Tapan Parikh.

Sa thèse portait sur l'intersection de la programmation informatique et de la culture, explorant la programmation en tant que pratique sociale et culturelle. Il a acquis de l'expérience dans l'application d'une vaste gamme de méthodes liées aux interfaces homme-machine (IHM), allant du travail de terrain ethnographique à la recherche archivistique, en passant par le développement de systèmes novateurs (utilisés par des milliers de personnes) et la réalisation d'études de convivialité. Actuellement, il travaille sur des projets au carrefour de la programmation, de l'IA et de l'IHM, notamment sur la manière dont les nouvelles capacités de l'IA peuvent nous aider à réimaginer la pratique de la programmation.

Il travaille également sur l'évaluation de grands modèles de langage (LLM), à travers des projets en code source libre à forte visibilité tels que ChainForge. Les articles auxquels il a contribué comme premier auteur ont remporté des prix lors de grandes conférences portant sur l’IHM, notamment la Conference on Human Factors in Computing Systems (CHI), la Conference on Computer-Supported Cooperative Work and Social Computing (CSCW) et le Symposium on User Interface Software and Technology (UIST).

Étudiants actuels

Maîtrise professionnelle - UdeM

Publications

Dynamic Abstractions: Building the Next Generation of Cognitive Tools and Interfaces
Sangho Suh
Hai Dang
Ryan Yen
Josh M. Pollock
Rubaiat Habib Kazi
Hariharan Subramonyam
Jingyi Li
Nazmus Saquib
Arvind Satyanarayan
ChainBuddy: An AI Agent System for Generating LLM Pipelines
Jingyue Zhang
Imagining a Future of Designing with AI: Dynamic Grounding, Constructive Negotiation, and Sustainable Motivation
Priyan Vaithilingam
Elena L. Glassman
An AI-Resilient Text Rendering Technique for Reading and Skimming Documents
Ziwei Gu
Kenneth Li
Jonathan K. Kummerfeld
Elena L. Glassman
ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing
Chelse Swoopes
Priyan Vaithilingam
Martin Wattenberg
Elena L. Glassman
Evaluating outputs of large language models (LLMs) is challenging, requiring making -- and making sense of -- many responses. Yet tools that… (voir plus) go beyond basic prompting tend to require knowledge of programming APIs, focus on narrow domains, or are closed-source. We present ChainForge, an open-source visual toolkit for prompt engineering and on-demand hypothesis testing of text generation LLMs. ChainForge provides a graphical interface for comparison of responses across models and prompt variations. Our system was designed to support three tasks: model selection, prompt template design, and hypothesis testing (e.g., auditing). We released ChainForge early in its development and iterated on its design with academics and online users. Through in-lab and interview studies, we find that a range of people could use ChainForge to investigate hypotheses that matter to them, including in real-world settings. We identify three modes of prompt engineering and LLM hypothesis testing: opportunistic exploration, limited evaluation, and iterative refinement.
Schrödinger's Update: User Perceptions of Uncertainties in Proprietary Large Language Model Updates
Zilin Ma
Yiyang Mei
Krzysztof Z. Gajos
Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences
Shreya Shankar
J.D. Zamfirescu-Pereira
Bjorn Hartmann
Aditya G Parameswaran
Antagonistic AI
Alice Cai
Elena L. Glassman
ChainBuddy: An AI-assisted Agent System for Helping Users Set up LLM Pipelines
Jingyue Zhang
ChainForge: An open-source visual programming environment for prompt engineering
Priyan Vaithilingam
Martin Wattenberg
Elena Glassman
Notational Programming for Notebook Environments: A Case Study with Quantum Circuits
Anthony DeArmas
Michael Roberts
Shrutarshi Basu
Tapan Parikh
We articulate a vision for computer programming that includes pen-based computing, a paradigm we term notational programming. Notational pro… (voir plus)gramming blurs contexts: certain typewritten variables can be referenced in handwritten notation and vice-versa. To illustrate this paradigm, we developed an extension, Notate, to computational notebooks which allows users to open drawing canvases within lines of code. As a case study, we explore quantum programming and designed a notation, Qaw, that extends quantum circuit notation with abstraction features, such as variable-sized wire bundles and recursion. Results from a usability study with novices suggest that users find our core interaction of implicit cross-context references intuitive, but suggests further improvements to debugging infrastructure, interface design, and recognition rates. Throughout, we discuss questions raised by the notational paradigm, including a shift from ‘recognition’ of notations to ‘reconfiguration’ of practices and values around programming, and from ‘sketching’ to writing and drawing, or what we call ‘notating.’
To Write Code: The Cultural Fabrication of Programming Notation and Practice