Jin Guo

Creating software tutorials involves developing accurate code examples and explanatory text that engages and informs the reader. Large Langu… (see more)age Models (LLMs) demonstrate a strong capacity to generate both text and code, but their potential to assist tutorial writing is unknown. By interviewing and observing seven experienced writers using OpenAI playground as an exploration environment, we uncover design opportunities for leveraging LLMs in software tutorial writing. Our findings reveal background research, resource creation, and maintaining quality standards as critical areas where LLMs could significantly assist writers. We observe how tutorial writers generated tutorial content while exploring LLMs’ capabilities, formulating prompts, verifying LLM outputs, and reflecting on interaction goals and strategies. Our observation highlights that the unpredictability of LLM outputs and unintuitive interface design contributed to skepticism about LLM’s utility. Informed by these results, we contribute recommendations for designing LLM-based tutorial writing tools to mitigate usability challenges and harness LLMs’ full potential.

2024-07-01

Conference on Designing Interactive Systems (published)

Motivating Users to Attend to Privacy: A Theory-Driven Design Study

Varun Shiri

Maggie Xiong

Jinghui Cheng

In modern technology environments, raising users’ privacy awareness is crucial. Existing efforts largely focused on privacy policy present… (see more)ation and failed to systematically address a radical challenge of user motivation for initiating privacy awareness. Leveraging the Protection Motivation Theory (PMT), we proposed design ideas and categories dedicated to motivating users to engage with privacy-related information. Using these design ideas, we created a conceptual prototype, enhancing the current App Store product page. Results from an online experiment and follow-up interviews showed that our design effectively motivated participants to attend to privacy issues, raising both the threat appraisal and coping appraisal, two main factors in PMT. Our work indicated that effective design should consider combining PMT components, calibrating information content, and integrating other design elements, such as visual cues and user familiarity. Overall, our study contributes valuable design considerations driven by the PMT to amplify the motivational aspect of privacy communication.

2024-07-01

Designing Interactive Systems Conference (published)

Why People Contribute Software Documentation

Deeksha M. Arya

Martin P. Robillard

2024-04-14

IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies (published)

Communicating Study Design Trade-offs in Software Engineering

Martin P. Robillard

Deeksha M. Arya

Neil Ernst

Maxime Lamothe

Mathieu Nassif

Nicole Novielli

Alexander Serebrenik

Igor Steinmacher

Klaas-Jan Stol

2024-03-02

ACM Transactions on Software Engineering and Methodology (published)

Properties and Styles of Software Technology Tutorials

Deeksha M. Arya

Martin P. Robillard

A large number of tutorials for popular software development technologies are available online, and those about the same technology vary wid… (see more)ely in their presentation. We studied the design of tutorials in the software documentation landscape for five popular programming languages: Java, C#, Python, Javascript, and Typescript. We investigated the extent to which tutorial pages, i.e. resources, differ and report statistics of variations in resource properties. We developed a framework for characterizing resources based on their distinguishing attributes, i.e. properties that vary widely for the resource, relative to other resources. Additionally, we propose that a resource can be represented by its resource style, i.e. the combination of its distinguishing attributes. We discuss three techniques for characterizing resources based on our framework, to capture notable and relevant content and presentation properties of tutorial pages. We apply these techniques on a data set of 2551 resources to validate that our framework identifies valid and interpretable styles. We contribute this framework for reasoning about the design of resources in the online software documentation landscape.

2024-02-01

IEEE Transactions on Software Engineering (published)

SUMMIT: Scaffolding Open Source Software Issue Discussion Through Summarization

Saskia Gilmer

Avinash Bhat

Shuvam Shah

Kevin Cherry

Jinghui Cheng

2023-10-04

Proceedings of the ACM on Human-Computer Interaction (published)

Aspirations and Practice of ML Model Documentation: Moving the Needle with Nudging and Traceability

Avinash Bhat

Austin Coursey

Grace Hu

Sixian Li

Nadia Nahar

Shurui Zhou

Christian Kästner

The documentation practice for machine-learned (ML) models often falls short of established practices for traditional software, which impede… (see more)s model accountability and inadvertently abets inappropriate or misuse of models. Recently, model cards, a proposal for model documentation, have attracted notable attention, but their impact on the actual practice is unclear. In this work, we systematically study the model documentation in the field and investigate how to encourage more responsible and accountable documentation practice. Our analysis of publicly available model cards reveals a substantial gap between the proposal and the practice. We then design a tool named DocML aiming to (1) nudge the data scientists to comply with the model cards proposal during the model development, especially the sections related to ethics, and (2) assess and manage the documentation quality. A lab study reveals the benefit of our tool towards long-term documentation quality and accountability.

2023-04-19

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (published)

Approach Intelligent Writing Assistants Usability with Seven Stages of Action

Avinash Bhat

Disha Shrivastava

2023-04-06

ArXiv (preprint)