Jin Guo

The Software Documentor Mindset

Deeksha M. Arya

Martin P. Robillard

Software technologies are used by programmers with diverse backgrounds. To fulfill programmers' need for information, enthusiasts contribute… (see more) numerous learning resources that vary in style and content, which act as documentation for the corresponding technology. We interviewed 26 volunteer documentation contributors, i.e. documentors, to understand why and how they create such documentation. From a qualitative analysis of our interviews, we identified a total of sixteen considerations that documentors have during the documentation contribution process, along three dimensions, namely motivations, topic selection techniques, and styling objectives. We grouped related considerations based on common underlying themes, to elicit five software documentor mindsets that occur during documentation contribution activities. We propose a structure of mindsets, and their associated considerations across the three dimensions, as a framework for reasoning about the documentation contribution process. This framework can inform information seeking as well as documentation creation tools about the context in which documentation was contributed.

2024-12-12

ArXiv (preprint)

The Software Documentor Mindset

Deeksha M. Arya

Martin P. Robillard

Software technologies are used by programmers with diverse backgrounds. To fulfill programmers' need for information, enthusiasts contribute… (see more) numerous learning resources that vary in style and content, which act as documentation for the corresponding technology. We interviewed 26 volunteer documentation contributors, i.e. documentors, to understand why and how they create such documentation. From a qualitative analysis of our interviews, we identified a total of sixteen considerations that documentors have during the documentation contribution process, along three dimensions, namely motivations, topic selection techniques, and styling objectives. We grouped related considerations based on common underlying themes, to elicit five software documentor mindsets that occur during documentation contribution activities. We propose a structure of mindsets, and their associated considerations across the three dimensions, as a framework for reasoning about the documentation contribution process. This framework can inform information seeking as well as documentation creation tools about the context in which documentation was contributed.

2024-12-12

ArXiv (preprint)

Do LLMs Meet the Needs of Software Tutorial Writers? Opportunities and Design Implications

Avinash Bhat

Disha Shrivastava

Creating software tutorials involves developing accurate code examples and explanatory text that engages and informs the reader. Large Langu… (see more)age Models (LLMs) demonstrate a strong capacity to generate both text and code, but their potential to assist tutorial writing is unknown. By interviewing and observing seven experienced writers using OpenAI playground as an exploration environment, we uncover design opportunities for leveraging LLMs in software tutorial writing. Our findings reveal background research, resource creation, and maintaining quality standards as critical areas where LLMs could significantly assist writers. We observe how tutorial writers generated tutorial content while exploring LLMs’ capabilities, formulating prompts, verifying LLM outputs, and reflecting on interaction goals and strategies. Our observation highlights that the unpredictability of LLM outputs and unintuitive interface design contributed to skepticism about LLM’s utility. Informed by these results, we contribute recommendations for designing LLM-based tutorial writing tools to mitigate usability challenges and harness LLMs’ full potential.

2024-07-01

Conference on Designing Interactive Systems (published)

Motivating Users to Attend to Privacy: A Theory-Driven Design Study

Varun Shiri

Maggie Xiong

Jinghui Cheng

In modern technology environments, raising users’ privacy awareness is crucial. Existing efforts largely focused on privacy policy present… (see more)ation and failed to systematically address a radical challenge of user motivation for initiating privacy awareness. Leveraging the Protection Motivation Theory (PMT), we proposed design ideas and categories dedicated to motivating users to engage with privacy-related information. Using these design ideas, we created a conceptual prototype, enhancing the current App Store product page. Results from an online experiment and follow-up interviews showed that our design effectively motivated participants to attend to privacy issues, raising both the threat appraisal and coping appraisal, two main factors in PMT. Our work indicated that effective design should consider combining PMT components, calibrating information content, and integrating other design elements, such as visual cues and user familiarity. Overall, our study contributes valuable design considerations driven by the PMT to amplify the motivational aspect of privacy communication.

2024-07-01

Designing Interactive Systems Conference (published)

Why People Contribute Software Documentation

Deeksha M. Arya

Martin P. Robillard

2024-04-14

IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies (published)

Why People Contribute Software Documentation

Deeksha M. Arya

Martin P. Robillard

Software technologies are used by a large population of programmers with diverse backgrounds. To fulfill their need for information, enthusi… (see more)asts contribute numerous learning resources that vary in style and content, and act as documentation for the corresponding technology. We interviewed 26 volunteer contributors to understand why they create such documentation. We surface five motivations our informants had for contributing documentation, including to overcome issues they had faced with documentation and to capture their own learning. Among other findings, our observations suggest that the unique experience and background of documentation contributors provides the opportunity to create documentation that caters to users who have information needs and preferences similar to that of the contributor. CCS CONCEPTS • Software and its engineering

2024-04-14

IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies (published)

Communicating Study Design Trade-offs in Software Engineering

Martin P. Robillard

Deeksha M. Arya

Neil Ernst

Maxime Lamothe

Mathieu Nassif

Nicole Novielli

Alexander Serebrenik

Igor Steinmacher

Klaas-Jan Stol

Reflecting on the limitations of a study is a crucial part of the research process. In software engineering studies, this reflection is typi… (see more)cally conveyed through discussions of study limitations or threats to validity. In current practice, such discussions seldom provide sufficient insight to understand the rationale for decisions taken before and during the study, and their implications. We revisit the practice of discussing study limitations and threats to validity and identify its weaknesses. We propose to refocus this practice of self-reflection to a discussion centered on the notion of trade-offs. We argue that documenting trade-offs allows researchers to clarify how the benefits of their study design decisions outweigh the costs of possible alternatives. We present guidelines for reporting trade-offs in a way that promotes a fair and dispassionate assessment of researchers’ work.

2024-03-02

ACM Transactions on Software Engineering and Methodology (published)

Communicating Study Design Trade-offs in Software Engineering

Martin P. Robillard

Deeksha M. Arya

Neil Ernst

Maxime Lamothe

Mathieu Nassif

Nicole Novielli

Alexander Serebrenik

Igor Steinmacher

Klaas-Jan Stol

2024-03-02

ACM Transactions on Software Engineering and Methodology (published)

Properties and Styles of Software Technology Tutorials

Deeksha M. Arya

Martin P. Robillard

A large number of tutorials for popular software development technologies are available online, and those about the same technology vary wid… (see more)ely in their presentation. We studied the design of tutorials in the software documentation landscape for five popular programming languages: Java, C#, Python, Javascript, and Typescript. We investigated the extent to which tutorial pages, i.e. resources, differ and report statistics of variations in resource properties. We developed a framework for characterizing resources based on their distinguishing attributes, i.e. properties that vary widely for the resource, relative to other resources. Additionally, we propose that a resource can be represented by its resource style, i.e. the combination of its distinguishing attributes. We discuss three techniques for characterizing resources based on our framework, to capture notable and relevant content and presentation properties of tutorial pages. We apply these techniques on a data set of 2551 resources to validate that our framework identifies valid and interpretable styles. We contribute this framework for reasoning about the design of resources in the online software documentation landscape.

2024-02-01

IEEE Transactions on Software Engineering (published)

Properties and Styles of Software Technology Tutorials

Deeksha M. Arya

Martin P. Robillard

A large number of tutorials for popular software development technologies are available online, and those about the same technology vary wid… (see more)ely in their presentation. We studied the design of tutorials in the software documentation landscape for five popular programming languages: Java, C#, Python, Javascript, and Typescript. We investigated the extent to which tutorial pages, i.e. resources, differ and report statistics of variations in resource properties. We developed a framework for characterizing resources based on their distinguishing attributes, i.e. properties that vary widely for the resource, relative to other resources. Additionally, we propose that a resource can be represented by its resource style, i.e. the combination of its distinguishing attributes. We discuss three techniques for characterizing resources based on our framework, to capture notable and relevant content and presentation properties of tutorial pages. We apply these techniques on a data set of 2551 resources to validate that our framework identifies valid and interpretable styles. We contribute this framework for reasoning about the design of resources in the online software documentation landscape.

2024-02-01

IEEE Transactions on Software Engineering (published)

SUMMIT: Scaffolding Open Source Software Issue Discussion Through Summarization

Saskia Gilmer

Avinash Bhat

Shuvam Shah

Kevin Cherry

Jinghui Cheng

2023-10-04

Proceedings of the ACM on Human-Computer Interaction (published)