Proceedings/Proceedings -- IEEE Symposium on Visual Languages and Human-Centric Computing

Designed as a wide-ranging guide to Domain Specific Languages (DSLs) and how to approach building them, this book covers a variety of different techniques available for DSLs. The goal is to provide readers with enough information to make an informed choice about whether or not to use a DSL and what kinds of DSL techniques to employ. Part I is a 150-page narrative overview that gives you a broad understanding of general principles. The reference material in Parts II through VI provides the details and examples you willneed to get started using the various techniques discussed. Both internal and external DSL topics are covered, in addition to alternative computational models and code generation. Although the general principles and patterns presented can be used with whatever programming language you happen to be using, most of the examples are in Java or C#.

Research is emerging on how end users can correct mistakes their intelligent agents make, but before users can correctly “debug” an intelligent agent, they need some degree of understanding of how it works. In this paper we consider ways intelligent agents should explain themselves to end users, especially focusing on how the soundness and completeness of the explanations impacts the fidelity of end users' mental models. Our findings suggest that completeness is more important than soundness: increasing completeness via certain information types helped participants' mental models and, surprisingly, their perception of the cost/benefit tradeoff of attending to the explanations. We also found that oversimplification, as per many commercial agents, can be a problem: when soundness was very low, participants experienced more mental demand and lost trust in the explanations, thereby reducing the likelihood that users will pay attention to such explanations at all.

In 1995, Boehm predicted that by 2005, there would be "55 million performers" of "end user programming" in the United States. The original context and method which generated this number had two weaknesses, both of which we address. First, it relies on undocumented, judgment-based factors to estimate the number of end user programmers based on the total number of end users; we address this weakness by identifying specific end user sub-populations and then estimating their sizes. Second, Boehm's estimate relies on additional undocumented, judgment-based factors to adjust for rising computer usage rates; we address this weakness by integrating fresh Bureau of Labor Statistics (BLS) data and projections as well as a richer estimation method. With these improvements to Boehm's method, we estimate that in 2012 there will be 90 million end users in American workplaces. Of these, we anticipate that over 55 million will use spreadsheets or databases (and therefore may potentially program), while over 13 million will describe themselves as programmers, compared to BLS projections of fewer than 3 million professional programmers. We have validated our improved method by generating estimates for 2001 and 2003, then verifying that our estimates are consistent with existing estimates from other sources.

Designers are skilled at sketching and prototyping the look of interfaces, but to explore various behaviors (what the interface does in response to input) typically requires programming using Javascript, ActionScript for Flash, or other languages. In our survey of 259 designers, 86% reported that the behavior is more difficult to prototype than the appearance. Often (78% of the time), designing the behavior requires collaborating with developers, but 76% of designers reported that communicatin1g the behavior to developers was more difficult than the appearance. Other results include that annotations such as arrows and paragraphs of text are used on top of sketches and storyboards to explain behaviors, and designers want to explore multiple versions of behaviors, but today’s tools make this difficult. The results provide new ideas for future tools.

Large software projects often require a programmer to make changes to unfamiliar source code. This paper describes a set of tools, called Team Tracks, designed to ease program comprehension by showing the source code navigation patterns of fellow development team members. One technique shows a list of related items, given that the user is viewing a given method or class. Another technique shows the favorite classes, by showing a class hierarchy view that hides less frequently visited classes, methods, and members. Two user studies, a laboratory study and a field study, were run to evaluate the effectiveness of these techniques. The results of the two studies demonstrate that sharing navigation data can improve program comprehension and is subjectively preferred by users.

In open-ended tasks where a program's behavior cannot be specified in advance, exploratory programming is a key practice in which programmers actively experiment with different possibilities using code. Exploratory programming is highly relevant today to a variety of professional and end-user programmer domains, including prototyping, learning through play, digital art, and data science. However, prior research has largely lacked clarity on what exploratory programming is, and what behaviors are characteristic of this practice. Drawing on this data and prior literature, we provide an organized description of what exploratory programming has meant historically and a framework of four dimensions for studying exploratory programming tasks: (1) applications, (2) required code quality, (3) ease or difficulty of exploration, and (4) the exploratory process. This provides a basis for better analyzing tool support for exploratory programming.

Although there are many systems designed to engage people in programming, few explicitly teach the subject, expecting learners to acquire the necessary skills on their own as they create programs from scratch. We present a principled approach to teach programming using a debugging game called Gidget, which was created using a unique set of seven design principles. A total of 44 teens played it via a lab study and two summer camps. Principle by principle, the results revealed strengths, problems, and open questions for the seven principles. Taken together, the results were very encouraging: learners were able to program with conditionals, loops, and other programming concepts after using the game for just 5 hours.

Intelligent systems are gaining in popularity and receiving increased media attention, but little is known about how people actually go about developing them. In this paper, we attempt to fill this gap through a set of field interviews that investigate how people develop intelligent systems that incorporate machine learning algorithms. The developers we interviewed were experienced at working with machine learning algorithms and dealing with the large amounts of data needed to develop intelligent systems. Despite their level of experience, we learned that they struggle to establish a repeatable process. They described problems with each step of the processes they perform, as well as cross-cutting issues that pervade multiple steps of their processes. The unique difficulties that developers like these face seem to point to a need for software engineering advances that address such machine learning systems, and we conclude by discussing this need and some of its implications.

Stack Overflow is a learning community for software developers to share and solve programming problems with each other. However, women are often deterred from contributing questions or answers. Research external to programming communities suggest the presence of peers can increase activity from underrepresented users in unfamiliar spaces. To investigate the concept of peer parity, we studied how women participate on Stack Overflow and if the presence of more women on a thread enhanced their activity. We found that women who encountered other women were more likely to engage sooner than those who did not. We discuss how these findings can support women in programming communities through peer mentorship and increase engagement.

Visual programming languages can be used to make computer science more accessible to a broad range of students. The evaluative focus of current research in the area of visual languages for educational purposes primarily aims to better understand motivational benefits as compared to traditional programming languages. Often these visual languages claim to teach students computational thinking concepts; however, although the evaluations show that students may exhibit more enthusiasm, it is not always clear what computational thinking concepts students have actually learned. In this paper we attempt to develop a visual semantic evaluation tool for student-created games and simulations that goes towards depicting the computational thinking concepts implemented by the students. Through semantically analyzing a given student's created projects over time, this visual evaluation tool, called the Computational Thinking Pattern (CTP) graph, can possibly indicate the existence of computational thinking transfer from games to science simulations.

Although birthed in the era of teletypes, the command line shell survived the graphical interface revolution of the 1980's and lives on in modern desktop operating systems. The command line provides access to powerful functionality not otherwise exposed on the computer, but requires users to recall textual syntax and carefully scour documentation. In contrast, graphical interfaces let users organically discover and invoke possible actions through widgets and menus. To better expose the power of the command line, we demonstrate a mechanism for automatically creating graphical interfaces for command line tools by translating their documentation (in the form of man pages) into interface specifications via AI. Using these specifications, our user-facing system, called GUIde, presents the command options to the user graphically. We evaluate the generated interfaces on a corpus of commands to show to what degree GUIde offers thorough graphical interfaces for users' real-world command line tasks.

Computer programming is an essential skill for the 21st-century workforce. However, mastering programming is challenging, especially for beginners. Recently, digital tools have become popular in supporting programming education. However, creating engaging programming materials remains difficult due to the many factors that need consideration. We introduce Codigen, an interactive tool powered by large language models (LLMs) that integrates visual authoring and prompt engineering to support programming education with various learning strategies. A user study with 29 participants showed that Codigen can assist teachers in generating quality programming materials. Statistical analysis showed that Codigen supports learners’ confidence and learning outcomes in programming. The perceived utility and engagement with the tool were also statistically significant. We discuss our findings to guide the design and use of LLM-powered authoring tools for educational purposes.

Usability evaluations are essential for ensuring that modern interfaces meet user needs, yet traditional heuristic evaluations by human experts can be time-consuming and subjective, especially early in development. This paper investigates whether large language models (LLMs) can provide reliable and consistent heuristic assessments at the development stage. By applying Jakob Nielsen’s ten usability heuristics to thirty open-source websites, we generated over 850 heuristic evaluations in three independent evaluations per site using a pipeline of OpenAI’s GPT-4o. For issue detection, the model demonstrated moderate consistency, with an average pairwise Cohen’s Kappa of 0.50 and an exact agreement of 84%. Severity judgments showed more variability: weighted Cohen’s Kappa averaged 0.63, but exact agreement was just 56%, and Krippendorff’s Alpha was near zero. These results suggest that while GPT-4o can produce internally consistent evaluations, especially for identifying the presence of usability issues, its ability to judge severity varies and requires human oversight in practice. Our findings highlight the feasibility and limitations of using LLMs for early-stage, automated usability testing, and offer a foundation for improving consistency in automated User Experience (UX) evaluation. To the best of our knowledge, our work provides one of the first quantitative inter-rater reliability analyses of automated heuristic evaluation and highlights methods for improving model consistency.

Technological advancements over the years have led to increasingly complex software systems. The recent surge in artificial intelligence (AI) has, while reducing human effort, further contributed to this complexity. Among the growing concerns in this domain are the ethical implications of developing and using such systems. Regardless of whether a system explicitly incorporates AI, underlying software infrastructures have demonstrated instances of discriminatory behavior, resulting in undesired outcomes. To address this challenge, my research focuses on discrimination testing as a means to ensure equitable behavior in software systems. As part of this effort, I plan to build on insights from my earlier work, which involved compiling a list of existing tools and user studies on their effectiveness. My goal is to design a more explainable and generalizable discrimination testing tool that can be applied across various systems.

Women researchers in computing face persistent, career-longchallenges that contribute to broader gender disparities in the field. While prior work has examined barriers faced by women in computing, little is known about the lived experiences of women researchers, specifically those navigating careers across academia and industry. To address this gap, we conducted in-depth, semi-structured interviews with ten novice and experienced women researchers across three continents. Using inductive analysis and open coding of transcribed interviews, we identified thirteen career-related challenges, organized into three overarching themes: absence of belonging, experiences of implicit bias, and increased emotional burden. Participants described heightened gender awareness in computing environments, which contributed to feelings of isolation and hindered professional growth. Our findings highlight the need for targeted interventions to support the career trajectories of women researchers, foster inclusive research communities, and ultimately reduce the gender gap in computing.

Students in programming languages and formal logic courses often struggle with constructing rule-based derivation trees due to the complexity of applying inference rules, the lack of immediate feedback, and the manual effort required for handwritten proofs. We present Hazel Deriver, a live, web-based editor designed to scaffold derivation construction through multiple layers of support. Built on the Hazel live programming environment, it provides a structured, interactive experience that encourages iterative exploration and real-time feedback. A preliminary user study with former students suggests that Hazel Deriver reduces the perceived difficulty of derivation tasks while improving conceptual understanding and engagement. We discuss the design of its layered scaffolding features and raise questions about balancing system guidance with learner autonomy.

Generative AI tools are increasingly being integrated into various workflows. Many powerful AI systems, such as ChatGPT, Gemini, and Grok, are accessed through simple chat interfaces. While these language-based interactions may seem intuitive and futuristic, they also bear a resemblance to early computer command lines. With our prototype system, Curompt (combined cursor+prompt), we explore how to integrate conversational interaction powered by generative AI into familiar graphical interfaces with direct manipulation. We focused on the 3D environment, where relying on language-based interactions alone proves to be challenging.

Google’s Blockly library provides rudimentary support for types that constrain how blocks can be connected to form larger fragments. We argue that Blockly’s current type language does not reach beyond the level of context-freeness, and show how this can be changed — in a non-disruptive way — by adopting elements from attribute grammar and constraint-based typing. We demonstrate the generality of our approach by applying it to different languages, and describe our implementation and integration into Blockly.

Understanding unfamiliar code often requires insight into the original developer’s intentions, design rationale, and on-the-fly problem-solving strategies. Traditional version-control systems capture only coarse commits, leaving web foraging and developer notes disconnected. We present DevTales, a VSCode extension that integrates fine-grained subgoal labels, captured web-foraging activity, and LLM-generated narrative summaries directly with code in the IDE. We evaluated DevTales in three mixed-methods user studies spanning debugging, understanding rationale, and code reuse. Results show that subgoal labels and “See in Code” links enable rapid top-down navigation and code localization, while embedded web resources clarify unfamiliar artifacts. Narrative summaries proved valuable for answering deep “why” questions by explicitly linking causal relationships among history items, but were avoided during quick sensemaking contexts due to their verbosity. We discuss design implications for history-aware code search, adaptive narrative summarization, and inference transparency in LLM-generated stories.