CS 347 HCI Foundations & Frontiers

Stanford course on what's next and why in Human Computer Interaction (Spr 2024)

Location: Lane History Corner 200-002
Time: MW 10:30am-12:20pm
Questions: cs347@cs.stanford.edu

Syllabus

Topic and reading schedule may shift as the quarter evolves. Stanford students can use the Stanford Library proxy for off-campus access to readings if they are behind a paywall.

Week 1
M Apr 1: Ubiquitous and Tangible Computing [Slides]
(No readings)
 
W Apr 3: Input/Output [Slides]
The Computer for the 21st Century. Mark Weiser. Scientific American. September 1991, pp. 94-104.
This is one of the most influential, highly-cited papers in all of human-computer interaction. Weiser's vision of ubiquitous computing articulated a vision in which computing recedes into the background rather than stands as a focal point of our attention. Researchers and practitioners have chased this vision since 1991. We read this article because it is the foundation upon which much of the Interaction component of the class sits. What do you think about Weiser's differentiation between calm computation that recedes into infrastructure, and simply spreading computers around everywhere in the environment? What are the implications of such a vision?
The Tangible Bits paper sits alongside Weiser's Ubiquitous Computing paper as one of the foundational works of modern HCI. It came out several years after Weiser's article, but in many ways, was the innovation that made Weiser's vision actionable. Tangible bits takes a strong stance that input and output need to be aligned in concrete physical artifacts, whereas Weiser allowed digital outputs such as screens and projectors. This paper has spawned many, many projects and even entire conferences. What tradeoffs does Tangible Bits make through its strict adherence to input/output co-location?
 
Week 2
M Apr 8: Activity, Health, Behavior [Slides]
BeWell: A Smartphone Application to Monitor, Model and Promote Wellbeing. Nicholas D. Lane, Mashfiqui Mohammod, Mu Lin, Xiaochao Yang, Hong Lu, Shahid Ali, Afsaneh Doryab, Ethan Berke, Tanzeem Choudhury, Andrew T. Campbell. Pervasive computing technologies for healthcare 2012.
A smartphone contains a variety of sensors that can be used to track all kinds of detailed user activity. This work shows how such sensor data can be combined together to estimate the overall wellbeing of the user. They suggest visualizations to communicate both the activities and wellbeing scores to users. Which visualizations would be most useful to you if you used this system? Which do you think would be most useful to others? Why?
Designing Ambient Narrative-Based Interfaces to Reflect and Motivate Physical Activity. Elizabeth L. Murnane, Xin Jiang, Anna Kong, Michelle Park, Weili Shi, Connor Soohoo, Luke Vink, Iris Xia, Xin Yu, John Yang-Sammataro, Grace Young, Jenny Zhi, Paula Moya, James A. Landay. CHI 2020.
This paper argues that storytelling can be persuasive in promoting health behavior, can help people make sense of their experiences and create opportunities for self-affirmation. They present a well-motivated design for a smartphone app that uses storytelling with visual encoding of passively tracked activity data to promote physical activity. They show that a good design can affect user behavior with various evaluations. But, with this kind of product-oriented design work, it is worth asking, what lessons do we learn that are transferable to other design problems?
 
W Apr 10: Design Cognition [Slides]
This book is one of the best known treatises on design and HCI. Norman describes interfaces and the problems with them from the perspective of a cognitive psychologist. Chapter two is about the psychology of everyday interactions with systems. It describes how people work with systems to get things done and more importantly what people think about as they are working with the systems. The descriptions of the gulf of of execution and evaluation, the seven stages of action and the corresponding seven principles of design will recur throughout this course. What other interface design problems have you encountered where they might have been relevant?
Quiz 1: in class, bring pencil
 
Week 3
M Apr 15: Design Process [Slides]
As we will discuss in class, participatory design is a method for bringing community stakeholders in directly to the full design process, rather than only allowing the designer to engage in the concept generation and selection stages. But, in this work, Harrington and her colleagues demonstrate through a series of co-design (participatory) workshops that bringing people into the room does not mean that they have, or view themselves as having, full status and authority. What do you think we ought to do about this problem? Does it suggest deeper issues with participatory design and its goals? Does it suggest alternative design processes that we might consider?
The Promise of Empathy: Design, Disability, and Knowing the "Other". Cynthia L. Bennet, Daniela K. Rosner. CHI 2019.
Needfinding, or developing empathy, are key processes in the design process. In this article, Bennett and Rosner step back and point out how the processes that designers use to build empathy may be causing harm rather than helping. They use the case of empathy activities around disability, and how designers walk away with false confidence that they understand and have empathized with disabled individuals' experiences. Just like with the Harrington et al. paper, we ask: what ought we to do about this problem? Does it suggest deeper issues with the needfinding and empathy-building processes we use in design? Can you think of alternative processes that might mitigate the risk of this occurring?
 
W Apr 17: Design Tools [Slides]
This book provides foundational theory that we draw on in HCI design: what is core to the activity of design, and what does expertise in design really mean? Schön's answer — reflection in action — is a frame that has proven useful, by articulating that the designer iterates by taking an action, reflecting on the result of that action, and using that reflection to gain insight into the problem in order to plan the next action. Schön's argument that design cannot be planned and is instead a process of iteration and reflection is foundational reasoning in HCI as to why our field works the way it does, and why we teach the way we do. Do you agree with his position? What are the theory's consequences for the goals of the tools and processes that we develop for design? Optional: read Chapter 2, which sets up Schön's critique of rationality in problem solving and lays out his theory of reflection-in-action.
Work in design tools has drawn on AI and data-driven models to support the reflection element of reflection-in-action. In particular, designers often have to imagine how people will react to their designs. This paper is one of the first to demonstrate that, within certain constraints, those reactions could be simulated effectively by an AI. Think about the other forms of feedback, and other kinds of designs, that could be supported by this kind of approach.
 
Week 4
M Apr 22: Social Media [Slides]
Beyond Being There. Jim Hollan, Scott Stornetta. CHI 1992.
A classic of social computing, which was originally called CSCW (Computer-Supported Cooperative Work). It essentially argues both that remote collaboration will never be as good, and that it could be even better. The question of "does this satisfy the Beyond Being There criteria?" is a good gut check for any social computing system. What implications do you see of this idea, or what alternatives might you pitch? What are the implications of the claim, if we take it in its strongest instantiation that we should never even try to mirror offline interactions?
Groupware and social dynamics: eight challenges for developers. Jonathan Grudin. Communications of the ACM 1994.
A second classic of social computing (Grudin calls it "groupware"). In this article in the Communications of the ACM, Jonathan Grudin lays out all the reasons why social applications fail to get over the cold start problem. These problems remain resiliently, frustratingly challenging today. Which do you feel are the most core of the problems that he articulates, and why are they the most central? Are there any core problems that he overlooked?
 
W Apr 24: Collaboration [Slides]
Distance Matters. Gary Olson, Judith Olson. Human-Computer Interaction 2000.
Rounding out a week of classics. CSCW ("computer-supported cooperative work") is a vibrant field of HCI focused on collaboration, and is the field that gave rise to social computing. (Social computing broadened CSCW out from office-based collaboration environments to user-contributed content online.) The Distance Matters paper is part of the CSCW canon. This paper raised an important question as to how effective collaboration software could really ever be, and why. Do you think the Distance Matters limit is fundamental? If so, why? If not, what could change the situation?
Quiz 2: in class, bring pencil
 
Week 5
M Apr 29: Human-Centered AI [Slides]
Direct Manipulation Vs. Interface Agents. Ben Shneiderman and Pattie Maes. Interactions 1997.
In a pair of debates in 1997 researchers Pattie Maes and Ben Shneiderman argued about the way in which future interfaces would work. Would users delegate tasks to proactive software agents (Maes' position) or would users always have full control even as automation increases (Shneiderman position). One could argue that until recently the direct manipulation paradigm has been dominant. But with the advent of LLMs and generative AI are we moving to a world of autonomous software agents?
Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design. Qian Yang, Aaron Steinfeld, Carolyn Rosé, John Zimmerman. CHI 2020.
What is it about AI-powered interaction that is so challenging to get right? Why hasn't the rise of modern deep learning technologies fundamentally altered our interactive systems? In this article, Qian Yang and her coauthors try to drill down into answers to this question. Do you agree with their take? Do you have an alternative theory?
 
W May 1: Working with Unpredictable Black Boxes [Slides]
Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. J.D. Zamfirescu-Pereira, Richmond Wong, Bjoern Hartmann, Qian Yang. CHI 2023.
While prompting is often hailed as an accessible interface for non-AI-experts to work with AI, crafting effective prompts is challenging. This paper examines why prompting is challenging for non-experts and suggests opportunities for designing more accessible tools that can aid prompt design. Are the techniques proposed here enough to fully mitigate the challenges of prompt engineering? Are there other approaches we might use?
Bridging the Gulf of Envisioning: Cognitive Challenges in Prompt Based Interactions with LLMs. Hariharan Subramonyam, Roy Pea, Christopher Lawrence Pondoc, Maneesh Agrawala, Colleen Seifert. CHI 2024.
A goal of interface design is to reduce the gulfs of execution and evaluation between the user and the system. This paper suggests that when working with LLM there is another gulf -- the gulf of envisioning -- that makes it hard for a user to know what exactly the task should be, how to instruct the LLM to do a task and what to expect from the LLM as output. How should we design LLM interactions to reduce this gulf? Is the gulf of envisioning one that only applies to LLMs, or does it apply to other systems as well?
 
Week 6
M May 6: Cognitive Models
Jef Raskin was an HCI expert who led the Macintosh project at Apple. In this chapter of his book he describes a set of methods developed by HCI researchers in the 1980s to quantify human performance based on models of human information processing. The keystroke-level model attempts to compute how long it might take the average person to perform a task using a GUI. The exact numbers produced by these methods are very rough. But they can be useful to obtain ballpark figures for how long it might take to perform a particular task. And they can help designers identify areas of the interface that might require too many clicks or keystrokes. But developing such low-level models is tedious and since the late 1990s these methods have fallen out of favor. Today they are little used in practice. Why do you think this might be? Are there issues today that could be addressed by building such models.
The Role of Working Memory in Program Tracing. Will Crichton, Maneesh Agrawala, Pat Hanrahan. CHI 2021.
How do limitations of human working memory affect programmers? This paper runs a sequence of experiments examining how human programmers trace snippets of code. It uses an interesting methodology (restricted focus viewing) to identify while lines of the program the user is examining and analyzes how often the programmer returns to lines they have already seen as a indication of items slipping from working memory. How would you build an IDE to address some of the limitations of working memory. How could this type of quantitative analysis and memory modeling be used for other kinds of interfaces?
 
W May 8: Information Visualization
Choosing the "right" representation for a problem can make the problem far easier to solve. The challenge is to match the representation to the problem or task. In visualization the goal is to represent numbers and data are visually. The best visual representation is also dependent on the task. Is the task to identify outliers in a set of home price data? Is it to compare the heights of children in different grades? Is it to understand multiple dimensions and the correlations between them? How does the task affect your choice of representation?
Quiz 3: in class, bring pencil
 
Week 7
M May 13: Software Tools and Toolkits
Past, Present and Future of User Interface Software Tools. Brad A. Myers, Scott Hudson, Randy Pausch. ACM ToCHI 2000.
User interface software toolkits helps developers design and implement the interface. But how should such toolkits be evaluated? This paper examines historical examples of interface toolkits and argues for several measures of success. How would these measures apply to AI interface toolkits today? Does a ChatBot have a low threshold and a high ceiling? Are ChatBots predictable? Do the measures of success proposed here, still apply?
By changing the questions that programmers can ask of their code, we can empower developers to create more effective code and feel more self-efficacy in doing so. Does Amy Ko's theory of 'why and why not' questions align with your point of view on how to make debugging easier? What other kinds of cognitive scaffolds would be helpful, if they could be done?
 
W May 15: Content Creation
Design principles for visual communication. Maneesh Agrawala, Wilmot Li, Floraine Berthouzoz. Communications of the ACM 2011.
This article reviews a methodology we developed in my group for building content creation tools. We identify design principles that expert content creators use, we instantiate these principles algorithmically and we then evaluate the effectiveness of the resulting tools and the content they produce. This paper argues that explicitly stating design principles and how the principles support cognition is a key part of the process. What are other content creation domains where this approach might be applied?
Draco: bringing life to illustrations with kinetic textures. Rubaiat Habib Kazi, Fanny Chevalier, Tovi Grossman, Shengdong Zhao, George Fitzmaurice. CHI 2014.
This paper is a beautiful illustration (literally?) of how computational tools can empower new types of creative expression. This research was eventually launched as a product by Autodesk, called Autodesk Sketchbook Motion. Could these authoring techniques be translated to other domains as well?
 
Week 8
M May 20: A Critical Reorientation in HCI
What is "Critical" about Critical Design. Jeffrey Bardzell, Shaowen Bardzell. CHI 2013.
This paper situated the term critical design in HCI, a term originally coined in Design by Dunne and Raby, which led to an increased adoption of critical design in HCI. The authors offer a compelling and generative lens for a critical reorientation in HCI. What are the opportunities that this framing of critical design in HCI provides? How might you apply this thinking to what you have so far seen in class? As you think about the generative possibilities of this work, consider what are the limitations that this reframing of critical design might bring about, such as tensions between Design and HCI?
An earlier paper by Shaowen Bardzell—one of the two authors of the previous paper, this work is a nice application of critical theory, namely feminist theory, in HCI or interaction design. Do we err by assigning genderless constructs such as 'the user' in HCI? Do our visions of HCI make implicit assumptions about what kinds of sensing and tracking are desirable, what kinds of social interactions ought to be designed for, or what kinds of problems are interesting, based on this notion of the prototypical user? As you read this article about refocusing attention from default users to the marginalized, ask: where else do these gaps arise, and what might we do about them? What other theories can offer a critical reorientation in HCI?
 
W May 22: Simulating People
Generative Agents: Interactive Simulacra of Human Behavior. Joon Sung Park, Joseph O'Brien, Carrie Cai, Meredith Ringel Morris, Percy Liang, Michael S. Bernstein. UIST 2023.
How might we craft simulations of human individuals and societies that reflect our lives? Many of the core design challenges in human-computer interaction that we discussed in this class, from building computational tools to designing computer-mediated social interactions, must reckon with the complex nature of our world and the way individuals behave and interact with technology. Generative agents point to a future in which the power to simulate hypothetical worlds, where we can ask 'what if' counterfactual questions and paint concrete pictures of how a multiverse of different possibilities might unfold, promises an opportunity to navigate this complexity. This week, reflect on how generative agents extend the original visions of the foundational literature on cognitive models that we covered in this class (e.g., GOMS, KLM) in the era of generative AI models. How might these simulacra of human behaviors inform the design of our interactions, and what new interaction paradigms will these agents enable?
Quiz 4: in class, bring pencil
 
Week 9
M May 27: Memorial Day Holiday
(No readings)
 
W May 29: Methodology Matters
In The Sciences of the Artificial, Herb Simon argues for the status of design as a rigorous method in and of itself. In doing so, similar to Schön, he contrasts design with traditional engineering subjects and the ways in which seeking to cast all goals in terms of formalization of optimization objectives narrows our view. Does the argument convince? How else might we argue for design as a rigorous method?
 
Week 10
M Jun 3: Something Old, Something New
As We May Think. Vannevar Bush. Atlantic Monthly 1945.
Bush's article As We May Think is often pinpointed as the moment that catalyzed the ideas that gave rise to human-computer interaction as a field. Up to this point, computers were calculation machines used by teams in large rooms to calculate ballistics. What did it take Bush to transition that vision to one of cognitive amplification? What claims that he made still resonate, and what did he foresee? Were there assumptions that he made that can no longer apply?
 
W Jun 5: Comprehensive Final Quiz
(No readings)
Quiz 5 (Comprehensive): in class, bring pencil