Interaction & Predictive HCI

draft

Norman (gulfs), affordance×signifier, GOMS/KLM (predicts seconds), Fitts/Hick, information scent, error taxonomy (slips×mistakes, Swiss-cheese).

Research grounding for a computable UI-quality index. Each entry tags measurability: [AUTO/predictive] = derivable from a machine-readable task flow / UI tree with no human; [AI-judge] = an LLM/vision model can score against the artifact; [human] = needs a user study (use only to calibrate the auto signals).

1. Norman's 7-Stage Action Cycle + the Two Gulfs

Definition. Don Norman (The Design of Everyday Things) models every interaction as a cycle of seven stages straddling the world and the mind:

  1. Goal (form the goal) — the "bridge" reference standard
  2. Plan (the action) — Execution side
  3. Specify (an action sequence)
  4. Perform (the action sequence)
  5. Perceive the state of the world — Evaluation side
  6. Interpret the perception
  7. Compare outcome with goal

Gulf of Execution = the gap between the user's intention and the actions the system allows (stages 2–4). "How do I do it?" Gulf of Evaluation = the gap between the system's output state and the user's ability to perceive/interpret whether the goal was met (stages 5–7). "Did it work?"

Each stage → a checkable design question (Norman's seven questions):

Stage Design question Index check
Goal What do I want to accomplish? Is the task's target state nameable? [AI-judge]
Plan What are the alternative action sequences? Are valid actions discoverable from current screen? [AUTO] affordance count
Specify What action can I do now? Is each control labeled/enabled-state visible? [AUTO]
Perform How do I do it? Is the control reachable (in DOM/tree, not hidden)? [AUTO]
Perceive What happened? Did an observable state change emit within latency budget? [AUTO] feedback presence
Interpret What does it mean? Is the new state self-describing (label/status)? [AI-judge]
Compare Is this what I wanted? Can success/failure be told apart at a glance? [AI-judge]

Index signal. Gulf-of-Execution score = fraction of intended actions that have a visible, enabled, reachable control on the current screen [AUTO]. Gulf-of-Evaluation score = fraction of actions that produce a perceivable, interpretable state change (feedback within budget) [AUTO] + naming clarity [AI-judge]. Maps directly to Norman's four design principles: visibility, conceptual model, mapping, feedback.

2. Affordances vs Signifiers (Gibson / Norman)

Definition. Affordance (Gibson, ecological psychology) = an action possibility offered by the relationship between an object and an actor; it exists whether or not perceived. Norman's perceived affordance = the action the user believes is possible based on prior experience. Signifier (Norman, revised DOET) = a perceptible cue that advertises where an affordance is and how to act (arrow, label, underline, handle, ripple). Norman's key correction: designers don't "add affordances," they add signifiers that make an existing affordance discoverable.

Threshold / rule. Every interactive affordance must carry ≥1 signifier perceivable without action (no hover-only discovery for primary actions).

Index signal. Signifier coverage = (interactive elements with a visible signifier: non-default cursor, label, icon, borderelevation, role) ÷ (all interactive elements). [AUTO] from the accessibilityrender tree: every node with a click/tap handler must expose a name + a visual affordance token. False-affordance rate = non-interactive elements that look clickable [AI-judge / vision].

3. Mental Models & Model–System Mismatch

Definition. The user's internal mental model is their belief about how the system works, built from the system image (everything the UI exposes). Trouble arises when the design model (what the designer intends) ≠ user's model ≠ system image. Mismatch is the root cause of most mistakes (§7) and gulf-of-evaluation failures.

Index signal. Hard to fully automate. Proxies: consistency score (same concept → same labeliconplacement across screens) [AUTO]; conceptual coherence of labels & flows vs the domain [AI-judge]; calibrate with task-success / first-click correctness [human].

4. GOMS & the Keystroke-Level Model — the predictive, computable core ⭐

GOMS (Card, Moran & Newell, 1983) decomposes expert, error-free task performance into Goals, Operators, Methods, Selection rules. It predicts execution time and learning time from a task description — no users required. Variants:

Variant What it adds Cost
KLM (Keystroke-Level Model) Flat sum of operator times Cheapest, most computable
CMN-GOMS Goal hierarchy + methods + selection rules Medium
NGOMSL Structured pseudocode; also predicts learning time Medium-high
CPM-GOMS Parallel ops via Model Human Processor critical-path (PERT chart) Highest fidelity, hardest

Keystroke-Level Model — standard operator times (CardMoranNewell)

KLM predicts T_execute = Σ operator times over the action sequence:

Op Operator Time (s) Notes
K Keystroke / button press 0.20 (avg skilled); 0.12 good typist · 0.28 poor · 1.20 non-typist per key
P Point with mouse to target 1.10 (avg); or compute via Fitts's Law move only
B Mouse button down or up 0.10 (0.20 for a full click)
H Home hands between keyboard ↔ mouse 0.40
D Drawing (n segments, length D) 0.9n + 0.16D rare
M Mental preparation / "think" 1.35 placed by heuristic rules
R(t) System response t (measured) only counts if user waits

Worked prediction — "select a menu item, then type a 5-char filename and hit Enter" (hand starts on keyboard): H (→mouse) + M + P + B·2 + H (→kbd) + M + 5K + K(Enter) = 0.40 + 1.35 + 1.10 + 0.20 + 0.40 + 1.35 + 5(0.20) + 0.20 = 6.00 s predicted task time.

M-placement heuristics (Card et al.): insert an M before each cognitively distinct unit (before pointing to a new target, before issuing a command); then delete M's that are fully anticipated (e.g., consecutive keystrokes of a known string), or that the system prompts for.

Two complementary predictive laws (feed KLM's P and choice steps):

  • Fitts's Law — pointingtap time: *T = a + b·log₂(2DW), where D = distance to target, W = target width; ID = log₂(2DW)*bits. Smallerfarther targets cost more. Directly computes a better P from on-screen geometry. [AUTO] from layout coordinates.
  • Hick–Hyman Law — decision time among n equally-likely choices: RT = a + b·log₂(n+1). Predicts cost of menu/option breadth. [AUTO] from option counts.

Measurability: [AUTO/predictive]. KLMFittsHick are fully auto-derivable from a recorded or specified task flow + UI geometry. This is the cluster's flagship computable efficiency metric: feed a task's action sequence → emit a predicted seconds number, with zero user testing.

Index signal. Predicted task time (s) per canonical task (KLM sum, P via Fitts). Operator-count / keystroke economy = predicted vs theoretical-minimum operators (detects unnecessary steps). Hick cost of primary menus. Lower = more efficient; track per release as a regression gate.

5. Information Foraging & Information Scent (Pirolli & Card, PARC)

Definition. Borrowed from optimal foraging theory: users seeking information behave like animals foraging in patches, trying to maximize value gained ÷ cost (time/clicks). Information scent = the user's (imperfect) estimate of valuecostpath of a target, read from proximal cues (link text, labels, icons, snippets). Strong scent → confident, fast navigation; weakdiffuse scent → backtracking and abandonment. The *atchdiet model predicts when a user leaves a "patch" (page/section) for another — the charnov marginal-value theorem* leave when local gain rate drops below the environment's average.

Index signal. Scent strength = semantic similarity between a navigation cue (linklabel) and its destination contentgoal — computable with embeddings [AUTO] (cue→target cosine), quality-rated [AI-judge]. Scent consistency = cue wording matches the landing page's heading [AUTO]. Low predicted scent flags likely backtracking/lostness before any user test.

6. Feedback & Feedforward

Definition. Feedback = communicating the result of an action after it happens (state change, confirmation). Feedforward = communicating what will happen / what is possible before acting (Wensveen, Djajadiningrat & Overbeeke, Interaction Frogger). Frogger couples action↔function on six dimensions: time, location, direction, dynamics, modality, expression; and distinguishes inherent (intrinsic to the control) vs augmented (added signal) information. Feedforward overlaps with signifiers (§2); feedback closes the Gulf of Evaluation (§1).

Thresholds (perceptual latency budgets, NielsenMillerCard): 0.1 s = feels instantaneous (direct manipulation must hit this); 1 s = keeps flow of thought (show feedback by here); 10 s = limit of attention (needs a progress indicator beyond this).

Index signal. Feedback presence/latency — every committed action emits a perceivable state change within budget [AUTO] (instrument event→paint). Feedforward coverage — controls expose pre-action cues (hoverpreviewlabels) [AUTO] + quality [AI-judge].

7. Error Taxonomy: Slips vs Mistakes, Swiss-Cheese, Prevention/Recovery

Definition (Norman / Reason). Two-level split by where it breaks:

  • Slips & lapses (execution failures) — right plan, wrong action. Slip = action-based (right intention, wrong motor act: hit adjacent button, capturedescriptionmode error). Lapse = memory-based (forgot a step). Caused by inattention/automaticity.
  • Mistakes (planning failures) — wrong plan. Rule-based (applied a bad/wrong rule) vs knowledge-based (faulty mental model, wrong problem framing).

Reason's Swiss-Cheese Model. Defenses are layers of cheese; each has holes (latent + active failures). An incident occurs only when holes momentarily align through all layers. Design implication: defense in depth — multiple independent barriers (constraints, confirmations, validation, undo) so no single hole reaches harm.

Prevention vs recovery (Nielsen heuristics #5 "Error Prevention", #9 "Help users recover").

  • Prevent slips: constraints, forcing functions, sensible defaults, disable invalid actions, confirm destructive ones.
  • Prevent mistakes: good signifiers + conceptual model + good feedforward.
  • Recovery: Undo/Redo (reversibility) is the single highest-leverage recovery affordance — it makes the system explorable and turns most slips into non-events; plus clear, constructive error messages (in plain language, with a fix).

Index signal. Reversibility coverage = destructivecommitting actions that offer undo or confirm [AUTO]. Constraint coverage = inputs with validationdisable-on-invalid [AUTO]. Error-message quality (plain language, names the problem, suggests a fix; no codes) [AI-judge] per specs/errors/user-facing-messages.kmd. Defense depth = independent barriers before irreversible harm [AUTO].

8. Recognition vs Recall, Progressive Disclosure, Chunking

Recognition over recall (Nielsen heuristic #6). Recognition (seeing a cue and knowing it) is far cheaper than recall (retrieving from memory unaided). Keep options, actions, and previously entered values visible or easily retrievable — don't force users to remember info across screens. Index: recall-burden count = pieces of info the user must carry between steps that aren't shown [AUTO]; shortcut/command surfaces that also expose a visible label [AUTO].

Chunking (Miller's "magical number 7±2", recast). Short-term memory holds a small number of chunks, not items — grouping raises effective capacity. Caveat (NN/g): menus rely on recognition, so 7±2 is not a limit on visible menu length; chunking helps memory tasks (phone numbers, codes, content scannability), not on-screen option counts. Index: grouping ratio — long flat listsforms broken into labeled groups [AUTO]; chunk size of codesIDs surfaced to users [AUTO].

Progressive disclosure. Show the few common/important options first; reveal advanced ones on demand. Reduces visual complexity and Hick's-Law decision cost (§4) without removing power. Index: initial-choice count (visible options on first view) vs total [AUTO] → feeds Hick prediction; primary actions surfaced, secondary deferred [AI-judge].

Shortlist — Computable Interaction-Efficiency Signals (for the index)

All [AUTO/predictive] unless noted — derivable from a task flow + UI/accessibility tree + render timing:

  1. KLM predicted task time (s) per canonical task — Σ operator times, P via Fitts's Law. Flagship efficiency metric; regression-gate per release.
  2. Operator/keystroke economy — predicted operators vs theoretical minimum (detects redundant steps).
  3. Fitts ID (bits) for primary targets — penalizes smalldistantedge-buried controls; computed from layout geometry.
  4. Hick decision cost — log₂(n+1) over visible primary options; lowered by progressive disclosure.
  5. Gulf-of-Execution coverage — intended actions with a visibleenabledreachable control ÷ all.
  6. Gulf-of-Evaluation / feedback latency — actions emitting perceivable state change within 0.1 / 1 / 10 s budgets.
  7. Signifier coverage — interactive nodes with a visible affordance cue ÷ all interactive nodes; + false-affordance rate [vision/AI-judge].
  8. Information-scent strength — embedding cosine(cue → destination/goal); flags backtracking.
  9. Recall-burden count — info the user must remember across steps but isn't shown.
  10. Reversibility & constraint coverage — destructive actions with undo/confirm; inputs with validation; defense-depth before irreversible harm.
  11. Consistency score — same concept → same labeliconplacement across screens.

AI-judge layer (spirit checks): conceptual-model coherence, error-message quality, label/scent clarity, false affordances. Human-calibration only: task success, first-click correctness, time-on-task — use to validate that the auto signals predict real outcomes, not as per-build gates.

Sources