Table of Links
-
Prior Work and 2.1 Educational Objectives of Learning Activities
-
3.1 Multiscale Design Environment
3.2 Integrating a Design Analytics Dashboard with the Multiscale Design Environment
-
5.1 Gaining Insights and Informing Pedagogical Action
5.2 Support for Exploration, Understanding, and Validation of Analytics
5.3 Using Analytics for Assessment and Feedback
5.4 Analytics as a Potential Source of Self-Reflection for Students
-
Discussion + Implications: Contextualizing: Analytics to Support Design Education
6.1 Indexicality: Demonstrating Design Analytics by Linking to Instances
6.2 Supporting Assessment and Feedback in Design Courses through Multiscale Design Analytics
2 PRIOR WORK
We situate the present research amidst prior work involving learning activities, design, creativity, and learning analytics. We begin by situating the student project tasks and courses within a broader framework of learning activities. We follow with further consideration of how multiscale design matters in people’s processes of organizing complex information. Next, we consider approaches for assessing creative visual design, which is tricky, because there can be many approaches to solving a given design problem. Because our research involves computational assessment methods in design education, we then present prior work on the use of learning analytics and dashboards.
2.1 Educational Objectives of Learning Activities
Bloom developed a taxonomy of educational objectives, organizing underlying cognitive processes in six categories: knowledge, comprehension, application, analysis, synthesis, and evaluation [14]. The taxonomy is organized as a pyramid with cognitive processes under the ‘knowledge’ category at the bottom and ‘evaluation’ at the top. As Armstrong describes, from bottom to top, objectives are organized in a continuum of simple to complex and concrete to abstract [5]. Krathwohl et al. revised the original taxonomy to distinguish the ‘knowledge’ and ‘cognitive’ dimensions of the objectives [46]. Knowledge dimension categories include factual, conceptual, procedural, and metacognitive. Cognitive dimension categories include remember, understand, apply, analyze, evaluate, and create.
In the design course contexts we study, students engage in multiscale design, as they work on open-ended projects involving iterative phases of requirement gathering, idea generation, prototyping, and evaluation. These design phases engage students in learning activities that represent abstract and complex cognitive processes—at the top of the pyramid in both original and revised taxonomies—such as analyze, evaluate, and create. Multiscale design—as a form of creative visual design—corresponds to the ‘create’ category in the revised taxonomy, which is described as “putting elements together to form a novel, coherent whole or make an original product.”
To effectively assess student learning, it is vital to develop measures aligned with educational objectives. In engineering design, Summers and Shah advocate developing measures such as size, coupling, and solvability [72]. According to them, these measures have the potential to assist instructors in assessing students’ ability to address complex design problems In the design course contexts we study, multiscale design is significant. Hence, we develop computational measures for how students use space and scale in their design work. These measures have the potential to assist instructors in assessing students’ ability to put diverse elements together into a new, coherent whole.
2.2 Multiscale Design
Ray and Charles Eames demonstrated how we humans conceptualize our knowledge of the universe across powers of ten [29]. Tufte articulates micro / macro readings, a strategy for constructing data narratives in which designers employ “fine details” that accumulate to form “larger coherent structures” [73]. Perlin and Fox conceptualized and actualized the importance of organizing information across scales, in the form of a zoomable user interface (ZUI) [61]. Bederson discusses how a ZUI helps people in developing a mental map of the information by taking advantage of human memory and spatial perception [9].
Design environments supporting multiscale design—e.g., Photoshop, Illustrator, InDesign, and IdeaMâché [52, 53]—enable going beyond 2D by organizing content across scales, i.e., across a range of zoom levels. Bar-Yam describes multiscale design as an approach to manage increasing complexity [7]. According to Barba, multiscale design supports a holistic analysis and development of ideas, across multiple scales, by allowing people to shift cognitive point of view up and down hierarchies [8]. As Alexander describes, by employing hierarchies, designers organize a component both as a unit and as a pattern consisting of other units, and thus address the important challenge of building up a form from components [4].
Lupfer et al.’s investigation of a landscape architecture classroom found that multiscale design pervades student projects [51]. In these projects, multiscale design supports students in schematically forming design proposals to meet the situated needs of sites involving waterways and land use. In a study involving computer science courses, multiscale design has been found to support students’ iterative and reflective ideation [52]. In a study involving interactive art and design course contexts, multiscale design has been found to enable students in referring to and reusing ideas across project deliverables, which facilitates consistency in their work [16]. The present research investigates AI-based multiscale design analytics to support instructors in assessing student work. This is part of a broader problem of how to assess design work [39].
Authors:
(1) Ajit Jain, Texas A&M University, USA; Current affiliation: Audigent;
(2) Andruid Kerne, Texas A&M University, USA; Current affiliation: University of Illinois Chicago;
(3) Nic Lupfer, Texas A&M University, USA; Current affiliation: Mapware;
(4) Gabriel Britain, Texas A&M University, USA; Current affiliation: Microsoft;
(5) Aaron Perrine, Texas A&M University, USA;
(6) Yoonsuck Choe, Texas A&M University, USA;
(7) John Keyser, Texas A&M University, USA;
(8) Ruihong Huang, Texas A&M University, USA;
(9) Jinsil Seo, Texas A&M University, USA;
(10) Annie Sungkajun, Illinois State University, USA;
(11) Robert Lightfoot, Texas A&M University, USA;
(12) Timothy McGuire, Texas A&M University, USA.
This paper is