Table of Links Abstract and 1. Introduction Abstract and 1. Introduction Prior Work and 2.1 Educational Objectives of Learning Activities
2.2 Multiscale Design
2.3 Assessing Creative Visual Design
2.4 Learning Analytics and Dashboards


Research Artifact/Probe
3.1 Multiscale Design Environment
3.2 Integrating a Design Analytics Dashboard with the Multiscale Design Environment


Methodology and Context
4.1 Course Contexts
4.2 Instructor interviews


Findings
5.1 Gaining Insights and Informing Pedagogical Action
5.2 Support for Exploration, Understanding, and Validation of Analytics
5.3 Using Analytics for Assessment and Feedback
5.4 Analytics as a Potential Source of Self-Reflection for Students


Discussion + Implications: Contextualizing: Analytics to Support Design Education
6.1 Indexicality: Demonstrating Design Analytics by Linking to Instances
6.2 Supporting Assessment and Feedback in Design Courses through Multiscale Design Analytics
6.3 Limitations of Multiscale Design Analytics


Conclusion and References Prior Work and 2.1 Educational Objectives of Learning Activities
2.2 Multiscale Design
2.3 Assessing Creative Visual Design
2.4 Learning Analytics and Dashboards Prior Work and 2.1 Educational Objectives of Learning Activities Prior Work and 2.1 Educational Objectives of Learning Activities 2.2 Multiscale Design 2.2 Multiscale Design 2.3 Assessing Creative Visual Design 2.3 Assessing Creative Visual Design 2.4 Learning Analytics and Dashboards 2.4 Learning Analytics and Dashboards Research Artifact/Probe
3.1 Multiscale Design Environment
3.2 Integrating a Design Analytics Dashboard with the Multiscale Design Environment Research Artifact/Probe Research Artifact/Probe 3.1 Multiscale Design Environment 3.1 Multiscale Design Environment 3.2 Integrating a Design Analytics Dashboard with the Multiscale Design Environment 3.2 Integrating a Design Analytics Dashboard with the Multiscale Design Environment Methodology and Context
4.1 Course Contexts
4.2 Instructor interviews Methodology and Context Methodology and Context 4.1 Course Contexts 4.1 Course Contexts 4.2 Instructor interviews 4.2 Instructor interviews Findings
5.1 Gaining Insights and Informing Pedagogical Action
5.2 Support for Exploration, Understanding, and Validation of Analytics
5.3 Using Analytics for Assessment and Feedback
5.4 Analytics as a Potential Source of Self-Reflection for Students Findings Findings 5.1 Gaining Insights and Informing Pedagogical Action 5.1 Gaining Insights and Informing Pedagogical Action 5.2 Support for Exploration, Understanding, and Validation of Analytics 5.2 Support for Exploration, Understanding, and Validation of Analytics 5.3 Using Analytics for Assessment and Feedback 5.3 Using Analytics for Assessment and Feedback 5.4 Analytics as a Potential Source of Self-Reflection for Students 5.4 Analytics as a Potential Source of Self-Reflection for Students Discussion + Implications: Contextualizing: Analytics to Support Design Education
6.1 Indexicality: Demonstrating Design Analytics by Linking to Instances
6.2 Supporting Assessment and Feedback in Design Courses through Multiscale Design Analytics
6.3 Limitations of Multiscale Design Analytics Discussion + Implications: Contextualizing: Analytics to Support Design Education Discussion + Implications: Contextualizing: Analytics to Support Design Education 6.1 Indexicality: Demonstrating Design Analytics by Linking to Instances 6.1 Indexicality: Demonstrating Design Analytics by Linking to Instances 6.2 Supporting Assessment and Feedback in Design Courses through Multiscale Design Analytics 6.2 Supporting Assessment and Feedback in Design Courses through Multiscale Design Analytics 6.3 Limitations of Multiscale Design Analytics 6.3 Limitations of Multiscale Design Analytics Conclusion and References Conclusion and References Conclusion and References A. Interview Questions A. Interview Questions 2 PRIOR WORK We situate the present research amidst prior work involving learning activities, design, creativity, and learning analytics. We begin by situating the student project tasks and courses within a broader framework of learning activities. We follow with further consideration of how multiscale design matters in people’s processes of organizing complex information. Next, we consider approaches for assessing creative visual design, which is tricky, because there can be many approaches to solving a given design problem. Because our research involves computational assessment methods in design education, we then present prior work on the use of learning analytics and dashboards. 2.1 Educational Objectives of Learning Activities Bloom developed a taxonomy of educational objectives, organizing underlying cognitive processes in six categories: knowledge, comprehension, application, analysis, synthesis, and evaluation [14]. The taxonomy is organized as a pyramid with cognitive processes under the ‘knowledge’ category at the bottom and ‘evaluation’ at the top. As Armstrong describes, from bottom to top, objectives are organized in a continuum of simple to complex and concrete to abstract [5]. Krathwohl et al. revised the original taxonomy to distinguish the ‘knowledge’ and ‘cognitive’ dimensions of the objectives [46]. Knowledge dimension categories include factual, conceptual, procedural, and metacognitive. Cognitive dimension categories include remember, understand, apply, analyze, evaluate, and create. In the design course contexts we study, students engage in multiscale design, as they work on open-ended projects involving iterative phases of requirement gathering, idea generation, prototyping, and evaluation. These design phases engage students in learning activities that represent abstract and complex cognitive processes—at the top of the pyramid in both original and revised taxonomies—such as analyze, evaluate, and create. Multiscale design—as a form of creative visual design—corresponds to the ‘create’ category in the revised taxonomy, which is described as “putting elements together to form a novel, coherent whole or make an original product.” To effectively assess student learning, it is vital to develop measures aligned with educational objectives. In engineering design, Summers and Shah advocate developing measures such as size, coupling, and solvability [72]. According to them, these measures have the potential to assist instructors in assessing students’ ability to address complex design problems In the design course contexts we study, multiscale design is significant. Hence, we develop computational measures for how students use space and scale in their design work. These measures have the potential to assist instructors in assessing students’ ability to put diverse elements together into a new, coherent whole. 2.2 Multiscale Design Ray and Charles Eames demonstrated how we humans conceptualize our knowledge of the universe across powers of ten [29]. Tufte articulates micro / macro readings, a strategy for constructing data narratives in which designers employ “fine details” that accumulate to form “larger coherent structures” [73]. Perlin and Fox conceptualized and actualized the importance of organizing information across scales, in the form of a zoomable user interface (ZUI) [61]. Bederson discusses how a ZUI helps people in developing a mental map of the information by taking advantage of human memory and spatial perception [9]. Design environments supporting multiscale design—e.g., Photoshop, Illustrator, InDesign, and IdeaMâché [52, 53]—enable going beyond 2D by organizing content across scales, i.e., across a range of zoom levels. Bar-Yam describes multiscale design as an approach to manage increasing complexity [7]. According to Barba, multiscale design supports a holistic analysis and development of ideas, across multiple scales, by allowing people to shift cognitive point of view up and down hierarchies [8]. As Alexander describes, by employing hierarchies, designers organize a component both as a unit and as a pattern consisting of other units, and thus address the important challenge of building up a form from components [4]. Lupfer et al.’s investigation of a landscape architecture classroom found that multiscale design pervades student projects [51]. In these projects, multiscale design supports students in schematically forming design proposals to meet the situated needs of sites involving waterways and land use. In a study involving computer science courses, multiscale design has been found to support students’ iterative and reflective ideation [52]. In a study involving interactive art and design course contexts, multiscale design has been found to enable students in referring to and reusing ideas across project deliverables, which facilitates consistency in their work [16]. The present research investigates AI-based multiscale design analytics to support instructors in assessing student work. This is part of a broader problem of how to assess design work [39]. Authors:
(1) Ajit Jain, Texas A&M University, USA; Current affiliation: Audigent;
(2) Andruid Kerne, Texas A&M University, USA; Current affiliation: University of Illinois Chicago;
(3) Nic Lupfer, Texas A&M University, USA; Current affiliation: Mapware;
(4) Gabriel Britain, Texas A&M University, USA; Current affiliation: Microsoft;
(5) Aaron Perrine, Texas A&M University, USA;
(6) Yoonsuck Choe, Texas A&M University, USA;
(7) John Keyser, Texas A&M University, USA;
(8) Ruihong Huang, Texas A&M University, USA;
(9) Jinsil Seo, Texas A&M University, USA;
(10) Annie Sungkajun, Illinois State University, USA;
(11) Robert Lightfoot, Texas A&M University, USA;
(12) Timothy McGuire, Texas A&M University, USA. Authors: Authors: (1) Ajit Jain, Texas A&M University, USA; Current affiliation: Audigent; (2) Andruid Kerne, Texas A&M University, USA; Current affiliation: University of Illinois Chicago; (3) Nic Lupfer, Texas A&M University, USA; Current affiliation: Mapware; (4) Gabriel Britain, Texas A&M University, USA; Current affiliation: Microsoft; (5) Aaron Perrine, Texas A&M University, USA; (6) Yoonsuck Choe, Texas A&M University, USA; (7) John Keyser, Texas A&M University, USA; (8) Ruihong Huang, Texas A&M University, USA; (9) Jinsil Seo, Texas A&M University, USA; (10) Annie Sungkajun, Illinois State University, USA; (11) Robert Lightfoot, Texas A&M University, USA; (12) Timothy McGuire, Texas A&M University, USA. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Dash

Microsoft

AI Tools Take Aim at Assessing Multiscale Design in the Classroom

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

A Coarse-to-Fine Approach to Text-Guided 3D Localization Without Ground Truth

AI Analysis Appears to Improve How Teachers Evaluate Student Design Projects, Study Finds

AI-Powered Analytics Offer New Pathways for Evaluating Multiscale Design Work

The Research Artifact Making AI-Powered Design Insights Visible at Multiple Scales

Will AI Tools Change the Way Educators Evaluate Student Work?

Critical Thinking to AI: Are you a Friend or Foe?

A Coarse-to-Fine Approach to Text-Guided 3D Localization Without Ground Truth

AI Analysis Appears to Improve How Teachers Evaluate Student Design Projects, Study Finds

AI-Powered Analytics Offer New Pathways for Evaluating Multiscale Design Work

The Research Artifact Making AI-Powered Design Insights Visible at Multiple Scales

Will AI Tools Change the Way Educators Evaluate Student Work?

Critical Thinking to AI: Are you a Friend or Foe?

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps