Table of Links Abstract and 1. Introduction Abstract and 1. Introduction Prior Work and 2.1 Educational Objectives of Learning Activities
2.2 Multiscale Design
2.3 Assessing Creative Visual Design
2.4 Learning Analytics and Dashboards


Research Artifact/Probe
3.1 Multiscale Design Environment
3.2 Integrating a Design Analytics Dashboard with the Multiscale Design Environment


Methodology and Context
4.1 Course Contexts
4.2 Instructor interviews


Findings
5.1 Gaining Insights and Informing Pedagogical Action
5.2 Support for Exploration, Understanding, and Validation of Analytics
5.3 Using Analytics for Assessment and Feedback
5.4 Analytics as a Potential Source of Self-Reflection for Students


Discussion + Implications: Contextualizing: Analytics to Support Design Education
6.1 Indexicality: Demonstrating Design Analytics by Linking to Instances
6.2 Supporting Assessment and Feedback in Design Courses through Multiscale Design Analytics
6.3 Limitations of Multiscale Design Analytics


Conclusion and References Prior Work and 2.1 Educational Objectives of Learning Activities
2.2 Multiscale Design
2.3 Assessing Creative Visual Design
2.4 Learning Analytics and Dashboards Prior Work and 2.1 Educational Objectives of Learning Activities Prior Work and 2.1 Educational Objectives of Learning Activities 2.2 Multiscale Design 2.2 Multiscale Design 2.3 Assessing Creative Visual Design 2.3 Assessing Creative Visual Design 2.4 Learning Analytics and Dashboards 2.4 Learning Analytics and Dashboards Research Artifact/Probe
3.1 Multiscale Design Environment
3.2 Integrating a Design Analytics Dashboard with the Multiscale Design Environment Research Artifact/Probe Research Artifact/Probe 3.1 Multiscale Design Environment 3.1 Multiscale Design Environment 3.2 Integrating a Design Analytics Dashboard with the Multiscale Design Environment 3.2 Integrating a Design Analytics Dashboard with the Multiscale Design Environment Methodology and Context
4.1 Course Contexts
4.2 Instructor interviews Methodology and Context Methodology and Context 4.1 Course Contexts 4.1 Course Contexts 4.2 Instructor interviews 4.2 Instructor interviews Findings
5.1 Gaining Insights and Informing Pedagogical Action
5.2 Support for Exploration, Understanding, and Validation of Analytics
5.3 Using Analytics for Assessment and Feedback
5.4 Analytics as a Potential Source of Self-Reflection for Students Findings Findings 5.1 Gaining Insights and Informing Pedagogical Action 5.1 Gaining Insights and Informing Pedagogical Action 5.2 Support for Exploration, Understanding, and Validation of Analytics 5.2 Support for Exploration, Understanding, and Validation of Analytics 5.3 Using Analytics for Assessment and Feedback 5.3 Using Analytics for Assessment and Feedback 5.4 Analytics as a Potential Source of Self-Reflection for Students 5.4 Analytics as a Potential Source of Self-Reflection for Students Discussion + Implications: Contextualizing: Analytics to Support Design Education
6.1 Indexicality: Demonstrating Design Analytics by Linking to Instances
6.2 Supporting Assessment and Feedback in Design Courses through Multiscale Design Analytics
6.3 Limitations of Multiscale Design Analytics Discussion + Implications: Contextualizing: Analytics to Support Design Education Discussion + Implications: Contextualizing: Analytics to Support Design Education 6.1 Indexicality: Demonstrating Design Analytics by Linking to Instances 6.1 Indexicality: Demonstrating Design Analytics by Linking to Instances 6.2 Supporting Assessment and Feedback in Design Courses through Multiscale Design Analytics 6.2 Supporting Assessment and Feedback in Design Courses through Multiscale Design Analytics 6.3 Limitations of Multiscale Design Analytics 6.3 Limitations of Multiscale Design Analytics Conclusion and References Conclusion and References Conclusion and References A. Interview Questions A. Interview Questions 6.2 Supporting Assessment and Feedback in Design Courses through Multiscale Design Analytics For learning analytics to be effective in open-ended, project-based contexts, there is a need to assess complex characteristics that can give insights into students’ creative strategies and abilities [13]. Toward addressing this need, our study investigates how multiscale design analytics support instructors’ assessment efforts in creative project-based learning contexts of design courses. Instructors in our study reported that multiscale design analytics can support them directly or indirectly in assessment and feedback processes. Instructors found that multiscale analytics have the potential to inform pedagogical intervention, based on whether or not students are able to effectively utilize the design environment. In I9’s words, “So if this number is extremely low for everybody...then maybe you need to [give] a tutorial [on the design environment].” Instructors shared that providing these analytics to students can help them reflect and improve their multiscale design skills. For example, in I1’s words, “I would love students to explore more zoom levels...they don’t really utilize being able to...zooming in to certain parts and elaborating.” “I would love students to explore more zoom levels...they don’t really utilize being able to...zooming in to certain parts and elaborating.” Implications. Our study demonstrates the potential of multiscale design analytics—which measure complex characteristics of design work—to assist instructors in assessing student work. The organization principles that design instructors expect their students to demonstrate map to the create category in Bloom’s revised taxonomy, i.e., “put elements together” into a “new, coherent whole” [2, 46]. As I4 expressed, “I think that I would definitely like to assign scales as a part of the rubric to say, I would like to see the big picture from out here, and then when you zoom in, see more.” Likewise, I1 expressed that cluster analytics could help students reflect on their design representation and become “more aware about how they separate”. We thus find that multiscale design analytics empower instructors in assessing students’ holistic thinking and creative capabilities. Implications Implications create new, coherent whole” “I think that I would definitely like to assign scales as a part of the rubric to say, I would like to see the big picture from out here, and then when you zoom in, see more.” We advocate for future research that investigates further development of multiscale design analytics, visual annotations, and dashboard interactions, as well as more diverse and in-depth studies, in order to develop new knowledge about how to support instructors, in developing effective pedagogical interventions, and students, in learning how to do design that involves thinking about and presenting complex information. This line of research has the potential to create new educational avenues for teaching how to present complex information that supports audiences in micro and macro readings—i.e., details and overviews [21]—and the formation of mental models [32] and maps. Since conveying and understanding such information is vital in so many areas of society, this mission has the potential for broad impact that benefits society, through the work these students will perform throughout their careers. Further, in design course contexts—where providing frequent feedback is vital—AI-based analytics demonstrate their utility in scaling the assessment. For example, I9 finds using analytics “better than having to go to every [design] and look for every single issue or having a much larger rubric [to run] by.” Instructors in diverse project-based learning contexts—e.g., arts and humanities [23]—engage students in creative, open-ended work. Thus, these contexts are similarly expected to benefit from analytics based on assessments of complex characteristics. “better than having to go to every [design] and look for every single issue or having a much larger rubric [to run] by.” The current research provides evidence for multiscale design measures to serve as descriptive analytics, i.e., analytics that provide insights into student work [76]. Going ahead, with data from the past iterations of a course, these analytics have the potential to function as prescriptive analytics, i.e., provide instructors and students alerts and suggestions based on computational modeling of the relationship between analytics and students’ course performance [6, 76]. On demand feedback through analytics has the potential to stimulate students’ learning-by-doing. Further, incorporating multiscale design analytics in widely distributed tools, such as Photoshop and Illustrator, has the potential to bring widespread benefits, as students use these tools in diverse design course contexts. 6.3 Limitations of Multiscale Design Analytics Multiscale design analytics are not a panacea. On the one hand, our findings show that the present multiscale design analytics provide value to instructors, in situated course contexts. I9 would like to see that students are able to effectively use the multiscale design environment. I2 values students’ presentation of structure at different levels. I1 would “love students to explore more zoom levels”, as she does not see them “zooming in to certain parts and elaborating”. “love students to explore more zoom levels”, “zooming in to certain parts and elaborating”. On the other hand, multiscale design analytics were not found to serve as a catch-all measure for design. I4 talks about not wanting all designs to look the same “like you don’t want to go somewhere and see every painting looks the same”. I1 says, “I would rather not control...how they see spatial clusters”. “like you don’t want to go somewhere and see every painting looks the same” “I would rather not control...how they see spatial clusters”. We build theory using creative cognition’s family resemblance principle, according to which, no particular characteristics are required for a work to be deemed creative [70]. Rather, a family of traits tends to serve as indicative. We find that multiscale design, as measured here, functions as one such design creativity trait. As another fruitful avenue for future research, we identify deriving analytics for families of design creativity traits—for example, feasibility, originality, and aesthetics [3, 20]; and gestalts principles, e.g., proximity, closure, continuity, symmetry, parallelism, and similarity of color, size, and orientation [79]—and applying these traits in education and even crowd-sourced design contexts. According to the family resemblance principle, as no particular trait is sufficient, design creativity analytics will never be perfect. But inasmuch as they work well enough, they can provide instructors, students, and other designers with insights so as to (1) provide first-order assessment; and (2) stimulate ongoing work. Authors:
(1) Ajit Jain, Texas A&M University, USA; Current affiliation: Audigent;
(2) Andruid Kerne, Texas A&M University, USA; Current affiliation: University of Illinois Chicago;
(3) Nic Lupfer, Texas A&M University, USA; Current affiliation: Mapware;
(4) Gabriel Britain, Texas A&M University, USA; Current affiliation: Microsoft;
(5) Aaron Perrine, Texas A&M University, USA;
(6) Yoonsuck Choe, Texas A&M University, USA;
(7) John Keyser, Texas A&M University, USA;
(8) Ruihong Huang, Texas A&M University, USA;
(9) Jinsil Seo, Texas A&M University, USA;
(10) Annie Sungkajun, Illinois State University, USA;
(11) Robert Lightfoot, Texas A&M University, USA;
(12) Timothy McGuire, Texas A&M University, USA. Authors: Authors: (1) Ajit Jain, Texas A&M University, USA; Current affiliation: Audigent; (2) Andruid Kerne, Texas A&M University, USA; Current affiliation: University of Illinois Chicago; (3) Nic Lupfer, Texas A&M University, USA; Current affiliation: Mapware; (4) Gabriel Britain, Texas A&M University, USA; Current affiliation: Microsoft; (5) Aaron Perrine, Texas A&M University, USA; (6) Yoonsuck Choe, Texas A&M University, USA; (7) John Keyser, Texas A&M University, USA; (8) Ruihong Huang, Texas A&M University, USA; (9) Jinsil Seo, Texas A&M University, USA; (10) Annie Sungkajun, Illinois State University, USA; (11) Robert Lightfoot, Texas A&M University, USA; (12) Timothy McGuire, Texas A&M University, USA. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Multiscale Design Analytics Offer Insight Into Students’ Creative Processes

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

3D Semantic Instance Maps: Leveraging Foundation Models for Language-Guided Navigation

AI Analysis Appears to Improve How Teachers Evaluate Student Design Projects, Study Finds

AI Tools Take Aim at Assessing Multiscale Design in the Classroom

AI-Powered Analytics Offer New Pathways for Evaluating Multiscale Design Work

The Research Artifact Making AI-Powered Design Insights Visible at Multiple Scales

AI Model Integration Streamlines Feedback in Multiscale Design Environments

3D Semantic Instance Maps: Leveraging Foundation Models for Language-Guided Navigation

AI Analysis Appears to Improve How Teachers Evaluate Student Design Projects, Study Finds

AI Tools Take Aim at Assessing Multiscale Design in the Classroom

AI-Powered Analytics Offer New Pathways for Evaluating Multiscale Design Work

The Research Artifact Making AI-Powered Design Insights Visible at Multiple Scales

AI Model Integration Streamlines Feedback in Multiscale Design Environments

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps