Table of Links Abstract and 1. Introduction Abstract and 1. Introduction Prior Work and 2.1 Educational Objectives of Learning Activities
2.2 Multiscale Design
2.3 Assessing Creative Visual Design
2.4 Learning Analytics and Dashboards


Research Artifact/Probe
3.1 Multiscale Design Environment
3.2 Integrating a Design Analytics Dashboard with the Multiscale Design Environment


Methodology and Context
4.1 Course Contexts
4.2 Instructor interviews


Findings
5.1 Gaining Insights and Informing Pedagogical Action
5.2 Support for Exploration, Understanding, and Validation of Analytics
5.3 Using Analytics for Assessment and Feedback
5.4 Analytics as a Potential Source of Self-Reflection for Students


Discussion + Implications: Contextualizing: Analytics to Support Design Education
6.1 Indexicality: Demonstrating Design Analytics by Linking to Instances
6.2 Supporting Assessment and Feedback in Design Courses through Multiscale Design Analytics
6.3 Limitations of Multiscale Design Analytics


Conclusion and References Prior Work and 2.1 Educational Objectives of Learning Activities
2.2 Multiscale Design
2.3 Assessing Creative Visual Design
2.4 Learning Analytics and Dashboards Prior Work and 2.1 Educational Objectives of Learning Activities Prior Work and 2.1 Educational Objectives of Learning Activities 2.2 Multiscale Design 2.2 Multiscale Design 2.3 Assessing Creative Visual Design 2.3 Assessing Creative Visual Design 2.4 Learning Analytics and Dashboards 2.4 Learning Analytics and Dashboards Research Artifact/Probe
3.1 Multiscale Design Environment
3.2 Integrating a Design Analytics Dashboard with the Multiscale Design Environment Research Artifact/Probe Research Artifact/Probe 3.1 Multiscale Design Environment 3.1 Multiscale Design Environment 3.2 Integrating a Design Analytics Dashboard with the Multiscale Design Environment 3.2 Integrating a Design Analytics Dashboard with the Multiscale Design Environment Methodology and Context
4.1 Course Contexts
4.2 Instructor interviews Methodology and Context Methodology and Context 4.1 Course Contexts 4.1 Course Contexts 4.2 Instructor interviews 4.2 Instructor interviews Findings
5.1 Gaining Insights and Informing Pedagogical Action
5.2 Support for Exploration, Understanding, and Validation of Analytics
5.3 Using Analytics for Assessment and Feedback
5.4 Analytics as a Potential Source of Self-Reflection for Students Findings Findings 5.1 Gaining Insights and Informing Pedagogical Action 5.1 Gaining Insights and Informing Pedagogical Action 5.2 Support for Exploration, Understanding, and Validation of Analytics 5.2 Support for Exploration, Understanding, and Validation of Analytics 5.3 Using Analytics for Assessment and Feedback 5.3 Using Analytics for Assessment and Feedback 5.4 Analytics as a Potential Source of Self-Reflection for Students 5.4 Analytics as a Potential Source of Self-Reflection for Students Discussion + Implications: Contextualizing: Analytics to Support Design Education
6.1 Indexicality: Demonstrating Design Analytics by Linking to Instances
6.2 Supporting Assessment and Feedback in Design Courses through Multiscale Design Analytics
6.3 Limitations of Multiscale Design Analytics Discussion + Implications: Contextualizing: Analytics to Support Design Education Discussion + Implications: Contextualizing: Analytics to Support Design Education 6.1 Indexicality: Demonstrating Design Analytics by Linking to Instances 6.1 Indexicality: Demonstrating Design Analytics by Linking to Instances 6.2 Supporting Assessment and Feedback in Design Courses through Multiscale Design Analytics 6.2 Supporting Assessment and Feedback in Design Courses through Multiscale Design Analytics 6.3 Limitations of Multiscale Design Analytics 6.3 Limitations of Multiscale Design Analytics Conclusion and References Conclusion and References Conclusion and References A. Interview Questions A. Interview Questions 5.3 Using Analytics for Assessment and Feedback The more design classroom sizes continue to grow, the more that instructors are challenged in having time to provide optimal levels of feedback to each student [47]. Prior studies have found analytics useful toward scaling assessment and feedback [49, 60]. Our findings for multiscale design analytics align. I1 and I4 expressed that the analytics can become a part of their rubrics and feedback they give to students. Further, according to I4, making these analytics a part of rubrics can motivate and provide students guidance on what instructors are looking for in their design. I1: I think [these analytics and my rubrics] complement each other. I think it will be very helpful...if there’s a way that I can just sort of make a rubric on [dashboard] and attach to when they get their feedback. I think [these analytics and my rubrics] complement each other. I think it will be very helpful...if there’s a way that I can just sort of make a rubric on [dashboard] and attach to when they get their feedback. I4: You know, give them something to shoot for...I think that I would say...here are the things that I’d like to see in your design...I think that I would definitely like to assign scales as a part of the rubric to say, I would like to see the big picture from out here, and then when you zoom in, see more. You know, give them something to shoot for...I think that I would say...here are the things that I’d like to see in your design...I think that I would definitely like to assign scales as a part of the rubric to say, I would like to see the big picture from out here, and then when you zoom in, see more. We observe the potential of multiscale design analytics toward expediting instructors’ assessment work. I9 preferred to utilize analytics as a quick-to-use indicator of underlying problems. The analytics help them reduce the time they otherwise would spend on assessing each design. I9: So, I won’t use the values in the column to directly give them points...But it’s better than having to go to every [design] and look for every single issue or having a much larger rubric that I ran by...So think of the analytics as the symptoms and [then] you actually identify diseases. So, I won’t use the values in the column to directly give them points...But it’s better than having to go to every [design] and look for every single issue or having a much larger rubric that I ran by...So think of the analytics as the symptoms and [then] you actually identify diseases. 5.4 Analytics as a Potential Source of Self-Reflection for Students Instructors (I1, I4, I5, I6) in our study expressed expectations that students would benefit from seeing multiscale design analytics. According to them, seeing analytics has the potential to help students reflect on their progress. More specifically, seeing analytics can help students in becoming aware of how they are organizing their ideas spatially across scales and clusters. Self-reflection through analytics plays a vital role in learning, as it helps students in understanding their progress and stimulates improvements in their work [77, 81]. I5: I’m all for giving students as much information as they can use...and you know...they can use [analytics] to look at their progress. I’m all for giving students as much information as they can use...and you know...they can use [analytics] to look at their progress. I1: Yeah, I would love students to explore more zoom levels...because usually, I think it is more like...I see it as an overall picture...but they don’t really utilize being able to kind of go in to certain areas or zooming in to certain parts and elaborating...[Also,] maybe spatial clusters just so that they could be more aware about how they separate. Yeah, I would love students to explore more zoom levels...because usually, I think it is more like...I see it as an overall picture...but they don’t really utilize being able to kind of go in to certain areas or zooming in to certain parts and elaborating...[Also,] maybe spatial clusters just so that they could be more aware about how they separate. While both I1 and I4 advocate for providing students with multiscale design analytics, they also caution against enforcing a specific type of visual organization. According to them, the goal of providing analytics would be to help students to reflect and effectively use multiscale organization, not to have a specific number of scales or clusters across scales. I4: I wouldn’t want them all to look the same like you don’t want to go somewhere and see every painting looks the same, but it was almost as if some people were painting with boards and nails and hammers versus paintbrushes and paint. They just didn’t really get what they’re supposed to be putting on the [multiscale design]. So then it was just like not as effective. I wouldn’t want them all to look the same like you don’t want to go somewhere and see every painting looks the same, but it was almost as if some people were painting with boards and nails and hammers versus paintbrushes and paint. They just didn’t really get what they’re supposed to be putting on the [multiscale design]. So then it was just like not as effective. I1: [While] they have to become a little bit more mindful of [space]...just seeing how they lay out everything themselves...I would rather not control whether intentionally or unintentionally at all how they see spatial clusters. [While] they have to become a little bit more mindful of [space]...just seeing how they lay out everything themselves...I would rather not control whether intentionally or unintentionally at all how they see spatial clusters. Authors:
(1) Ajit Jain, Texas A&M University, USA; Current affiliation: Audigent;
(2) Andruid Kerne, Texas A&M University, USA; Current affiliation: University of Illinois Chicago;
(3) Nic Lupfer, Texas A&M University, USA; Current affiliation: Mapware;
(4) Gabriel Britain, Texas A&M University, USA; Current affiliation: Microsoft;
(5) Aaron Perrine, Texas A&M University, USA;
(6) Yoonsuck Choe, Texas A&M University, USA;
(7) John Keyser, Texas A&M University, USA;
(8) Ruihong Huang, Texas A&M University, USA;
(9) Jinsil Seo, Texas A&M University, USA;
(10) Annie Sungkajun, Illinois State University, USA;
(11) Robert Lightfoot, Texas A&M University, USA;
(12) Timothy McGuire, Texas A&M University, USA. Authors: Authors: (1) Ajit Jain, Texas A&M University, USA; Current affiliation: Audigent; (2) Andruid Kerne, Texas A&M University, USA; Current affiliation: University of Illinois Chicago; (3) Nic Lupfer, Texas A&M University, USA; Current affiliation: Mapware; (4) Gabriel Britain, Texas A&M University, USA; Current affiliation: Microsoft; (5) Aaron Perrine, Texas A&M University, USA; (6) Yoonsuck Choe, Texas A&M University, USA; (7) John Keyser, Texas A&M University, USA; (8) Ruihong Huang, Texas A&M University, USA; (9) Jinsil Seo, Texas A&M University, USA; (10) Annie Sungkajun, Illinois State University, USA; (11) Robert Lightfoot, Texas A&M University, USA; (12) Timothy McGuire, Texas A&M University, USA. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Dash

dashboard

Microsoft

Educators Embrace Multiscale Analytics to Help Students Understand Their Design Processes Better

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

3D Semantic Instance Maps: Leveraging Foundation Models for Language-Guided Navigation

AI Analysis Appears to Improve How Teachers Evaluate Student Design Projects, Study Finds

AI Tools Take Aim at Assessing Multiscale Design in the Classroom

AI-Powered Analytics Offer New Pathways for Evaluating Multiscale Design Work

The Research Artifact Making AI-Powered Design Insights Visible at Multiple Scales

AI Model Integration Streamlines Feedback in Multiscale Design Environments

3D Semantic Instance Maps: Leveraging Foundation Models for Language-Guided Navigation

AI Analysis Appears to Improve How Teachers Evaluate Student Design Projects, Study Finds

AI Tools Take Aim at Assessing Multiscale Design in the Classroom

AI-Powered Analytics Offer New Pathways for Evaluating Multiscale Design Work

The Research Artifact Making AI-Powered Design Insights Visible at Multiple Scales

AI Model Integration Streamlines Feedback in Multiscale Design Environments

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps