Creating an Immersive AR Experience: Lessons From a Hackathon

In February 2022, a former classmate from my UX Bootcamp invited me to join a hackathon organized by Hackmakers DAO. I jumped at the chance to explore a new challenge. But, for those unfamiliar with Hackmakers DAO or the competition's theme, let me provide an overview.

Hackmakers DAO is an organization that manages global hackathon events with sponsoring organizations. The 2022 FormulaAI 4-day Hackathon, run remotely, was a collaboration with Oracle and focused on the theme of "Driven by data." The event called for talented individuals with skills in;

• Data analysis,

• Augmented reality (UX/AR/iOS Swift Designers/Developers), and

• 3D modeling.

  
  

Our project focused on leveraging augmented reality to allow users to replay a 3D experience of a sporting event, providing a unique perspective for analyzing performance. Although I did not know what to expect, being that it is my first ever hackathon, it was an invaluable experience.

Assembling the team

Eligibility to participate in the challenge required everyone to be in a team of no less than 2 people. After the live introductory meeting with the organizers, we began the process of finding teammates and mentors. The organizers had set up channels for finding and recruiting teammates from over 2000 participants. My first challenge was finding others who were interested in and had the skills we needed for the AR challenge.

After a long search, We successfully found 2 interested participants, making us a team of 4, with 2 Ux designers, 1 iOS Swift developer, and a 3D modeler.

My Role and Responsibility

I served as the Interaction Designer in a team of two UX persons and as the lead team moderator. As team moderator, I delegated each day in the hackathon to specific tasks, proactively setting up channels to ensure that we would have easy access to needed resources for the duration of the event.

• Day 1 Ideation
• Day 2 Design, Tests & Reiterations
• Day 3 More Tests, Development & Testing
• Day 4 Last minutes, Wrap up & Early Submitting

The Ideation Process

For us designers, our unique challenge in the words of the organizers was;

“Think about any events that generate 3-dimensional data, and how those can be translated into impressive replays. We would like for you to tell us how UX should work and look like to deliver the most immersive AR Replay experience... Swift UI is preferred”.

With that in mind, we began describing the problem, referring to the existing design while individually providing points for improvement.

The Target Users

Formula One racing enthusiasts and spectators.

Assumptions we made about our users:

• Some users may be unfamiliar with augmented reality.

• They may become overwhelmed or frustrated if they see a large amount of information on their screen while the AR sessions are running.

  
  

For this reason, each teammate got to interact with AR apps, studying the UI components and interaction of AR experiences. We concentrated on how information was presented, finally, agreeing to keep the experience as focused, simple, and uncluttered as possible.

The Design Process

The design process proved to be more challenging than I had imagined. Approaching AR as a designer, I quickly understood how immersive it is. It was even clearer how huge the difference gap is between AR and conventional web or smartphone interactions.

The specific challenges we experienced included;

• Displaying information to overlay over existing real-world surroundings at varied distances while ensuring that the user's field of view was not obstructed;
• Providing the user with the ability to readily access all of the material accessible to them without feeling overwhelmed by information cluttering their already constrained field of view.

We wanted to express simplicity and clarity so that users could engage with the interface successfully to achieve their goals.

The Design Decisions

As our developer worked on integrating the current API into our project, us Ux duo began mapping the experience flow. Our objective was to create a flow that was essentially easy to navigate even for a complete AR novice.

We designed the AR page to hold only the current replay session and necessary information that adds to the experience. Other information was placed in select parts according to their category and functions, accessible by 2 on-screen buttons, sized and placed strategically to allow easy access without distractions. The left category grants access to other replay sessions which allows users easily switch between sessions. The right category allowed users to control and record the current replay session.

We designed and developed as a team alongside our developer and modeler, sharing and testing ideas as we prototyped in real-time. When we were satisfied that the most basic version of our goal had been reached, it was time to hand over to the developer.

The Fourth Day

It was both exhilarating and scary to present our final product to the rest of the hackathon participants. More distressing was our last-minute problem, in which our developer vanished without a trace. But I'm pleased with how everything came out. It was great fun working with such knowledgeable, talented people and completing our tasks under a tight schedule.

Lessons Learned

A useful takeaway was learning how to work efficiently with global team members remotely. Personally, I learned to adapt my communication style to different personality types. I became more proactive to analyze and take on responsibility when needed, promoting participation, cultivating diversity, and implementing short deadlines. These lessons will accompany me on my future adventures. I am grateful to Hackmakers for organizing this event. I look forward to next year's challenge!

If you'd like, try out our prototype. I'd love to hear your feedback. 🖤

Also published here.

lead image generated with stable diffusion.