Nimisha Malreddy
Product Designer & Researcher based in New York City🗽🍎. Currently a Product Design Intern @ The Met Museum



About
Resume 
Art & Design 


Interactive & Inclusive Learning using AR.  




Role
Developer
UI Designer

Tools
Unity 3D, Vuforia, VS Code, 
Blender, Adobe Photoshop cc Contribution
Research
3D Modelling
UI Asset Design
Development Duration
16 weeks

Team
Self + 2 Student








Overview



This project is an attempt at developing an Interactive & Inclusive Learning Application using Augmented Reality for the Grade 10 (CBSE NCERT) Science textbook of India. The focus was mainly on the Biology chapters of the textbook since we felt it would help bring out the best of this application owing to the various unique diagrams included in this subject. 


Exciting news !!We successfully published a Research Paper in Springer journal, on the literature survey we conducted while developing this application.






Timeline










Identifying the problem statement


The lack of interactive learning tools in conventional school education prompted us to look for a solution that would not only make learning fun but also fosters inclusivity for students who may face challenges in concentrating without visual cues, including those who are deaf or mute.




90% of information transmitted to the brain is visual.65% of the world’s population are visual learners.400% improvement in learning with the incorporation of visuals.


We identified the need for an interactive, immersive and inclusive learning application to solve the problem of non- collaborative and un-engaging traditional learning. 







Literature survey highlights


We analysed 30+ research papers (published before 2022) related to augmented reality in the field of education to identify gaps in research and areas of improvement. The gaps in research can be best summarised below:

Lack of audio-assisted content.
Limited to only microorganisms.
No use of 3D assets, only AR multimedia such as videos.
Lack of information for the models used, i.e, no text/audio descriptions.
No quizzes integrated to test understanding.
Less to no user interaction with the visualization i.e., no buttons for toggle, zoom in/out, or rotate models options.






Our solution


Exploring the integration of augmented reality into education captivated our interest as a promising solution to enhance the interactive and engaging aspects of learning. The proposed system includes:

 Unity-based edu-application for grade 10 CBSE NCERT Science textbook.
3D models illustrating the biology chapters.
Audio/ Text description functionalities. 
Interactive models with zoom-in/ zoom-out functionality. 
Quiz section to assess students' understanding and facilitate effective study planning.



Low-fidelity wireframe exploration
Low-fidelity wireframes provided a rapid visualization of the design, allowing for quick iterations. They served as a foundational blueprint.








Development


01. Adding marker images to Vuforia database

AR marker images, featured in the textbook are uploaded in the Vuforia database. They are designed to trigger virtual content, creating an augmented reality experience. Only markers with a 4-star rating or above are selected.



02. Annotating 3D Models and developing an AR scene

3D models were obtained from an open source website called Sketchfab. The models were labelled in Blender and an AR scene was created in Unity 3D.


03. Developing the information panel

The information panel includes a description of the 3D model and an audio description in the form of text-to-speech. 


04. Integrating the quiz feature

Students can test their biology knowledge with multiple-choice questions, earning 10 points for each correct answer and losing 10 points for each wrong one. The total score is displayed at the end of the quiz.







Tada! The live prototype


Honestly, I wish it was this easy to produce a final prototype xD. If you would like to read all the technical jargon, view the detailed report in the appendix.

For now, here’s a demo of the prototype!









Feedback from our target users  


To test this application to the best of its ability, a sample of 30 users were asked to use this application and answer the survey questions listed below. These individuals are in the ages of 15/16.

75% of respondents found it easy to navigate
95% did not encounter any errors or issues in the application
85% felt like they were able to accomplish their tasks efficiently
90% found the language easy to understand





Appendix


Detailed Report
Published Research Paper