VREducation

Roadmap Creators: | Divya Iyer | Don Duval | Mridula Prakash

Time Stamp: 5 December 2023

Roadmap Overview

Virtual Reality (VR) in education offers an immersive, interactive, and engaging environment, revolutionizing traditional education and teaching methods. By simulating real-world experiences, VR promotes experiential learning, allowing students to visualize complex concepts, explore inaccessible locales, and engage in realistic scenarios. VR technology fosters better comprehension, retention, and enthusiasm for subjects ranging from history to science to training in skilled labour industries and beyond. In classrooms, VR can transport students to ancient civilizations, distant planets, or even inside human cells. For skills training, such as medical or technical fields, VR provides safe, repeatable environments for practice, reducing risks and resource costs. Moreover, VR caters to diverse learning styles, accommodating visual, auditory, and kinesthetic learners. With the potential for global virtual classrooms, students from varied geographies and backgrounds can interact, democratizing education access. While challenges remain, including technological equity and motion sickness, the transformative potential of VR in education is undeniable.

Our technology roadmap identifies the importance of the integration between hardware technical capabilities – including the VR headset and hand-held devices – along with the educational content and curriculum. Getting both “right” is integral to the future utilization and success of using VR as an immersive learning tool. Our roadmap demonstrates that VR for education is still in the “Rapid Progress” phase of the traditional “S Curve” and countless additional / supplementary innovations are still forthcoming such as utilizing AI for bespoke / custom learning opportunities for students / trainees around the world. Our technology roadmap lives at the epicenter of three key aspects as outlined in Figure I below. This graphical representation integrates well with the Mixed Reality technology roadmap outlined in (https://roadmaps.mit.edu/index.php/Mixed_Reality_(Augmented_%26_Virtual)#Roadmap_Overview). The key difference is the extended and deliberate engagement of educational content and curriculum into our roadmap.

• Human Operator. This includes both the trainee and the instructor that may (or may not) be involved in the training and educational experience. Some of the hands-on technical training using VR may still leverage the opportunity for an in-person instructor such as training in hazardous situations or medical environments.

• Content / Curriculum and Learning Environment. Student / trainee educational outcomes are rooted in the development and deployment of the “right” educational content and curriculum in the VR immersive experience.

• VR Hardware / Computer Systems. The quality, cost, and user experience with the VR hardware and supporting computer systems will be integral to the success of our VR for Education technology development and deployment. Significant investments and progress has been made and continue to position VR to transform the global education market.

Position in Technology Matrix

Our technology of choice is VR for Education. Per below, VR for Education is indicated in the “Exchanging Information” category because it involves exchanging educational content in the form of information with students. The VR hardware (headset and hand controllers) combined with educational content is a mechanism to exchange information between a student/trainee and the content delivered/exchanged via the VR technology.

DSM Allocation

In order to create the roadmap, the dependencies involved in the related technologies was explored. The DSM methodology has been used to understand the various levels of the components and the interdependencies between them. The physical and informational/digital relationships have been mapped. The DSM is constructed in such a way that the flow of connectivity is from the horizontal element in the left to the vertical element in the top. For the DSM, the entire VR education concept is considered in the first level and then it is broken into the educational content and the enablers involved as

Level 1 : VR Based Education

Level 2 : Hardware, Software, Educational Content and Human Participants

Level 3 : Further Decomposition of Level 2 into lower-level components

DSM Tree

The DSM tree indicates how the levels are decomposed.

Figure : DSM Tree of VR Education

DSM Matrix format

The decomposed components of all levels have been considered to create the DSM in a matrix format. The physical and informational exchanges have been indicated.

Figure : DSM of VR Education

On analyzing the DSM, it is possible to observe Technology Pullers and Pushers. The VR Headset's Hardware plays a major role as both a technology pusher and puller. The success of VR-based Education will depend heavily upon the quality and performance of the VR Headsets. This is also later analyzed in the Technological models.

From the DSM, it is also evident that the Human participants involved in the VR Education process play a significant role in the success of the technology. The key factors that need to be considered are

1. The way in which the VR Developers develop the technology for suitability to education
2. The way educators incorporate the educational information into the system
3. The way educators utilize the available features and encourage the students to explore the features
4. The way students themselves understand and explore the technology

The understanding of the interdependencies is useful to plan the roadmap in such a way that the key dominating factors are considered appropriately and given due importance.

Roadmap Model using Object Process Methodology(OPM)

In order to understand the Technical Roadmap better, the functions, forms associated with VR based education, and interactions between them are studied. In order to study this, the OPM diagram of the system was generated on the OPCloud.

The primary focus is on the components of the VR headset, its performance and the various Human participants in the system. Their interaction has been captured along with the key FOMs.

Included in our OPM are six Figures of Merit, which are explained in detail in Table of FOMs: • Two of the FOMs are related to the educational / training outcomes associated with our technology roadmap • Three of the FOMs are related to the VR headset hardware • One FOM is related to the unit cost to build both the hardware and the content/curriculum to support the educational aspect of our system. Please note, that our OPM integrates well with and builds upon the existing Mixed Reality Technology Roadmap in the EM.427 portal: (https://roadmaps.mit.edu/index.php/Mixed_Reality_(Augmented_%26_Virtual). However, given our focus on education/training, we have expanded our OPM to incorporate FOMs such as User Experience and Content Knowledge Retention.

Figure : OPM Diagram of VR Education system

From the OPM diagram, it becomes more clear that the success of the system depends upon the VR headset quality primarily and it is also possible to understand how the various forms of the system come together to perform the different functions necessary.

The content of the education plays a major role in the success of the system. Only if the content developers use the VR system to the full potential, it will be possible for the students to immerse better into the experience. This is included in the "Education Content" component. Further deep diving into this particular component has not been considered for this roadmap.

Figures of Merit

VR technology engages/immerses users in a computer-generated, interactive 3D environment creating a user experience that gives them an illusion that they are physically present in the environment. The figures of Merit have been identified to measure the progress of the technology and also to understand which parameters are of most importance to the roadmap. The key figures of merit have been listed below.

Table : Figures of Merit of VR Based Education

Hardware-Related Figures of Merit

• Display Resolution:

This refers to the number of distinct pixels in each dimension that can be displayed. It's often given as a pair of numbers, which represent the number of pixels in the horizontal and vertical directions. For example, a resolution of 1920x1080 means there are 1,920 pixels horizontally and 1,080 pixels vertically. As a measurement calculation, we assume in VR that each eye will have the same resolution and to calculate the overall display resolution we multiply the horizontal and vertical directions to get the total display resolution. A higher resolution allows for a sharper and clearer image. In VR, this can contribute to the realism and immersion of the experience.

Unit of Merit : Number of Pixels  

• Latency:

This is the delay between a user's movement and the corresponding change in the VR display measured in milliseconds. A lower latency provides a more meaningful experience. The figure of merit measurement calculation is calculated via: End Time – Start Time of Desired Movement. The current expectation in VR is approximately 20 milliseconds.

Unit of Merit : milliseconds  

• Unit Cost:

The total cost to purchase the hardware – including both the VR headset and the VR hand controller and the cost to develop and build the content/curriculum. The measurement calculation is calculated using US dollars. On our OPM, you will notice three distinct Unit Cost ($USD) entities connected to the VR Headset, the VR hand controller, and the Content / Curriculum respectively.

Unit of Merit : USD  

• Refresh Rate:

It indicates the number of times in a second the display redraws or refreshes the image. The measurement calculation is calculated using Hz via the Number of Display Refreshes per Second. For example, a 60Hz display refreshes its image 60 times per second. Higher refresh rates typically result in smoother and clearer motion representation. In the context of VR, this can make the virtual environment feel more real and reduce motion sickness for some users.

Unit of Merit : Hz

User Experience-Related Figures of Merit

• User Experience Rating:

Using the Net Promoter Score this figure of merit is calculated by asking the user (student/trainee) the following question: “Would you recommend this learning experience to others?”. The measurement calculation would give the users only two answers: “Yes” or “No”. As the population of student/trainee feedback grows using the VR educational experience, we would have a percentage indicating “Likelihood to Recommend” calculated by (Total “Yes” answers) divided by (Total Users) x 100%. This would give a percentage rating related to the likelihood to recommend – an indicator of overall user experience.

Unit of Merit : Marked on customized scale  

• Content Knowledge Retention:

Given that our technology is VR for Education, an important figure of merit is content knowledge retention. The measurement calculation would be rooted in standardized in-VR environment testing. The measurement would be a percentage calculated via: Number of testing questions answered correctly divided by the total number of questions x 100%. A higher content knowledge retention score would be an indication that the learning experience is providing the student/trainee would learned knowledge.

Unit of Merit : Number or count

Among the various Figures of Merit, outlined in our OPM, we selected to focus on the evolution of Display Resolution as our FOM of choice. The reason we selected this FOM was rooted in our research to validate the importance and influence the display resolution has on the user experience and overall engagement with VR. Simply put, the better the display resolution, the forecasted better experience the trainee / student will have using VR. That said, we recognize the importance of other variables that must be considered in tandem such as refresh rate, computing power, etc. Outlined in the graph below, we have tracked the progress of display rate using the FOM measured in pixels per eye. We have started our graph in 1985 with the development / deployment of the NASA / VPL VR headset which is often regarded as the first true VR headset. That said, in our research we did identify an interesting evolution of VR headset prior to 1985 including the “Sword of Damocles” developed in 1968 by computer scientist Ivan Sutherland (a MIT graduate), who was aided by his students Bob Sproull, Quintin Foster, and Danny Cohen. However, the mechanics of this system did not use pixel measurement and therefore no data exists to estimate the display resolution – which can be assumed to be rather primitive.

Figure : Figure of Merit: VR Display Resolution (Total Pixel Count) Improvement Over Time

Theoretical Limit Estimate

On our graph, we have estimated our theoretical limit based on an article assessing the level of resolution “generally accepted to be what the human eye can discern.” (6) The article infers that a resolution of 16,000 x 16,000 pixels equating to 256,000,000 pixels is the limit that the human eye can discern. However, this inference is only an estimate, but we felt the science beyond inferring what the human eye can discern in terms of resolution quality was a great estimate of the theoretical limit for VR headsets / visual displays. Furthermore, when looking at our graph and the trendline and where the VR technology is today, we believe, as the article further cites “It’s yet another reminder that headsets today are still just the beginning of VR, and it has a long and promising path ahead.”

Rate of Improvement Estimate and Lifecycle Technology Maturity Estimate (S Curve Location)

Using our source data and research outlined in Table 3, we have estimated the exponential growth year-over-year percentage of display resolution for VR headsets using the following assumptions:

• Modern-era VR headsets began in 1985 with the NASA / VPL VR headset. Based on the best research we could find, the display resolution at this time was approximately 4,096 pixels.
• In 2023, the Apple Vision Pro is estimated to have a total pixel display resolution of 22,800,000.
• To calculate the estimated Compound Annual Growth Rate (CAGR) of the display resolution using pixels over the modern ear, we used a 38-year timeframe (2023 – 1985). CAGR formula: (Ending Value/Beginning Value) ^ (1/No. of Periods) – 1

Using the CAGR formula the estimated annual technology progress rate is 25%, which coupled with our research outlined in Table below, suggests VR headset technology is still in the “Rapid Progress” stage (as outlined in Figure 4.15 in the class textbook). Three trendlines and corresponding equations have also been outlined on the graph using the least squares regression/optimization.

Table : Source Data and References for Display Resolution Figure of Merit Calculation

Alignment with Company Strategic Drivers : FOM Targets

For VR-based education, the strategic drivers that the industry demands are listed here. The Strategic Drivers, motivation behind these drivers and the FOM targets are mentioned below. The goal here is not to pick the highest display quality, but to find a balance between the display, cost and the educational impact.

Our “hypothetical” company is an education-based company using VR to transport the information to the student / trainee in a meaningful immersive method. Our strategic drivers, motivation behind the strategic drivers, and the alignment to our select FOM’s are outlined in the Table below.

Our desired outcomes, as measured by our FOM’s, are rooted in both educational outcomes (user experience rating and knowledge retention) combined with more tactical hardware FOM’s rooted in the VR headset (display resolution, latency, and refresh rate). When we look at the V-Red product, we envision a “good enough” hardware solution at a lower cost that can meaningfully convey content to support the trainee / student in achieving the desired learning outcomes.

Table : Alignment with Company Strategic Drivers

Positioning of Company Vs. Competition : FOM charts

The key players in the VR space are Meta, Sony and Apple (whose details are not completely available at this point). For our competitive analysis, we have selected to review the global VR headset/hardware marketplace because the learning experience depends on the quality of the VR experience.

Table : Market Share Overview: Leading VR Headset Manufacturers

Comparing the cost vs performance of these headsets results in the below representation.

Figure : Comparison of Market competitors

It can be concluded that this domain has very few players and they are competing rigidly on the technology as well as pricing strategies. It will be interesting to see the impact of the Apple VR / AR headset on the technology and pricing strategies of Meta and Sony. In order to make VR Education affordable, V-Red plans to develop a high-resolution headset at a lower cost. That said, based on the emerging market trends, we believe the growth opportunities for immersive learning will be combined VR / AR headsets with strong access to content libraries – thereby creating alternative revenue streams for the incumbents and new market entrants rooted in software enhancements and potential leading to a premium pricing model. Part of the rationale for this opinion is rooted in the core hardware technology within VR / AR headsets is getting closer to the point that will ensure the overall user experience will be positive – thereby driving meaningful educational assuming the pedagogical aspects of the educational content are designed correctly.

Technical Model (Modeling and Sensitivity of FOMs 1 & 2)

To understand the working of the technology, various parameters and FOMs impacting the performance of the VR based education are considered. Initial focus was to understand the market share of products and their latency and display resolution. We analyzed both display resolution and latency to understand where they fit within the specifications of the VR headsets for the market leaders – Meta and Sony – for their current in-market product at their current price.

Figure: Latency and Display Resolution Specs aligned to current Market Share Leader Products

Technical Modeling - Latency

Starting with latency, research insights have indicated VR headset manufacturers are seeking to reduce latency to below 5 milli-seconds, but the question becomes two-fold: First, can a human being identify a latency difference between 2 milli-seconds vs. 5 milli-seconds vs. 10 milli-seconds and second, are they willing to pay for the incremental improvement? Research we conducted indicated that “while humans can detect very small relative delays in parts of the visual field, when absolute delays are below approximately 20 milliseconds, they are generally imperceptible.” (14) When we conduct a regression analysis assessing to assess the sensitivity of incremental unit cost of a VR headset relative to latency improvements – over the past ten years based on our research / source information – we derive a linear equation as labeled on the charts below.

Figure: Sensitivity Analysis Comparing Latency and Market Cost of VR Headset

This model illustrates an interesting analysis in that, if we assume the current best-in-market latency for VR headsets is 20 milli-seconds and this corresponds to current products of the duopoly market share leaders Meta and Sony, the question becomes what would be the incremental market price impact, if we improve latency by 10 milli-seconds (50% improvement) – essentially getting latency to 10 milli-seconds. Using our regression equation model, we demonstrate the impact on total cost of the VR headset could be ~$1,311 USD – an increase of over 150% in consumer cost. Given our understanding of the incumbent VR headset price that dominates over 80% of the market ($500 to $550 USD per VR headset), we believe that consumers would not be willing to pay for this.

Technical Modeling – Display Resolution

Similar to the model we developed for latency, our model for Display Resolution derived an equation for the sensitivity on consumer price of a VR headset on display resolution improvement using the past ten years of display resolution/cost data. In this example, a 50% improvement in display resolution (measure in total pixels) over the incumbent market share leaders’ suggests a market cost of $1,171USD for the VR headset – an increase of >200% over the incumbent products of the current market leaders.

Figure: Sensitivity Analysis Comparing Display Resolution and Market Cost

Hence it can be seen that the market leading VR headsets have a minimum latency of ~15 milliseconds and a total pixel count of ~22,000,000 pixels.

Technical Model Discussion Each of our selected FOM’s have distinct governing equations and four to five variables within each equation that influence each FOM’s. In each equation, both display resolution and latency play a critical role and for added interest / insight we provided a regression analysis and sensitivity analysis to show the impact on these specific variables on cost of the VR headset in the consumer marketplace for each of these variables. For building the model, the below parameters were shortlisted. Apart from the Display resolution and latency, other factors that impact the VR Education have been considered. Beyond the supplementary and interesting analysis of display resolution and latency, our technical model is focused on Display Quality and Educational Output. We have highlighted the parameters influencing these FOM’s and derived equations.

Display Quality The model was built to identify the various combinations and possible outputs based on these parameters. A Tradespace was generated. Various combinations of the parameters are considered. This FOM assesses the total display quality experience by the trainee / student immersed in the VR experience (rooted in tangible parameters related to hardware / software of the VR headset)

Display Quality = (Display Resolution * Screen Diagonal^2) / [(Screen Area * Viewing distance^2) * (1 – Latency factor)]

Educational Output This FOM assesses the total educational outcome experience by the trainee / students immersed in the VR experience (rooted in the user experience and the parameters influencing the educational outcomes / desired knowledge retention of the trainee. This FOM is quite unique and is based on actual feedback that NORCAT, the VR company that Don Duval works for, from data and insight about educational outcomes of students using the VR immersive headset training experience. We recognize there subjectivity with this FOM, but for this assignment and our technology roadmap we felt it was important to include this given our focus is on VR for Education).

Educational Output = [(Content Quality Factor * Display Resolution) + (Immersion Factor * Screen Area)] * (Interaction Factor – Latency factor)

Due to the use of constants in computing, we will assume that the FOM is a dimensionless quantity as the units will get cancelled out by the use of the constants which will counteract on the units.

Technical Model : Morphological Matrix and Tradespace

Morphological Matrix The morphological matrix was created by comparing the key differentiating variables between the key players of the market. Apple is not considered because its specs are still not clear. These variables govern the FOMs.

Table : Morphological Matrix considering 3 main players

Figure : Tradespace for VR for Education

With that information, we identified which variables are fixed and which were modified – feeding into the creation of three unique models (Model 1, Model 2, and Model 3). Using that information, we developed an initial definition of our tradespace that analyzed the tension between our two selected FOM’s for each of Model 1, 2, and 3) and highlighted our utopia front and an estimation of our pareto front. As you would expect, our utopia point is where the display quality is “high”, and the corresponding education output is “high” (top right corner of our chart).

Sensitivity Analysis and Normalized Sensitivities

The sensitivity analysis was performed by making derivatives for the equation. For the sensitive parameters, normalization was applied to understand how 1% increase in the parameters impact the performance of the overall VR Education itself. In our normalized sensitivity analysis outlined in our technical model, we assessed the impact on each FOM by modifying each parameter by 1% (arbitrary for illustrative purposes) and then calculated the impact using partial derivatives for each variable based on our FOM equations. As can be seen in our Tornado Charts, the key parameters impacting each FOM is unique:

For Display Quality:

Figure : Tornado Diagram of Display Quality

The primary parameter that has the most impact on display quality is screen area. Rationale behind this makes sense, as if we fixed the pixelation / display resolution and increase the screen area, the overall display quality will be reduced impacting the overall experience.

For Educational Output:

Figure : Tornado Diagram of Educational Output

The primary parameter that has the most impact on educational output is interaction factor – the responsiveness and engagement of the VR experience by the trainee – measured from direct feedback and interaction with the educational journey the student / trainee is on.

Discussion

Based on these FOMs, it is possible to understand the way the VR Educational output will change based on the different features of the VR Headset. To understand the impact deeper, more detailed evaluation is necessary. The factors influencing the Educational output like Immersion, Interaction and Content Quality have been derived based on a research at NORCAT. More samples may result in the values for specific subjects and specific age groups. This will help in finetuning the performance to reach better results. It is also needed to further enhance our variables to better understand the key functions and form of forthcoming VR headsets that are driving market share and meeting customer expectations.

One of the more interesting analysis, using the Tornado diagrams, relates to the real-life market entry strategy of Apple’s Vision Pro – which is being sold at $3,500 USD and yet the latency and display resolution improvements over the incumbent market share leading products is seemingly very modest…if at all (fully specifications details are still being revealed). Given that, using our exclusively these two FOM’s, it makes absolutely no sense for the Apple Vision Pro to be priced at $3,500 USD. Therefore, the specifications driving the significant valuation / consumer product cost of the Apple Vision Pro must be rooted in FOM’s other than latency and display resolution. Based on our research, we believe the combined AR / VR functionality, access to content from Apple, and other variables are driving their pricing and market positioning strategy.

Financial Model

The NPOV is used for the financial modelling of the system. As mentioned, the strategy is not to develop a VR headset from scratch but to use the available market products to incorporate them into the educational system. The company will focus on building the software and content for the educational system and most of the costs in the first 5 years will be spent in that direction.

After the initial working model has been established, the company will enter the early stage of market testing for the next 5 years but the cash flow is still expected to be negative.

After 10 years, it is expected that the technology will ramp up and grow tremendously. This is also practically feasible because in the next 10 years VR headsets will become much more affordable and will be much more accessible to a larger market of students. Hence the financial model can be used as an indicator of the maturity of technology's revenue.

Figure : Financial Model of VR Education System

Key Publication and Patents

Utilization of Patent Search Process

This technology roadmap includes information on literature trends, papers published at key conferences, and patents. Summaries of the publication types and companies involved are found below.

Figure: Patent filing per year (Reference: https://www.mdpi.com/2076-3417/12/3/1755)

Figure illustrates the yearly filing of patents from 2016 to 2020, We observed an initial period spanning the years 2016 and 2017, with no patents registered. This trend changed in 2018, followed by growth in 2019 and 2020. This showed 2016 marked as a technological breakthrough in the domain of virtual reality. Before 2016, commercial virtual reality systems required users to connect a headset, controllers, and sensors to an external high-end computer, which was an expensive, bulky, and inconvenient setup.

Figure: VR and AR patent applications per year (Reference: https://www.iam-media.com/article/how-patents-are-shaping-virtual-and-augmented-reality)

Figure illustrates the number of patent applications between 2010 and 2018. Patent filings have more than doubled in the past four years. The data confirms that a growing number of companies are filing for patent protection.

Figure: Graph shows top ten jurisdictions for VR and AR patent filings (reference: https://www.dolcera.com/web/blog/ar-vr-everywhere-but-not-a-clear-winner/)

As shown in the graph, the US accounts for ~32% of the global IP filings and is at the top place followed by China.

Patents

Below are the patent records retrieved using the search strategy. Rather than a relatively equal-sized portfolio, large portfolios are held by a few companies, which indicates an active competitive market with strong investments by multiple companies, suggesting that the market is difficult to enter. During our research, we discovered a large number of assignees, each with a small number of records, demonstrating a developing technology space.

Table : List of prominent patents From the list of patents identified, the above were shortlisted as most relevant to the topic of VR in education. We also used the International Patent Classification (IPC) approach to determine a standard classification for registered patents, thereby enabling the search for and access to technical information accessible in documents connected to the same subject.

US20210011593A1 – System for rendering applications based on real-time accessibility assessment

Inventor(s): VAIDYANATH MADHAV [IN]; ANBALAGAN VINOD [IN]; KANNAIYAN THILAGARAJ [IN]; VASUDEVAN VARSHINI RAMIYA [IN]

Assignee: Bank of America [US]

Patent #: US20210011593A1

Classification: International: G06F16/28; G06F3/0484; G06F3/16; G06F9/451; G06N20/00; G10L13/04

Date: 14th Jan 2021

This patent proposes a system for producing applications based on real-time accessibility assessments. The system identifies that a user is accessing an application on a device (such as a virtual-reality device), captures the real-time accessibility data, inputs it into a machine learning model, generates an accessibility score, and renders the application based on this score. The advantage of the invention is that the system identifies whether the user is accessing another application on the same user device and utilizes the accessibility score stored in a data repository to render the other application; if the amount of data stored in the data repository increase over a period of time, this improves the efficiency with which they are processed. The method enables the calculation of a real-time accessibility score to comprise the operating configurations of the user device, as well as the status of its hardware components, to establish internet connectivity in an effective manner.

US20190227626A1 - Neuro-adaptive body sensing for user states framework

Inventor(s): MOHAMMADREZAZADEH IMAN [US]; BHATTACHARYYA RAJAN [US]

Assignee: HRL LAB LLC [US]

Patent #: US20190227626A1

Classification: International: A61B5/00; A61B5/0476; A61B5/16; A61N1/36; G06F3/01; G06N20/00

Date: 25th July 2019

This patent proposes a system for personalizing a human–machine interface (HMI) device based on the mental and physical state of a user. During the performance of a task in a simulation environment (such as training in virtual reality), the system extracts biometric features from data collected from body sensors and brain entropy features from electroencephalogram signals. Both data are correlated to generate a mental-state model. The mental-state model is deployed in a, HMI device during the performance of a task in an operational environment for the continuous adaptation of the HMI device to its user’s mental and physical states. The advantage of the invention would be that the continuous adaptation of the HMI to the mental and physical states of the user would reduce the workload and enhance decision-making. The need for unnecessary modifications in the interfaces would be eliminated, and the designs would be more user-centred and customized to the real needs of users.

US20210090343A1 - A method and a system for design reviews and trainings

Inventor(s): GODI MAHESH [US]; KOMMABHATLA SATWIK [IN]

Assignee: ACTIVA INNOVATIONS SOFTWARE PRIVATE LTD [US]

Patent #: US20210090343A1

Classification: International: G06F3/0481; G06N20/00; G06T17/00;G06T19/00

Date: 25th March 2021

The patent revolutionizes training and design reviews with a radically new experience using virtual/augmented reality technique. Using the present invention, multiple users may collaborate remotely and perform design reviews and trainings within the virtual/augmented reality environments.

Relevancy of papers

The domain of virtual reality for education may be entered by acquisition or rapid development. There are several companies, each with a small number of patents, indicating an opportunity to enter this area while it is still young, either by licensing existing technology, purchasing one of the players, or developing new technology that is not already patented. In terms of patent filings per year, the yearly patent filing of patents from 2016 to 2022, we saw that in the years 2016 and 2017 there were no patents registered for VR. The trend changed in 2018, followed by rapid growth from 2019 to 2022. This could have been accelerated due to COVID and more favoring towards digital education. This shows that 2018 marked a technological breakthrough in the domain of virtual reality. Before 2016, commercial virtual reality systems required users to connect a headset, controllers, and sensors to an external high-end computer, which was an expensive, bulky, and inconvenient setup. Thus, the current all-in-one virtual reality systems are a significant step forward and are changing the face of education using VR. This is the conclusion based on the fast-increasing research in the area of VR.

Scientific mapping:

Virtual Reality articles based on search results for the below questions: Q1: In terms of application fields, methods, hardware, and software involved, how is current knowledge on the application of virtual reality in usability testing and design review defined in the industry? Q2: What are the benefits and challenges of using virtual reality for usability testing and design review? The table below shows articles and papers selected by using a search strategy using Google Scholar.

Table 2 : List of other papers (cited and link updated in references) We have analyzed virtual reality across various industries and see that the most frequently researched field is the automotive industry and the most used device is HTC VIVE head-mounted display, which is frequently paired with motion capture systems and Unity 3D game engines. Based on the research, it was identified that there has been a significant technological breakthrough in VR only a few years ago, with the first public release of technologically mature virtual reality equipment. VR devices remain relatively expensive and technical constraints exist, but the price of virtual reality equipment decreased year on year, technological constraints have been reduced, and new features have been developed, resulting in the increased development of VR applications.

There has since been a significant growth in global research on virtual reality. In terms of inventions, we observed a scenario in which patent applications have boomed in a technology space that is rapidly evolving, offering opportunities to enter an area while it is still young. Thus, this exponential growth in patent applications for virtual reality, design review, and usability testing should continue for a few more years.

There has been an increasing number of companies are contemplating implementing virtual-reality-based design reviews and usability tests without fully comprehending the overall advantages and restrictions. This research has helped us understand more about how VR technology in education is evolving. It also helped us learn more about the advancements in VR headsets and the hardware associated because this has a direct impact on the learning experience.

Description of papers

Assessment of Augmented Reality Technology’s Impact on Speed of Learning and Task Performance in Aeronautical Engineering Technology Education

In this paper the study compared the learning and skill transfer among University students using AR vs traditional paper-based instructions. The study resulted in showing how AR users had significantly reduced task execution times. Pre-task “gamification and play” were also observed among the AR users, which could impact how educators and the industry assess and leverage learning strategies when using AR for job task training.

A Systematic Review of AR and VR Enhanced Language Learning

This paper provides a systematic review of previous Augmented Reality (AR) and Virtual Reality (VR) studies on language learning. The analysis was conducted according to the approaches of integrating AR and VR in language learning, the main users of AR and VR technologies, research findings, ways of AR and VR in improving language proficiency and research implications of previous research. In terms of research findings, most of the studies found AR and VR tools were effective in promoting learning, enhancing motivation and students held positive attitudes towards using those tools. Due to the effectiveness of AR and VR tools, there was an inevitable trend of integrating these tools in education.

Direct manipulation is better than passive viewing for learning anatomy in a three-dimensional virtual reality environment The goal of this paper was to investigate the impact of direct manipulation in virtual reality on anatomy learning. Building upon prior research showing mixed findings for the role of direct manipulation in spatially-intensive learning environments they showed 1. how the hand-held VR controller was designed to be highly intuitive and enable movements that were spatially congruent with the actions that would be taken to manipulate a physical model. 2. This study incorporated a realistic anatomical structure, with a specific learning task, rather than a more abstract, “anatomy-like” structure. The results of the study demonstrate that 1) participants are capable of successfully embodying virtual representations of internal anatomical structures if they can control the presentation; 2) participants who passively view the movement of this structure are most successful when presented with a limited number of canonical viewpoints; and 3) where the VR environment is designed to be intuitive and similar to the physical interaction in the real world, it is the participants with low spatial ability who tend to benefit most from the advantages of manipulation and interactivity, as compared with those with high spatial ability.

Challenges and strategies for educational virtual reality This qualitative study invited experts from a range of disciplines and sectors to identify common challenges in the visualization and analysis of 3D data, and the management of VR programs, for the purpose of developing a national library strategy.

In this paper major challenges identified are (1) interest and awareness for the emerging technology among faculty, students, and administrators (2) locating necessary expertise in VR within their communities when knowledge is unevenly distributed (3) getting enough buy-in from administrators to support the allocation of necessary resources (4) encouraging researchers to share their projects and research outputs for the benefit of the larger community.

Supporting teamwork in industrial virtual reality applications This study shows how VR systems can be used for visualization and interactions to support engineering design reviews. The work in this paper presents approaches to counteract this issue in a shared VR space for industry purpose. The paper wanted to evaluate the influence on the communication and cooperation among group members conducting design review in VR using the tools described in the paper. The results in this paper shows that they couldn’t reproduce the issues related to social exclusions in VR.

List of R&D projects

The R&D projects that are based on Educational output through VR Based education and VR Headset quality are listed below. Currently VR Headset based education is being prototyped though we do not have exact results on how the quality of VR Headset and the quality of the content impact the learning experience.

Table : List of R&D projects in VR Education

Demonstrator project

Virtual Reality for Training in Volvo Trucks Volvo Trucks has embarked on a pioneering venture into the realm of Virtual Reality (VR) for training, placing a distinct emphasis on the intricacies of electromobility and Liquified Natural Gas (LNG) (https://www.volvogroup.com/en/news-and-media/news/2022/sep/virtual-reality-training-with-oculus-headsets.html) This strategic move enables training sessions without the need for the physical presence of products, leading to significant cost savings, particularly in industries dealing with substantial and costly equipment like trucks. The adoption of Oculus 2 headsets facilitates seamless remote participation in trainer-led sessions, enhancing flexibility and inclusivity in the learning process. Moreover, the safety considerations intrinsic to VR training provide employees with a secure environment for mastering skills, especially in contexts where real-world training might pose potential risks. Currently, a pilot program is underway within Volvo Trucks in the UK, reflecting a commitment to active testing and refinement of VR training initiatives. The in-house production of VR environments by Volvo Group Digital & IT, in collaboration with preferred suppliers, ensures a bespoke approach, allowing for greater control over content creation. The focus extends beyond content, encompassing the secure management of VR devices and support for end-users utilizing Oculus Headsets. As interest in VR training techniques grows, Volvo is strategically planning an expanded implementation across various brands and employee categories, signaling a transformative shift in training methodologies. The positive outcomes observed in cost savings, accessibility, and safety considerations position VR as a cornerstone in Volvo Trucks' evolving training landscape.

Technology Strategy Statement

Our target is to develop a VR based Education model that has high display quality and high Educational output at lower cost. The system focuses on improving the quality of content, ease of creating content, user interaction, immersion and the overall learning experience. Our primary strategic driver is to develop interactive learning environment for students to understand concepts quicker. To achieve these targets, we will focus on Product Development costing $22M over the next 5 years by focusing on Software and content development for VR Headsets. In current scenarios it is not possible for the educators to create VR specific content easily. Our product will solve this gap. Our second strategic driver is to enable performing experiments in virtual space without direct physical contact with elements. We will invest in R&D projects like ZSpace and DreamFusion to create a virtual space that is easy to interact with and obtain hands-on experience. We plan to use existing headsets as it is not our intention to reinvent the wheel at a high cost. Instead, we will focus on making the interactive elements more user friendly with the educational aspect in focus. After 5 years of initial development, we will move towards marketing to capture the market. Till results in a negative cash flow for the first ten years. Beyond that we expect the project to ramp up and start generating profits, as a result of which we will move towards a positive cash flow situation.

Figure : Roadmap of VR based Education

References

[1] https://doi.org/10.1021/acsnano.2c12606

[2] https://www.uspto.gov/patents-application-process/search-patents

[3] https://www.mdpi.com/2076-3417/12/3/1755#B51-applsci-12-01755

[4] https://doi.org/10.1021/acsnano.2c12606

[5] https://www.tandfonline.com/doi/abs/10.1080/24721840.2021.1881403

[6] Cook, M. and Grime, J., 2021. Motivations, design, and preliminary testing for a 360 vision simulator. Virtual Reality, 25, pp.247-255.

[7] Cook, M., Lischer-Katz, Z., Hall, N., Hardesty, J., Johnson, J., McDonald, R. and Carlisle, T., 2019. Challenges and strategies for educational virtual reality. Information Technology and Libraries, 38(4), pp.25-48.

[8] https://www-statista-com.libproxy.mit.edu/chart/29398/vr-headset-kpis/

[9] https://techcrunch.com/2023/10/09/meta-quest-3-takes-a-step-closer-to-mainstream-ar-vr/#:~:text=Looking%20at%20some%20numbers%20%2C,to%20the%20PSVR%202

[10] https://www.idc.com/promo/arvr

[11] https://www.cnbc.com/2021/08/30/tiktok-owner-bytedance-acquires-pico-and-takes-first-step-into-virtual-reality.html

[12] https://www.wepc.com/vr/compare/meta-quest-3-vs-psvr-2/

[13] https://content-na1.emarketer.com/meta-captures-90-of-vr-headset-market-share).

[14] https://varjo.com/learning-hub/latency/#:~:text=In%20research%2C%20a%20meaningful%20latency,milliseconds%2C%20they%20are%20generally%20imperceptible

[15] Kristoffer B. Borgen, Timothy D. Ropp & William T. Weldon (2021) Assessment of Augmented Reality Technology’s Impact on Speed of Learning and Task Performance in Aeronautical Engineering Technology Education, The International Journal of Aerospace Psychology, 31:3, 219 229, DOI: 10.1080/24721840.2021.1881403

[16] Josef Wolfartsberger, Jan Zenisek, Norbert Wild (2020) Supporting Teamwork in Industrial Virtual Reality Applications, Procedia Manufacturing, 42, DOI https://doi.org/10.1016/j.promfg.2020.02.016

[17] Cook, M. ., Lischer-Katz, Z., Hall, N., Hardesty, J., Johnson, J., McDonald, R., & Carlisle, T. (2019). Challenges and Strategies for Educational Virtual Reality: Results of an Expert-led Forum on 3D/VR Technologies Across Academic Institutions. Information Technology and Libraries, 38(4), 25–48. https://doi.org/10.6017/ital.v38i4.11075

[18] Susan Jang, Jonathan M. Vitale, Robert W. Jyung, John B. Black (2017), Direct manipulation is better than passive viewing for learning anatomy in a three-dimensional virtual reality environment, Computers & Education, 106, DOI: https://doi.org/10.1016/j.compedu.2016.12.009

[19] Huang X, Zou D, Cheng G, Xie H. A Systematic Review of AR and VR Enhanced Language Learning. Sustainability. 2021; 13(9):4639. https://doi.org/10.3390/su13094639

[20] Safi, M., Chung, J. and Pradhan, P. (2019), "Review of augmented reality in aerospace industry", Aircraft Engineering and Aerospace Technology, Vol. 91 No. 9, pp. 1187-1194. https://doi.org/10.1108/AEAT-09-2018-0241

[21] J. Lee, K.S. Nam (2019) A STUDY OF DERIVING USABILITY EVALUATION FACTORS ON VIRTUAL REALITY CONTENTS, EDULEARN19 Proceedings, pp. 2272-2278.

[22] Sai Anirudh Karre, Neeraj Mathur, and Y. Raghu Reddy. 2019. Usability evaluation of VR products in industry: a systematic literature review. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing (SAC '19). Association for Computing Machinery, New York, NY, USA, 1845–1851. https://doi.org/10.1145/3297280.3297462

[23] Ingrid Pettersson, MariAnne Karlsson, and Florin Timotei Ghiurau. 2019. Virtually the Same Experience? Learning from User Experience Evaluation of In-vehicle Systems in VR and in the Field. In Proceedings of the 2019 on Designing Interactive Systems Conference (DIS '19). Association for Computing Machinery, New York, NY, USA, 463–473. https://doi.org/10.1145/3322276.3322288

[24] Aparajithan Sivanathan, James M Ritchie & Theodore Lim (2017) A novel design engineering review system with searchable content: knowledge engineering via real-time multimodal recording, Journal of Engineering Design, 28:10-12, 681 708, DOI: 10.1080/09544828.2017.1393655

[25] Berg, L. P., and Vance, J. M. (November 7, 2016). "An Industry Case Study: Investigating Early Design Decision Making in Virtual Reality." ASME. J. Comput. Inf. Sci. Eng. March 2017; 17(1): 011001. https://doi.org/10.1115/1.4034267

[26] de Clerk, M., Schmierer, G., Dangelmaier, M., Spath, D. (2017). Interaction Techniques for Virtual Reality Based Automotive Design Reviews. In: Barbic, J., D'Cruz, M., Latoschik, M., Slater, M., Bourdot, P. (eds) Virtual Reality and Augmented Reality. EuroVR 2017. Lecture Notes in Computer Science(), vol 10700. Springer, Cham. https://doi.org/10.1007/978-3-319-72323-5_3

[27] Kristoffer B. Borgen, Timothy D. Ropp & William T. Weldon (2021) Assessment of Augmented Reality Technology’s Impact on Speed of Learning and Task Performance in Aeronautical Engineering Technology Education, The International Journal of Aerospace Psychology, 31:3, 219-229, DOI: 10.1080/24721840.2021.1881403

[28] Josef Wolfartsberger, Jan Zenisek, Norbert Wild, Supporting Teamwork in Industrial Virtual Reality Applications, Procedia Manufacturing, Volume 42,2020, Pages 2-7, ISSN 2351-9789, https://doi.org/10.1016/j.promfg.2020.02.016.

[29] Cook, M. ., Lischer-Katz, Z., Hall, N., Hardesty, J., Johnson, J., McDonald, R., & Carlisle, T. (2019). Challenges and Strategies for Educational Virtual Reality: Results of an Expert-led Forum on 3D/VR Technologies Across Academic Institutions. Information Technology and Libraries, 38(4), 25–48. https://doi.org/10.6017/ital.v38i4.11075

[30] Marougkas, A.; Troussas, C.; Krouska, A.; Sgouropoulou, C. Virtual Reality in Education: A Review of Learning Theories, Approaches and Methodologies for the Last Decade. Electronics 2023, 12, 2832. https://doi.org/10.3390/electronics12132832

[31] Noble, S.M., Saville, J.D. & Foster, L.L. VR as a choice: what drives learners’ technology acceptance?. Int J Educ Technol High Educ 19, 6 (2022). https://doi.org/10.1186/s41239-021-00310-w

[32] Helsel, Sandra. “Virtual Reality and Education.” Educational Technology, vol. 32, no. 5, 1992, pp. 38–42. JSTOR, http://www.jstor.org/stable/44425644. Accessed 22 Oct. 2023.

[33] Cochrane, Thomas. "Mobile VR in Education: From the Fringe to the Mainstream." IJMBL vol.8, no.4 2016: pp.44-60. http://doi.org/10.4018/IJMBL.2016100104

[34] Y. Slavova and M. Mu, "A Comparative Study of the Learning Outcomes and Experience of VR in Education," 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Tuebingen/Reutlingen, Germany, 2018, pp. 685-686, doi: 10.1109/VR.2018.8446486.

[35] Rojas-Sánchez, M.A., Palos-Sánchez, P.R. & Folgado-Fernández, J.A. Systematic literature review and bibliometric analysis on virtual reality and education. Educ Inf Technol 28, 155–192 (2023). https://doi.org/10.1007/s10639-022-11167-5

[36] Kuna, P.; Hašková, A.; Borza, Ľ. Creation of Virtual Reality for Education Purposes. Sustainability 2023, 15, 7153. https://doi.org/10.3390/su15097153

[37] Burov, O.Y. and Pinchuk, O.P., 2023. A meta-analysis of the most influential factors of the virtual reality in education for the health and efficiency of students’ activity. Educational Technology Quarterly [Online], 2023(1), pp.58–68. Available from: https://doi.org/10.55056/etq.435 [Accessed 22 October 2023].