VR Training & Simulation

Authoring

Umajin provides tools to make creating scenarios and deploying them with users fast and efficient

Overview

The VR Scenario builder makes it easy to import 3D assets, animations, images and audio content to create sophisticated interactions.

It’s a powerful environment that allows for rapid development and iteration. You can make changes to your scenario over the network and instantly deploy these changes to users who are viewing in VR.

Umajin is ideal for building real world scenarios and process oriented training. Scenarios can be configured to automatically react to users when they move into an target area, pick up an object, or verbally say a phrase. However its also an option for a trainer to manually trigger behaviors in real time providing very nuanced interactions.

Content import and scene editing

Content importing into the scene

  • Import most major mesh formats
  • Easy to scale, rotate and move models while in VR
  • Texture and light models with Physically Based Rendering (PBR)
  • Add actions to objects in the scene to provide interactivity
  • Add images, text and audio into scenes

Skeletal animation

  • Umajin supports the FBX animation formats and provides a complete model and rigging pipeline using Real Illusion
  • Inverse Kinematics allows avatars and virtual actors to look at or hold onto other objects
  • Simple setup for attachments so characters can hold or wear props

Voice

  • Easy to record new audio and place it in the scene
  • Simple system for voice playback and facial animation
  • Background audio layer for ambient sounds
  • Voice commands allow users to interact with voice

Rendering

The Physically Based Rendering system for materials allows for high quality visuals.   The image based lighting, reflective metallic materials and shadow maps help provide a more realistic and immersive VR experience.

High precision motion tracking

Multiple users and multiple props can all be tracked with high accuracy at high frequency.

This co-spacing capability allows real objects to be used in Virtual Reality scenarios as they are perfectly aligned in the real and virtual worlds.

Body and posture tracking is also used for real time feedback of users for their bio-mechanics.

Flexible triggers and actions

The flexible triggers and actions system supports

  • Multi choice question and answer
  • Step by step processes (do steps in the right order)
  • Timed triggers (the right thing at the right time – like the drumming example)
  • Hand, Head, Glance and Pose detection (where and how the user is positioned)
  • Prop position, velocity, orientation and triggers (props put in the right physical place can trigger actions)
  • Branching actions (different choices will change the path through the scenario)

Drum video example (link)

Fusion Reactor Example

The fusion reactor scene is a step by step assembly process.  While the reactor is fictional the scenario requires two simultaneous users and real props.  The VR view has prompts instructing the users of their next step. 

The props are tracked for speed (don’t move the reactor core too quickly), for position (put the elements in the right place) and orientation (the fuel cells must be placed the correct way around). 

fusion reactor video (link)

These are the seven on screen HUD instructions for the fusion reactor.  This shows the users the steps of charging, preparing and placing the two fuel cells followed by the fusion core.

Sound effects and animations on the virtual props let the user know they have met the requirements – or warn them when they trigger an error state (like moving too fast with the charged cells before containment)

Police Stop Example

The police stop example is a procedure where a cadet must complete a series of verbal and physical tasks.  They can do this with prompts when learning and without prompts when being assessed.  The simulation will repeat with subtle changes, like different vehicles being stopped and different characters with their own unique dialog.

  • Step by step processes (go through a traffic stop procedure)
  • Voice commands (talk directly to the characters in the scene)
  • Hand triggers (take the wallet out of the characters hand)
  • Look trigger (ensure you check the number plate)
  • Head position triggers (don’t walk past the drivers side pillar or step into the traffic)

Scenario Objects

  • Audio: Play an audio file.  This includes wav, ogg and mp3 audio
  • Image: Display an image in the scene
  • Model: Display a 3D model in the scene.  The model can be animated and its position, rotation and scale properties can be animated but it will not act like a person/character
  • Model Character: Display a 3D character model in the scene.  The model can be animated and knows how to animate its mouth, look at things and hold things in it’s hands.  The model can also be connected to live motion capture data to act as an avatar for a user
  • Module: Load in a separate file so complex scenes can be split across files, or so specific parts can be reused in multiple scenarios
  • Rigid: A 3D model or other virtual object can be scaled and tracked to match a real world object (prop). The prop is represented by a PhaseSpace rigid_id.  This is great for actual tools and equipment that can be tracked and used in the VR scenario
  • State: This helps you to manage the state you are up to in a process or simulation. It has no visual representation but can be setup to respond  to on_activate or on_deactivate
  • Text: Show a block of text. The width must be set so the text can be rendered and wrapped correctly.  This is great for labels, explanatory text for the next step or to connect to enterprise data for digital twin’s
  • Trigger_speech: Create a trigger that detects one of the phrases being spoken.  This allows voice commands to be added to your simulation
  • Trigger_velocity: Tracks an object’s movement, and fires on_over_threshold when its velocity goes over threshold, and on_under_threshold when it goes under
  • Trigger_volume_hand: Create a trigger sphere that raises an event when your hand enters or leaves the sphere. You can make it only trigger with a specific hand by using trigger_volume_hand_left, trigger_volume_hand_right, or either hand by using trigger_volume_hand
  • Trigger_volume_head: Create a trigger cylinder that raises an event when your head enters or leaves the cylinder.  For example when a user stands in a specific place such as a safety zone
  • Trigger_volume_object: Create a trigger sphere that raises an event when a specified object enters or leaves the sphere.  For example when a tool is placed near the correct bolt in a machine being constructed.

Scenario Actions

  • activate: Activate a scene object
  • attach:Attach a scene object to another scene object (such as holding a tool in a characters hand)
  • cancel: Cancel a delayed action, including a tween action
  • deactivate:Deactivate a scene object
  • deactivate_all_triggers: Deactivate all trigger objects in the scene
  • detach: Detach a scene object from whatever it is currently attached to (such as releasing a tool being held)
  • hide_popup: Hide the floating popup
  • play_animation: Play an animation of a model
  • set: Set a property of a scene object
  • set_all: Set a property for all scene objects of a given type
  • show_popup: Show a floating popup
  • tween: Animate over time one or more properties of a scene object
  • vibrate: Vibrate a controller if a VR controller or vibration motor is available
Scroll to Top