Monitoring and Analyzing VR Applications Using the Generic Events Plugin

The Generic Events Plugin has been used for monitoring VR applications. For this, the VR applications have been equipped with a monitoring component, that sends monitored events to the Generic Event Monitor as described here. An example for a monitoring component are the scripts attached to this page. These scripts can be integrated into ​Unity 3D projects. Unity 3D is a game engine that, in addition to others, can be used to create VR applications. One of the scripts must be added to a game object in a VR scene. On initialization, the script starts to monitor head movements of users as well as grabbing and using objects. For this, they register with event handling mechanisms attached to other game objects in the scene to be notified about any interaction. One of the scripts, the one with VRTK in its name, is an extension to the other. It works with the ​Virtual Reality Toolkit (VRTK) and extends the functionality of the other script with framework specific monitoring capabilities.

Both scripts need to be configured with a valid endpoint of a Generic Event Monitor instance. When correctly setup, the send JSON message as required to the monitor instance. The monitor in turn stores the data locally.

Using AutoQUEST and the Generic Events Plugin, the data can be read and processed. For this, the command parseDirGenericEvents must be called first to load the data. Then other commands like generateTaskTrees and evaluateUsability can be used for further analysis. An example script for use with AutoQUEST is the following:

% read the recorded data. Thereby exclude events with type "eventToExclude".
% In addition, exclude events of type "eventWithTargetToExclude" where the target id is "targetToExclude"

parseDirGenericEvents <directory containing data> <name of sequences within AutoQUEST> eventToExclude eventWithTargetToExclude.targetToExclude

% generate the task trees for the recorded data
generateTaskTree <name of sequences within AutoQUEST> <name of task trees within AutoQUEST>

% perform the usability smell detection for the task trees
evaluateUsability <name of task trees within AutoQUEST> <name of evaluation results within AutoQUEST>