Teaching robots through Games with a Purpose
The objective of RobCoG is to equip robots with commonsense and naive physics knowledge using games with a purpose. In the games users are asked to execute various tasks in different scenarios. The games are equipped with a semantic logging system which captures and stores symbolic and sub-symbolic data during the gameplay.
VR Kitchen
Game prototype created in Unreal Engine where users are asked to execute various kitchen related tasks. During gameplay symbolic and sub-symbolic data is automatically collected. The data is then stored in the web-based knowledge service openEASE.
Game features
Various user interactions: full body motion capture systems, VR or standard game controllers. The virtual environment is modeled having photo- and physical realism in mind. Using destructible and modifiable meshes for simulating various effects such as cracking an egg or cutting objects. Using particle based physics (Nvidia FleX) to be able to simulate soft objects or liquids. Robotic simulations by generating robots from URDF.
EEG Extension
The game environment can be extended to collect EEG data during task execution. The EEG channels are continously monitored and logged as sub-symbolic data to the knowledge base. Using openEASE users can couple the collected data with the various events that happened during execution.
