VR Charades Research Project

Structure

• app.apk - VR Charades application
• experiment-scripts/ - Control scripts and web interface
• data-analysis/ - Analysis notebook and face/hand tracking data
• videos/ - Recorded experiment footage
• vrcharades-unity/ - Unity source code
• VR_Charades_Paper.pdf - Research paper
• Einverstaendniserklaerungen.pdf - Consent forms

Running Experiments

Prerequisites

1. Create and activate a virtual environment:

# Windows
cd experiment-scripts
python -m venv venv
venv\Scripts\activate

# Linux/Mac
cd experiment-scripts
python3 -m venv venv
source venv/bin/activate

2. Install dependencies:

pip install -r requirements.txt

Experiment Scripts Overview

• app.py: Web interface for controlling the experiment and word list
• server.py: UDP relay server for communication between VR clients
• control.py: Command-line tool for setting VR client modes and server IP
• index.html: Web UI served by app.py
• word-list.txt: Default list of charade words

Setup Instructions

1. Network Setup

• Connect all VR headsets to the same network as the server
• Note the IP addresses of the VR headsets (you'll need these for configuration)
• Important: Update experiment-scripts/server.py lines 19-20 with your actual VR headset IPs

  # Replace these example IPs with your actual headset IPs
  DEVICE1_ADDR = ("YOUR_PLAYER1_IP", 5001)  # e.g., ("192.168.1.100", 5001)
  DEVICE2_ADDR = ("YOUR_PLAYER2_IP", 5001)  # e.g., ("192.168.1.101", 5001)
  

2. Start the Relay Server

cd experiment-scripts
python server.py

3. Start the Web Interface

cd experiment-scripts
python -m fastapi dev app.py

Then navigate to http://localhost:8000

4. Configure VR Clients

Tell clients which server to connect to:

# Windows PowerShell (replace with your actual VR headset IPs)
cd experiment-scripts
$env:TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" ; python control.py "IP:127.0.0.1"

# Linux/Mac (replace with your actual VR headset IPs)
cd experiment-scripts
TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" python3 control.py "IP:127.0.0.1"

5. Set Experiment Condition

Choose one of these modes:

# Dynamic Face only
$env:TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" ; python control.py "MODE:1;1;1;0"

# Dynamic Hands only  
$env:TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" ; python control.py "MODE:0;0;0;1"

# Dynamic Hands + Face
$env:TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" ; python control.py "MODE:1;1;1;1"

# Static Face only
$env:TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" ; python control.py "MODE:1;0;0;0"

# Static Hands only (requires controllers)
$env:TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" ; python control.py "MODE:0;0;0;1"

# Static Hands + Face (requires controllers for hands)
$env:TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" ; python control.py "MODE:1;0;0;1"

6. Prepare Word List

Shuffle the word list for randomization:

# Windows PowerShell
cd experiment-scripts
Get-Content word-list.txt | Sort-Object {Get-Random} | Out-File word-list-shuffled.txt

# Linux/Mac
cd experiment-scripts
shuf word-list.txt > word-list-shuffled.txt

Web Interface Usage

Once you have the web interface running at http://localhost:8000:

Setting Up a Charades Session

1. Configure Player IPs

   • Enter the IP addresses of Player 1 and Player 2 VR headsets
   • These should match the IPs you used in the control.py commands

2. Prepare Word List

   • Copy your shuffled word list from word-list-shuffled.txt
   • Paste it into the large text area on the right side
   • Click the "Modify" button to generate interactive word items

3. Set Game Parameters

   • Current Player For Acting: Select which player will be acting out the words
   • Time (s): Set the time limit for each word (e.g., 30 seconds)
   • Last Word Status: Set to "None" for the first word

Running the Experiment

Manual Mode (Individual Words)

1. Enter a word in the "Word" field
2. Select the acting player (Player 1 or Player 2)
3. Set the time limit
4. Click "Send" to transmit the word to the VR headsets

Automatic Mode (Word List)

1. After clicking "Modify" with your word list, the interface shows all words with timers
2. The system automatically starts with the first word and counts down
3. During the session:
   • Mark words correct: Hover over a word and check the checkbox if guessed correctly
   • Visual indicators: ▶ = Current word, ✅ = Correct, ❌ = Time expired
4. Stop the session: Click "Stop" to end early

Exporting Results

1. After completing the word list, click "Save as CSV"
2. This downloads a CSV file with word name, correctness, and time remaining

Mode Details

Mode format: <show_head>;<show_facial_expression>;<show_eye_rotation>;<show_hands>

• Dynamic modes: Real-time face/hand tracking
• Static modes: Participants use controllers instead of natural movement