diff --git a/README.md b/README.md index 96e18d8..cfbc31e 100644 --- a/README.md +++ b/README.md @@ -68,124 +68,133 @@ pip install -r requirements.txt ### Experiment Scripts Overview -- **app.py**: Web interface for controlling the experiment and word list -- **server.py**: UDP relay server for communication between VR clients -- **control.py**: Command-line tool for setting VR client modes and server IP -- **index.html**: Web UI served by app.py -- **word-list.txt**: Default list of charade words +- **app.py**: Main server - web interface, experiment control, and automatic tracking recorder +- **server.py**: UDP relay for communication between VR clients (auto-started by app.py) +- **index.html**: Web UI for configuring and running experiments +- **static/**: Frontend assets (CSS, JavaScript, player display) +- **data/**: Word lists (English and German) ### Setup Instructions #### 1. Network Setup -- Connect all VR headsets to the same network as the server -- Note the IP addresses of the VR headsets (you'll need these for configuration) -- **Important**: Update `experiment-scripts/server.py` lines 19-20 with your actual VR headset IPs - ```python - # Replace these example IPs with your actual headset IPs - DEVICE1_ADDR = ("YOUR_PLAYER1_IP", 5001) # e.g., ("192.168.1.100", 5001) - DEVICE2_ADDR = ("YOUR_PLAYER2_IP", 5001) # e.g., ("192.168.1.101", 5001) - ``` +- Connect all VR headsets to the same network as your computer +- Note the IP addresses of both VR headsets +- Note your computer's IP address (the server IP) -#### 2. Start the Relay Server +To find your server IP: ```bash -cd experiment-scripts -python server.py +# Linux/Mac +hostname -I + +# Windows +ipconfig ``` -#### 3. Start the Web Interface +**Note**: The UDP relay automatically detects and forwards data between connected VR headsets - no manual IP configuration needed in server.py! + +#### 2. Start the Server + ```bash cd experiment-scripts -python -m fastapi dev app.py -``` -Then navigate to `http://localhost:8000` - -#### 4. Configure VR Clients - -Tell clients which server to connect to: -```bash -# Windows PowerShell (replace with your actual VR headset IPs) -cd experiment-scripts -$env:TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" ; python control.py "IP:127.0.0.1" - -# Linux/Mac (replace with your actual VR headset IPs) -cd experiment-scripts -TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" python3 control.py "IP:127.0.0.1" +fastapi dev app.py ``` -#### 5. Set Experiment Condition +This single command automatically starts: +- Web interface on http://localhost:8000 +- UDP relay server (server.py) +- Tracking data recorder -Choose one of these modes: -```bash -# Dynamic Face only -$env:TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" ; python control.py "MODE:1;1;1;0" - -# Dynamic Hands only -$env:TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" ; python control.py "MODE:0;0;0;1" - -# Dynamic Hands + Face -$env:TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" ; python control.py "MODE:1;1;1;1" - -# Static Face only -$env:TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" ; python control.py "MODE:1;0;0;0" - -# Static Hands only (requires controllers) -$env:TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" ; python control.py "MODE:0;0;0;1" - -# Static Hands + Face (requires controllers for hands) -$env:TARGET_IP="YOUR_PLAYER1_IP,YOUR_PLAYER2_IP" ; python control.py "MODE:1;0;0;1" -``` - -#### 6. Prepare Word List - -You can shuffle the word list directly in the web interface (see Web Interface Usage section below). +Navigate to `http://localhost:8000` to access the control interface. ### Web Interface Usage -Once you have the web interface running at `http://localhost:8000`: +All experiment configuration and control is done through the web interface at `http://localhost:8000`. -#### Setting Up a Charades Session +#### 1. Configure VR Headsets -1. **Configure Player IPs** - - Enter the IP addresses of Player 1 and Player 2 VR headsets - - These should match the IPs you used in the `control.py` commands +In the **VR Headset Configuration** section: +- Enter **Server IP Address**: Your computer's IP address +- Enter **Player 1 IP Address**: First VR headset IP +- Enter **Player 2 IP Address**: Second VR headset IP +- Select **Experiment Mode** (see Experiment Conditions below) +- Click **"Send Configuration to Headsets"** -2. **Prepare Word List** - - Copy your word list from `word-list.txt` and paste it into the large text area on the right side - - Click the **"Shuffle"** button to randomize the word order - - Click the **"Modify"** button to generate interactive word items +Wait for confirmation that the configuration was sent successfully. -3. **Set Game Parameters** - - **Current Player For Acting**: Select which player will be acting out the words - - **Time (s)**: Set the time limit for each word (e.g., 30 seconds) - - **Last Word Status**: Set to "None" for the first word +#### 2. Configure Experiment Session -#### Running the Experiment +In the **Session Configuration** section: +- **Group ID**: Identifier for this session (used in CSV filenames) +- **Time per Word**: Duration in seconds for each word (e.g., 30) +- **Total Duration**: Total experiment time in minutes (0 = unlimited) -**Manual Mode (Individual Words)** -1. Enter a word in the "Word" field -2. Select the acting player (Player 1 or Player 2) -3. Set the time limit -4. Click **"Send"** to transmit the word to the VR headsets +In the **Network Configuration** section: +- **Active Player**: Select which player will be performing (Player 1 or Player 2) -**Automatic Mode (Word List)** -1. After clicking "Modify" with your word list, the interface shows all words with timers -2. The system automatically starts with the first word and counts down -3. **During the session**: - - **Mark words correct**: Hover over a word and check the checkbox if guessed correctly - - **Visual indicators**: ▶ = Current word, ✅ = Correct, ❌ = Time expired -4. **Stop the session**: Click **"Stop"** to end early +#### 3. Prepare Word List -#### Exporting Results +In the **Word List** section: +- Copy words from `data/word-list.txt` or enter your own (one word per line) +- Click **"Shuffle Words"** to randomize order +- Click **"Start Experiment"** when ready -1. After completing the word list, click **"Save as CSV"** -2. This downloads a CSV file with word name, correctness, and time remaining +#### 4. Run the Experiment -### Mode Details +When you click **"Start Experiment"**: +- The system automatically sends words to the active player +- Tracking data recording starts automatically +- Words advance based on the timer +- Check the checkbox next to each word if guessed correctly +- Click **"Stop Experiment"** to end early -Mode format: `;;;` +#### 5. Export Data -- **Dynamic modes**: Real-time face/hand tracking -- **Static modes**: Participants use controllers instead of natural movement +After the experiment: +- **"Save Results (CSV)"**: Downloads word results + - Format: `{group_id}_results_{timestamp}.csv` + - Contains: word, correct/incorrect, time remaining + +- **"Download Tracking Data (CSV)"**: Downloads tracking data + - Format: `{group_id}_tracking_{timestamp}.csv` + - Contains: camera and controller positions/rotations at 60Hz + - Includes: timestamps, current word, condition, elapsed time + +### Experiment Conditions + +The experiment supports six different conditions that control which body parts are tracked and how: + +| Condition | Description | Settings | +|-----------|-------------|----------| +| **Dynamic Face** | Real-time face tracking with expressions and eye rotation | 1;1;1;0 | +| **Dynamic Hands** | Real-time hand tracking with finger gestures | 0;0;0;1 | +| **Dynamic Hands+Face** | Full tracking: face, expressions, eyes, and hands | 1;1;1;1 | +| **Static Face** | Head position tracking only (no expressions) | 1;0;0;0 | +| **Static Hands** | Controller tracking (no finger tracking) | 0;0;0;1 | +| **Static Hands+Face** | Head position + controller tracking | 1;0;0;1 | + +**Mode format**: `;;;` + +**Notes**: +- **Dynamic modes**: Use natural face/hand tracking via Quest Pro sensors +- **Static modes**: Participants must use controllers for hand input +- Select the condition in the web interface before starting the experiment + +### Tracking Data + +The system automatically records tracking data from the **active player** (the one performing charades) at approximately 60Hz: + +**Recorded data**: +- Center eye camera position (x, y, z) and rotation (w, x, y, z) +- Left hand controller position and rotation +- Right hand controller position and rotation +- Current word being performed +- Timestamps and elapsed time + +**Data recording**: +- Starts automatically when experiment starts +- Stops automatically when experiment stops +- Only records from the active player (not the guesser) +- Exports as CSV with group ID and timestamp in filename ## Unity Project Details diff --git a/experiment-scripts/server.py b/experiment-scripts/server.py index 7303115..b62a189 100644 --- a/experiment-scripts/server.py +++ b/experiment-scripts/server.py @@ -1,23 +1,9 @@ import socket -import threading from datetime import datetime -from time import sleep -CONTROL_ADDR = ("127.0.0.1", 5001) - -# TODO: Adjust the following addresses so they match the IP addresses of the -# VR headsets. -# In our case the IP addresses were: -# - for player 1: 10.42.0.38 -# - for player 2: 10.42.0.72 -# -# The ports are hardcoded to 5001 inside the Unity application, so you -# shouldn't change those. -# -# Note: For this to work the VR headsets must be connected to the same network -# as this server. -DEVICE1_ADDR = ("10.42.0.38", 5001) -DEVICE2_ADDR = ("10.42.0.72", 5001) +# Track connected devices dynamically +# No manual IP configuration needed! +connected_devices = {} # {ip: last_seen_timestamp} sock_from_A = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) sock_from_A.bind(("0.0.0.0", 5000)) @@ -26,30 +12,36 @@ def forward(source_socket): while True: try: data, addr = source_socket.recvfrom(1024 * 16) - target_ip = DEVICE1_ADDR[0] if addr[0] == DEVICE2_ADDR[0] else DEVICE2_ADDR[0] - label = "A→B" if addr == DEVICE1_ADDR else "B→A" - if addr != DEVICE1_ADDR and addr != DEVICE2_ADDR: - label = f"unknown {addr}" + source_ip = addr[0] + timestamp = datetime.now() - # Forward to other player - sock_from_A.sendto(data, (target_ip, 5000)) + # Register this device + connected_devices[source_ip] = timestamp + + # Get list of other active devices (excluding localhost and source) + other_devices = [ip for ip in connected_devices.keys() + if ip != source_ip and not ip.startswith("127.")] + + # Forward to all other connected devices + for target_ip in other_devices: + sock_from_A.sendto(data, (target_ip, 5000)) # Also forward to app.py tracking listener on port 5002 # Prepend source IP so app.py can identify which player sent the data - source_ip = addr[0] tagged_data = f"SOURCE_IP:{source_ip}|".encode('utf-8') + data sock_from_A.sendto(tagged_data, ("127.0.0.1", 5002)) - timestamp = datetime.now().strftime("%H:%M:%S") - # Logging - #if next(counter) % 20 == 0: - # if addr[0] != DEVICE2_ADDR[0]: - # print(f"[{timestamp}] {label}: {data.decode()}") - # print('sent to ', (target_ip, 5000)) + if len(other_devices) > 0: + label = f"{source_ip} → {', '.join(other_devices)}" + else: + label = f"{source_ip} (no other devices connected)" + + # Uncomment for verbose logging: + # print(f"[{timestamp.strftime('%H:%M:%S')}] {label}") except Exception as e: - print(f"Fehler {label}: {e}") + print(f"Error in relay: {e}") print("UDP Relay läuft. Strg+C zum Beenden.") try: