Master-Arbeit-Tom-Hempel/Unity-Master/Assets/Scripts/README_UDP_AvatarSync.md

UDP Avatar Sync System

A simplified, efficient replacement for the HTTP/JSON avatar sync system using UDP broadcasting and compact binary data.

Key Improvements

Old System (HTTP/JSON)

• ❌ Required Python server (avatar_sync_server.py)
• ❌ Large JSON payloads (~1792 lines, complex nested objects)
• ❌ HTTP request/response overhead
• ❌ Complex serialization with multiple C# classes
• ❌ All 80+ bones transmitted every frame
• ❌ Inefficient for real-time sync

New System (UDP Broadcasting)

• ✅ No server required - pure peer-to-peer broadcasting
• ✅ Compact binary format - ~670 bytes per packet (vs ~50KB+ JSON)
• ✅ UDP broadcasting - automatic discovery, no IP configuration
• ✅ Priority-based data - only essential bones and facial data
• ✅ Fixed packet size - predictable network usage
• ✅ Real-time optimized - designed for 30-60 Hz updates

Components

UDPAvatarBroadcaster

Captures and broadcasts your avatar data to the local network.

Key Features:

• Selects only priority bones (20 most important)
• Compact binary serialization
• Configurable update rates (1-120 Hz)
• Optional facial blend shape data
• Network performance monitoring

UDPAvatarReceiver

Receives and applies avatar data from other players.

Key Features:

• Multi-threaded UDP listening
• Smooth interpolation options
• Packet loss detection
• Player ID filtering
• Thread-safe data handling

Quick Setup

For Player 1 (Broadcaster)

1. Add UDPAvatarBroadcaster component to your avatar
2. Set Player ID to 1
3. Assign your avatar root transform
4. Enable broadcasting

For Player 2 (Receiver)

1. Add UDPAvatarReceiver component to target avatar
2. Set Target Player ID to 1 (to receive from Player 1)
3. Assign target avatar root transform
4. Enable receiver

For Bidirectional Sync

• Each player needs both components
• Use different Player IDs (1, 2, 3, etc.)
• Set Target Player ID to receive from specific players

Configuration

Network Settings

• Port: 8080 (default, ensure firewall allows UDP)
• Broadcast Address: 255.255.255.255 (local network)
• Update Rate: 30 Hz recommended (balance of smoothness vs bandwidth)

Data Selection

• Priority Bones: 20 essential bones (customizable array)
  • Core: Hips, Spine, Neck, Head
  • Arms: Shoulders, Arms, Forearms, Hands
  • Legs: Upper legs, Legs, Feet
• Facial Data: 10 most important blend shapes
• Root Transform: Position, rotation, scale options

Performance Options

• Smooth Transitions: Interpolate between updates
• Transition Speed: How fast to blend changes
• Debug Info: Network stats and diagnostics

Data Structure

The system uses a fixed binary format:

Header (9 bytes):
- Player ID (1 byte)
- Sequence Number (4 bytes) 
- Timestamp (4 bytes)

Root Transform (40 bytes):
- Position (12 bytes: 3 floats)
- Rotation (16 bytes: 4 floats) 
- Scale (12 bytes: 3 floats)

Priority Bones (560 bytes):
- 20 bones × 28 bytes each
- Position (12 bytes) + Rotation (16 bytes) per bone

Facial Data (40 bytes):
- 10 blend shape weights (4 bytes each)

Total: ~649 bytes per packet

Network Requirements

• Local Network: All devices must be on same subnet
• Firewall: Allow UDP port 8080 (or configured port)
• Bandwidth: ~20KB/s per player at 30 Hz (very lightweight)

Troubleshooting

No Data Received

1. Check firewall settings (allow UDP on chosen port)
2. Verify devices are on same network subnet
3. Ensure different Player IDs for broadcaster/receiver
4. Check Debug Info for network statistics

Performance Issues

1. Reduce update rate (try 15-20 Hz)
2. Disable smooth transitions for lower latency
3. Reduce max blend shapes count
4. Check packet loss statistics

Bone Mapping Issues

1. Verify bone names match priority bone list
2. Check Debug Info to see cached bone count
3. Customize priority bones array for your rig

Advanced Usage

Custom Bone Sets

Modify the priorityBones array to match your avatar rig:

private string[] priorityBones = {
    "Root", "Spine1", "Spine2", "Head",
    "LeftArm", "RightArm", 
    // Add your specific bone names
};

Multiple Players

• Each broadcaster needs unique Player ID (1-255)
• Receivers can filter by Target Player ID
• Set Target Player ID to 0 to receive from any player

Network Optimization

• Increase update rate for smoother motion (higher bandwidth)
• Decrease update rate for lower bandwidth usage
• Adjust transition speed for different responsiveness

Migration from Old System

1. Remove old components:

   • AvatarSyncClient
   • AvatarDataUploader
   • AvatarDataDownloader
   • AvatarDataWriter
   • AvatarDataReader

2. Remove server dependency:

   • Stop avatar_sync_server.py
   • No longer need HTTP endpoints

3. Add new components:

   • UDPAvatarBroadcaster (for sending)
   • UDPAvatarReceiver (for receiving)

4. Configure network:

   • Ensure UDP port is open
   • Set unique Player IDs
   • Test on local network first

The new system is much simpler to set up and requires no server infrastructure!