178 lines
5.1 KiB
Markdown
178 lines
5.1 KiB
Markdown
# UDP Avatar Sync System
|
||
|
||
A simplified, efficient replacement for the HTTP/JSON avatar sync system using UDP broadcasting and compact binary data.
|
||
|
||
## Key Improvements
|
||
|
||
### Old System (HTTP/JSON)
|
||
- ❌ Required Python server (`avatar_sync_server.py`)
|
||
- ❌ Large JSON payloads (~1792 lines, complex nested objects)
|
||
- ❌ HTTP request/response overhead
|
||
- ❌ Complex serialization with multiple C# classes
|
||
- ❌ All 80+ bones transmitted every frame
|
||
- ❌ Inefficient for real-time sync
|
||
|
||
### New System (UDP Broadcasting)
|
||
- ✅ **No server required** - pure peer-to-peer broadcasting
|
||
- ✅ **Compact binary format** - ~670 bytes per packet (vs ~50KB+ JSON)
|
||
- ✅ **UDP broadcasting** - automatic discovery, no IP configuration
|
||
- ✅ **Priority-based data** - only essential bones and facial data
|
||
- ✅ **Fixed packet size** - predictable network usage
|
||
- ✅ **Real-time optimized** - designed for 30-60 Hz updates
|
||
|
||
## Components
|
||
|
||
### UDPAvatarBroadcaster
|
||
Captures and broadcasts your avatar data to the local network.
|
||
|
||
**Key Features:**
|
||
- Selects only priority bones (20 most important)
|
||
- Compact binary serialization
|
||
- Configurable update rates (1-120 Hz)
|
||
- Optional facial blend shape data
|
||
- Network performance monitoring
|
||
|
||
### UDPAvatarReceiver
|
||
Receives and applies avatar data from other players.
|
||
|
||
**Key Features:**
|
||
- Multi-threaded UDP listening
|
||
- Smooth interpolation options
|
||
- Packet loss detection
|
||
- Player ID filtering
|
||
- Thread-safe data handling
|
||
|
||
## Quick Setup
|
||
|
||
### For Player 1 (Broadcaster)
|
||
1. Add `UDPAvatarBroadcaster` component to your avatar
|
||
2. Set `Player ID` to `1`
|
||
3. Assign your avatar root transform
|
||
4. Enable broadcasting
|
||
|
||
### For Player 2 (Receiver)
|
||
1. Add `UDPAvatarReceiver` component to target avatar
|
||
2. Set `Target Player ID` to `1` (to receive from Player 1)
|
||
3. Assign target avatar root transform
|
||
4. Enable receiver
|
||
|
||
### For Bidirectional Sync
|
||
- Each player needs both components
|
||
- Use different Player IDs (1, 2, 3, etc.)
|
||
- Set Target Player ID to receive from specific players
|
||
|
||
## Configuration
|
||
|
||
### Network Settings
|
||
- **Port**: 8080 (default, ensure firewall allows UDP)
|
||
- **Broadcast Address**: 255.255.255.255 (local network)
|
||
- **Update Rate**: 30 Hz recommended (balance of smoothness vs bandwidth)
|
||
|
||
### Data Selection
|
||
- **Priority Bones**: 20 essential bones (customizable array)
|
||
- Core: Hips, Spine, Neck, Head
|
||
- Arms: Shoulders, Arms, Forearms, Hands
|
||
- Legs: Upper legs, Legs, Feet
|
||
- **Facial Data**: 10 most important blend shapes
|
||
- **Root Transform**: Position, rotation, scale options
|
||
|
||
### Performance Options
|
||
- **Smooth Transitions**: Interpolate between updates
|
||
- **Transition Speed**: How fast to blend changes
|
||
- **Debug Info**: Network stats and diagnostics
|
||
|
||
## Data Structure
|
||
|
||
The system uses a fixed binary format:
|
||
|
||
```
|
||
Header (9 bytes):
|
||
- Player ID (1 byte)
|
||
- Sequence Number (4 bytes)
|
||
- Timestamp (4 bytes)
|
||
|
||
Root Transform (40 bytes):
|
||
- Position (12 bytes: 3 floats)
|
||
- Rotation (16 bytes: 4 floats)
|
||
- Scale (12 bytes: 3 floats)
|
||
|
||
Priority Bones (560 bytes):
|
||
- 20 bones × 28 bytes each
|
||
- Position (12 bytes) + Rotation (16 bytes) per bone
|
||
|
||
Facial Data (40 bytes):
|
||
- 10 blend shape weights (4 bytes each)
|
||
|
||
Total: ~649 bytes per packet
|
||
```
|
||
|
||
## Network Requirements
|
||
|
||
- **Local Network**: All devices must be on same subnet
|
||
- **Firewall**: Allow UDP port 8080 (or configured port)
|
||
- **Bandwidth**: ~20KB/s per player at 30 Hz (very lightweight)
|
||
|
||
## Troubleshooting
|
||
|
||
### No Data Received
|
||
1. Check firewall settings (allow UDP on chosen port)
|
||
2. Verify devices are on same network subnet
|
||
3. Ensure different Player IDs for broadcaster/receiver
|
||
4. Check Debug Info for network statistics
|
||
|
||
### Performance Issues
|
||
1. Reduce update rate (try 15-20 Hz)
|
||
2. Disable smooth transitions for lower latency
|
||
3. Reduce max blend shapes count
|
||
4. Check packet loss statistics
|
||
|
||
### Bone Mapping Issues
|
||
1. Verify bone names match priority bone list
|
||
2. Check Debug Info to see cached bone count
|
||
3. Customize priority bones array for your rig
|
||
|
||
## Advanced Usage
|
||
|
||
### Custom Bone Sets
|
||
Modify the `priorityBones` array to match your avatar rig:
|
||
```csharp
|
||
private string[] priorityBones = {
|
||
"Root", "Spine1", "Spine2", "Head",
|
||
"LeftArm", "RightArm",
|
||
// Add your specific bone names
|
||
};
|
||
```
|
||
|
||
### Multiple Players
|
||
- Each broadcaster needs unique Player ID (1-255)
|
||
- Receivers can filter by Target Player ID
|
||
- Set Target Player ID to 0 to receive from any player
|
||
|
||
### Network Optimization
|
||
- Increase update rate for smoother motion (higher bandwidth)
|
||
- Decrease update rate for lower bandwidth usage
|
||
- Adjust transition speed for different responsiveness
|
||
|
||
## Migration from Old System
|
||
|
||
1. **Remove old components**:
|
||
- AvatarSyncClient
|
||
- AvatarDataUploader
|
||
- AvatarDataDownloader
|
||
- AvatarDataWriter
|
||
- AvatarDataReader
|
||
|
||
2. **Remove server dependency**:
|
||
- Stop `avatar_sync_server.py`
|
||
- No longer need HTTP endpoints
|
||
|
||
3. **Add new components**:
|
||
- UDPAvatarBroadcaster (for sending)
|
||
- UDPAvatarReceiver (for receiving)
|
||
|
||
4. **Configure network**:
|
||
- Ensure UDP port is open
|
||
- Set unique Player IDs
|
||
- Test on local network first
|
||
|
||
The new system is much simpler to set up and requires no server infrastructure! |