Virtual Tutor

Disclaimer: This project is still a work in progress and in early development.

Project Structure

Unity VR/AR Application (`Unity/`)

Scenes

Assets/Scenes/VR/ - VR learning tasks (1-4, task 1 currently usable)
Assets/Scenes/AR/ - AR learning tasks (1-4, task 1 currently usable)
Lobby.unity - Main lobby/menu scene

Key Scripts

TaskTimer.cs - Configurable countdown timer with UI display, color warnings when time is low, and auto-return to Lobby on expiration
TaskButtonManager.cs - Scene navigation using number keys (1-8 for tasks, L/0 for Lobby) and UDP remote control (default port 5555)
PositionTracker.cs - Records Transform positions to CSV at configurable intervals for data collection

Technologies

Convai SDK for AI-powered conversations
Ready Player Me for avatars
Meta Quest (Oculus) XR support

Setup Instructions

Enable Quest Link
- Open the Oculus app on your PC
- Connect your Quest headset via USB-C cable
- Put on your headset and enable Quest Link when prompted
- Alternatively, use Air Link for wireless connection (Settings → Experimental)
Enable Beta Passthrough (for AR tasks)
- On your PC, open the Oculus app
- Go to Settings → Beta
- Enable "Passthrough over Oculus Link" or "Passthrough API"
- Restart the Oculus app and reconnect your headset
Run the Unity Project
- Open the project in Unity Editor
- Load the Lobby scene from Assets/Scenes/
- Press Play in Unity Editor
- The application will run on your Quest via Link

Number Keys 1-8 - Switch between VR/AR tasks
L or 0 - Return to Lobby
VR controllers for interaction within tasks

UDP Remote Control

The Unity application listens for UDP commands on port 5555 (configurable in Unity Inspector) for remote scene switching.

Command Format:

SWITCH:SceneName

Example Commands:

# Switch to VR Task 1
echo "SWITCH:VR-Task1" | nc -u localhost 5555

# Switch to AR Task 2
echo "SWITCH:AR-Task2" | nc -u localhost 5555

# Return to Lobby
echo "SWITCH:Lobby" | nc -u localhost 5555

Python Example:

import socket

def switch_scene(scene_name, host='localhost', port=5555):
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    message = f"SWITCH:{scene_name}"
    sock.sendto(message.encode('utf-8'), (host, port))
    sock.close()

# Usage
switch_scene("VR-Task1")

Available Scene Names:

VR-Task1, VR -Task2, VR -Task3, VR -Task4
AR-Task1, AR-Task2, AR-Task3, AR-Task4
Lobby

Chat Application (`Chat-App/`)

Flutter-based cross-platform chat interface.

Note: To get API key and Character ID, contact me at tom.hempel@stud.hs-coburg.de or if no longer a student at contact@tomhempel.com

Features

Cross-platform support (Android, iOS, Windows, Linux, macOS, Web)
AI-powered conversations using Convai SDK
Conversation history with persistent storage
Automatic translation support via DeepL API

Installation

Pre-built installers available in App-Installers/:

Android - AI-Chat-Lab.apk
Windows - AIChatLab-Setup.exe
Linux - ai_chat_lab-0.1.0-x86_64.AppImage

Development

To develop or run the application from source, you'll need to set up your Flutter development environment.

Install Flutter SDK
- Follow the official Flutter installation guide for your operating system: https://flutter.dev/docs/get-started/install
- Ensure you have all necessary dependencies (e.g., Android Studio, Xcode for iOS, VS Code, etc.) and run flutter doctor to verify your setup.
Run the Application
- Navigate to the Chat-App/ directory in your terminal:
```
cd Chat-App/
```
- Get the project dependencies:
```
flutter pub get
```
- Run the application on your desired platform (e.g., Android, Windows, Web):
```
flutter run
```
  - To run on a specific device or platform, use the -d flag (e.g., flutter run -d windows, flutter run -d chrome, flutter run -d <device_id>).
  - Ensure you have a device connected or an emulator/simulator running for mobile platforms.

4.3 KiB Raw Blame History