Virtual Tutor

Disclaimer: This project is still a work in progress and in early development.

Project Structure

Unity VR/AR Application (Unity/)

Scenes

• Assets/Scenes/VR/ - VR learning tasks (1-4, task 1 currently usable)
• Assets/Scenes/AR/ - AR learning tasks (1-4, task 1 currently usable)
• Lobby.unity - Main lobby/menu scene

Key Scripts

• TaskTimer.cs - Configurable countdown timer with UI display, color warnings when time is low, and auto-return to Lobby on expiration
• TaskButtonManager.cs - Scene navigation using number keys (1-8 for tasks, L/0 for Lobby) and UDP remote control (default port 5555)
• PositionTracker.cs - Records Transform positions to CSV at configurable intervals for data collection

Technologies

• Convai SDK for AI-powered conversations
• Ready Player Me for avatars
• Meta Quest (Oculus) XR support

Setup Instructions

1. Enable Quest Link

   • Open the Oculus app on your PC
   • Connect your Quest headset via USB-C cable
   • Put on your headset and enable Quest Link when prompted
   • Alternatively, use Air Link for wireless connection (Settings → Experimental)

2. Enable Beta Passthrough (for AR tasks)

   • On your PC, open the Oculus app
   • Go to Settings → Beta
   • Enable "Passthrough over Oculus Link" or "Passthrough API"
   • Restart the Oculus app and reconnect your headset

3. Run the Unity Project

   • Open the project in Unity Editor
   • Load the Lobby scene from Assets/Scenes/
   • Press Play in Unity Editor
   • The application will run on your Quest via Link

Navigation Controls

• Number Keys 1-8 - Switch between VR/AR tasks
• L or 0 - Return to Lobby
• VR controllers for interaction within tasks

UDP Remote Control

The Unity application listens for UDP commands on port 5555 (configurable in Unity Inspector) for remote scene switching.

Command Format:

SWITCH:SceneName

Example Commands:

# Switch to VR Task 1
echo "SWITCH:VR-Task1" | nc -u localhost 5555

# Switch to AR Task 2
echo "SWITCH:AR-Task2" | nc -u localhost 5555

# Return to Lobby
echo "SWITCH:Lobby" | nc -u localhost 5555

Python Example:

import socket

def switch_scene(scene_name, host='localhost', port=5555):
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    message = f"SWITCH:{scene_name}"
    sock.sendto(message.encode('utf-8'), (host, port))
    sock.close()

# Usage
switch_scene("VR-Task1")

Available Scene Names:

• VR-Task1, VR -Task2, VR -Task3, VR -Task4
• AR-Task1, AR-Task2, AR-Task3, AR-Task4
• Lobby

Timer Commands:

• TIMER:START - Start the timer
• TIMER:PAUSE - Pause the timer
• TIMER:RESUME - Resume the timer
• TIMER:RESET - Reset the timer to initial duration

---

Chat Application (Chat-App/)

Flutter-based cross-platform chat interface.

Note: To get API key and Character ID, contact me at tom.hempel@stud.hs-coburg.de or if no longer a student at contact@tomhempel.com

Features

• Cross-platform support (Android, iOS, Windows, Linux, macOS, Web)
• AI-powered conversations using Convai SDK
• Conversation history with persistent storage
• Automatic translation support via DeepL API

Installation

Pre-built installers available in App-Installers/:

• Android - AI-Chat-Lab.apk
• Windows - AIChatLab-Setup.exe
• Linux - ai_chat_lab-0.1.0-x86_64.AppImage

Development

To develop or run the application from source, you'll need to set up your Flutter development environment.

1. Install Flutter SDK

   • Follow the official Flutter installation guide for your operating system: https://flutter.dev/docs/get-started/install
   • Ensure you have all necessary dependencies (e.g., Android Studio, Xcode for iOS, VS Code, etc.) and run flutter doctor to verify your setup.

2. Run the Application

   • Navigate to the Chat-App/ directory in your terminal:

     cd Chat-App/
     

   • Get the project dependencies:

     flutter pub get
     

   • Run the application on your desired platform (e.g., Android, Windows, Web):

     flutter run
     

     • To run on a specific device or platform, use the -d flag (e.g., flutter run -d windows, flutter run -d chrome, flutter run -d <device_id>).
     • Ensure you have a device connected or an emulator/simulator running for mobile platforms.