Tools
Sibilla Live Link
A voice-driven AI shopping assistant for the АОЭ shopping centre. A virtual character on a holographic panel answers visitors’ questions and helps them find the right store.
About the project
Sibilla Live Link is a virtual shopping assistant project for the АОЭ shopping centre. A 9 × 16-inch holographic panel introduces Sibilla, a digital character visitors can talk to in one of three supported languages.
The idea is to make finding products and stores part of a natural conversation. Visitors simply address Sibilla: she answers their questions and supports her explanations with clear visual information.
How Sibilla works
- Voice activation. A visitor says “Hello, Sibilla” in one of the three supported languages. This starts the interaction, and the character plays a greeting animation.
- AI conversation. The system receives the visitor’s spoken request, and ChatGPT processes it to generate a response.
- Speech and facial animation. An audio file of the response is sent to NVIDIA Audio2Face. The resulting facial animation is streamed through Live Link into Unreal Engine 5.3 to keep the character’s expressions aligned with her speech.
- Shopping guidance. Sibilla speaks her answer and displays visual information showing where visitors can buy the product and where the store is located within the shopping centre.
Technology
- Unreal Engine 5.3 and C++ — the application, character rendering and interaction logic.
- ChatGPT — processing requests and generating responses.
- NVIDIA Audio2Face — generating facial animation from audio.
- Live Link — streaming character animation into Unreal Engine.
- TCP/IP — communication between system components.
