Language exercises often remove the uncertainty that makes real conversation difficult. This project explored the opposite: placing a learner inside a scene where they could speak, receive a contextual response and continue a generated narrative.
- Experience
- Conversational VR
- Languages
- 38+ supported
- Client
- Unity and C#
- AI services
- Gemini, LangGraph and Whisper
The interaction
Practice a language inside a situation
The learner speaks to an NPC rather than selecting a fixed dialogue option. OpenAI Whisper transcribes the speech, the application interprets it in the current scene and the character responds as part of the ongoing interaction.
The purpose was not to simulate unrestricted conversation. It was to give learners a reason to use language: respond to a character, move the scene forward and deal with an answer they had not memorised in advance.
Orchestration
A narrative that remembers where it is going
LangGraph coordinates the dynamic narrative, Gemini provides generated dialogue and a Python FastAPI service connects the AI workflow to the Unity client. The structure supports more than 38 languages while keeping each response grounded in the current conversational context.
The interesting engineering problem was not generating one reply. It was maintaining a coherent learning experience across an unfolding conversation.
Next questions
Measure learning, not novelty
A future version of this case study should document the interaction latency, how interruptions are handled and how a learner’s progress could shape later scenes. It should also test whether the open-ended experience improves confidence or retention compared with a simpler conversational exercise.
Those questions are the difference between an interesting AI prototype and a useful learning product—and they are where I would take the work next.