Summary
Students at the GRILL® worked to create a plugin aiming to improve language learning, prevent culture shock and insensitivity, and provide better social understanding with cultural awareness and advanced AI.
The project seeks to revolutionize language education by optimizing learning outcomes and promoting a deeper appreciation for diverse cultures and languages.
Features and Design
Software
Unity
ChatGPT 3.5-FAST
LLaMA 2-Tiny LLM
Blender
Overtone
Undertone
Source Tree
MoCap Animations
Oculus Lip-Sync Plugin
_
Contributions
Team:
Ethan Cochran
Sean Gupta
Gavin Kendall
Manasi Singh
Mentor:
Jon Diemunsch
_
Clients:
DOD Defense Language Institute
Master Sargent High
Outcomes and Additional Information
The purpose of the project was to create program that can increase cultural comprehension, prevent culture shock and insensitivity, and support better social understanding.
The students modeled animated facial movements as well as speech tones. Users can engage with the AI model through natural language conversations, asking questions and receiving contextually appropriate information.
Prompt engineering utilized ChatGPT with personality customization and emotional analysis. A "human editor" was created to customize visual, behavioral, and cognitive attributes.
Although the project started with ChatGPT, the team shifted to the Local Large Language Model (LLAMA 2-Tiny LLM), inputting information to train the model to the use case.
Future projects can feature large language models, more emotionally expressive facial animations, and integration of additional languages and cultures.