This project was developed in Unity using the C# programming language
Below is a SAMPLE WALKTHROUGH OF THE THEATER EXPERIENCE. Feel free to watch the video and follow along the page for details on the interactive theater simulation.
UI Overview
When it is the User's turn to speak their lines, they are prompted with stage directions, an indication that it is there turn, and an option to ask for a hint using voice command, as shown in the UI screen to the right.
The USer could ask for a Hint if they are not sure of their current line by saying something like, "What is my current line?". As shown in the demo video above, when the user asks for their current line, the UI would present it to them at the top, as shown on the left.
But if the user doesn't want to be given their line directly, they could ask for the previous line. This way, they could be prompted to the current context, but still have the challenge of remembering their line on their own.
Note:
In theatrical performance, context is a huge contributor to helping performers remember their lines. Providing users with a line (even if it's from some other character in the play) that comes before or after their current line can allow performers not only remember their lines better, but also associate it with other elements of the play that are concurrent with their lines.
In the same way you could ask for the previous line to gain some context, you could also ask for the next line, as shown to the left.
Feedback Mechanisms
AFTER THE USER SAYS HIS OR HER LINE AND ARE READY TO MOVE ON, THE CORRECT LINE IS SHOWCASED AT THE BOTTOM OF THE SCREEN FOR SEVERAL SECONDS for the user to see.
Add picture of End of experience scene
Digging into the code: Key Programming Features
Voice Activation
One of the key features of the Theater experience is the ability to use voice commands to get hints, just like an actor or actress would receive help from the play's director in real life.
Here, we use the Oculus Voice SDK and WIt AI, a voice recognition api for Natural Language, in order to capture user voice interactions within our Experience. The code on the left shows the setup for Wit Ai voice recognition.
voice recognition is activated through controller Input.
We can handle the voice requests through the function below, which takes in key values configured in the Wit API.
Processing Script
At the beginning of each experience, we must process the script by first extracting all of the characters in the scene, act, or play.
We can then provide the user an option for which character to play at the beginning of the experience.
Above, we parse the lines into 3 segments that are important for our experience: Characters, Lines, and Stage Directions.
CHaracter: The character who is currently speaking
Line: The character's line
Stage Direction: Where the character is supposed to be (or moving toward) during the line.
Handling UI in Update Loop
Controlling UI when it's the User's Turn
Controlling Ui when it's the turn of some other character (not User's Character)
Note: audio plays a recording of the Character's line. In future Iterations of the project, i will incorporate text-to-speech to more quickly create audio for characters' lines in the play to create a more natural and realistic theatrical experience.
Start coroutine to display correct line when user is finished saying his/her line.
Future Goals, Features, and Improvements
Allow user to Physically walk around stage (may also be dependent on available real world space), since movement around stage is an important part of the immersive theater experience.
Include a Stage Direction Indicator, where the user is alerted if they are not in the proper area of the stage for their current line. This ensures that the user is not only trying to follow their lines, but also their stage directions.
Change the color of the Line Confirmation from Blue to Red, or something more readable. Also have the line confirmation after the user says their line to display for a longer period of time (4-6 seconds).
Have two different modes: Rehearsal Mode and Performance Mode. In performance mode, have a virtual audience that causes slight noises/disruptions at random times throughout the play.
Have an avatar to act as the other characters in a play. It could also be a floating labeled block.
