VIPO
VIPO - Virtual Interactive Puppet Overlay
VIPO, a term coined by content creators DoigSwift, is a constantly evolving, interactive video game project. Built in Godot, the application works as an overlay for creators to use while streaming, providing controls for a character, built-in custom interactions for viewers to use, and a rendering system to display the contents of a separate monitor within the application.
Major Systems
Screen Rendering
One of the core features of the VIPO system is the ability to display the contents of a specific monitor in real time. This is done by rendering the contents of the desired screen onto a subviewport, which is then assigned as the material on a plane in the 3D environment of the VIPO. This allows the contents of the user’s screen to exist within the environment and behave, if desired, like any other object in a 3D space. For the purposes of this project currently, the screen has no collision detection and cannot move, though like many elements of this project, this may be subject to change.
The use of Godot’s system access to each screen had one drawback, while it will render almost anything that displays on the selected screen, it will not render the user’s cursor unless it is an explicit part of the application being accessed. To supplement this missing detail, there is a custom cursor asset that translates the cursor’s coordinates on the desired screen into 3D space within the VIPO so it aligns with the viewport’s position.
Video Rendering
Oftentimes while streaming, the creator may need to step away from the microphone momentarily. It is a common approach to have separate scenes setup in your streaming software, such as OBS, with a “Be Right Back” screen. This caused an issue while using Godot projects. During my testing, I found that swapping between scenes that focused on Godot projects in OBS caused the Godot project to crash.
While I am still ironing out the cause of these crashes, I circumvented the issue at least for now by implementing a video renderer that can be toggled at any point within the VIPO application. This enables the user to cover up the screen entirely momentarily while they step away. This work required more understanding of how subviewports work, as well as the video renderer elements within Godot.
Stream Integration
The application supports viewer interaction through integration with the Streamer.Bot tool. To do so, I created a custom websocket within the VIPO application that connects to the local Streamer.Bot client I was running. This websocket allows for the development of custom actions that can be called via specific commands typed in chat, actions available for purchase through the specific streaming client (Twitch, in this use case), and more.
Currently, viewers are able to temporarily change the size of the VIPO character, with more actions coming in the future.
Audio Detection (WIP)
Moving into the specific character, or Puppet, of the VIPO, this application supports audio detection by slightly shifting the colors of the character model to reflect audio breaching the assigned threshold. Recently, I began the process of upgrading the character model from the original default capsule to the Jelly Jar. As a result, the audio detection is not always visible, as it currently only displays during periods where the model’s arms are spawned in. The plan is to continue working on the model for the Jelly Jar until it, too, supports audio detection in some shape or form.
Character & Camera Control
The Puppet character within the VIPO is controllable by the user primarily via a gamepad controller. The user is able to move the character at a set speed, jump, toggle different entities, such as the video renderer and occasional character decorations. Examples being a party hat during a birthday stream and a Jack-O-Lantern. All character inputs are still enabled even as external stream integration actions are called.
Additionally, the user has a designated button for unlocking the camera from its default location, and a reset button to bring the character and camera back to their default locations. This allows for future exploration of the 3D space within the VIPO, as work on the environment progresses.
Future State
As stated before, this project is a Work In Progress. It is being developed by myself during stream. The direction for this project is to work with a handful of artists to develop concept art for both the character and the environment and slowly work on creating the necessary assets in Blender and integrate them into the project. This is a long term project, intended to illustrate growth over time and development of a critical complex skillset necessary for other larger gaming projects I hope to tackle in the future.
To keep up with the progress of this project live, come stop by my stream Mondays - Thursdays, 6PM-10PM US EST.
Development
Team Size: 1 (1 programmer/designer)
Status: In Progress
Duration: TBD (Start: 9/4/2024)
Engine: Godot
Platforms: PC
