What we accomplished
We created the puzzle mode that allows users to progress through levels that require users to use the programming concepts we implemented. We continued improving hand gesture recognition by improving the algorithm we use to match poses and excluding gestures that were only held for a very short amount of time. We also continued to improve the module library.
- Yuma: Scaled all blocks and grid size down so the entire scene is more visible to the user. Implemented backend for the puzzle mode - added prefabs for puzzles, verification of user’s block structure, a “submit block” for users to indicate they completed a puzzle, and 3 level designs.
- Sea-Eun: Implemented the level transitions. This includes revealing the correct level when, for example, Level 1 is clicked and transitioning to the next level once the user completes the current stage. Also added the popups that contains prompts and directions for the user to follow.
- Logan: Fixed module drag-and-drop flow and added support for dynamically adding modules to the module library. Prototyped an algorithm that uses multiple reference poses to improve recognition accuracy.
- Erik: Implemented code to discard gestures that were only held for a very short amount of time by requiring that a gesture be held for a set amount of frames before triggering its event. Added a menu that pops up when levels are complete.
Plan for next week
For next week, we want to finalize the puzzle mode and potentially add more levels. We also want to merge everything together and polish the entire user experience. We plan on meeting up during this upcoming week in case there are any issues during the final merging process. We’ll all be contributing to the powerpoint, Spatial room, and 1 minute video for next week, and below is the Blockly-specific task breakdown.
- Yuma: Add functionality to the puzzle backend that checks that the user created a module for levels that require modules. Add functionality to delete blocks when the user makes an “Emit” gesture where there already is a block. Add music for when user successfully completes (or fails) the level. Determine appropriate grid sizing.
- Sea-Eun: Make directions per level more clear (i.e. what the module for level 2 should contain, how many times to loop for level 3, etc.) Also improve the UI for the popups by merging with Erik’s code, which works on the Quest. Extra: consider additional levels to work up towards the ~castle~
- Logan: Finish pose recognition improvements. Add pose calibration spheres (for user-customizable poses). Add move gesture support to right hand. Combine module dragging with the loop gesture to loop modules.
- Erik: Implement using the pinch gesture to select the prompt to move to the next level. Add a cheat sheet so that users can reference all of the available poses.
Upcoming Challenges
We have been working on separate components for a couple of weeks now and need to merge everything together for the final demo, so this could prove to be a challenge in this upcoming week. Also, since our group worked with the Oculus quest, we might have to spend some extra time on the Spatial room for Thursday.