LOOPING

Blockly Module Library Frontend/Backend Integration looping a module looping gesture

What we accomplished

We refined the module backend so that we can support more modules to be added to the module library. We are still working on finalizing the modules (in particular, the step of applying a saved module and what exactly the module library will look like). In addition, we finished both the frontend and backend for looping. By the end of this week, we hope to integrate the two parts together to complete the looping construct. Lastly, we have started on the UI by incorporating a main menu for the game.

  • Yuma: Worked on module library backend – added support for displaying multiple saved modules in the library and added cursor positions to modules in the library. Added code to fail out of applying module if there is no space.
  • Sea-Eun: Implemented the backend for looping over modules. Also started the UI for the game - inserting a main menu consisting of puzzle mode and free play mode. Upon puzzle mode, levels will pop up for the user to select.
  • Logan: Finished the Oculus + custom pose recognition merge (modulo a few bugs), so now we can simultaneously use Oculus’s pinch recognition and ray tool with our custom pose recognition. Connected the backend and frontend module libraries, though there is still a bug preventing dragging and dropping modules to apply them in only some of our Unity scenes. The module library now displays a miniature version of the recorded module that the user can then drag and drop into the play area to apply it.
  • Erik: Added the gesture for looping and a counter to the backend that shows how many times the user has looped with that gesture. Continued to debug the issues introduced when adding module hand gesture (ie. module hand gesture is very close to open hand).

Plan for next week

For this upcoming week, we want to polish and finish up the current features (such as looping and modules). We also want to work on adding UI and backend support for the tutorial and puzzles that will guide the user through the application to learn the programming constructs and their corresponding gestures.

  • Yuma: Work on backend for puzzles - add target structure prefabs (wired blocks) and implement verification of user’s completed structure against target structure (including cursor’s end position, which will be used to indicate when a user submits). Scale the blocks to be smaller so they are more easily visible.
  • Sea-Eun: Create tutorial pop-ups - displaying directions on each level. Update main menu correspondingly. Assist Yuma with the backend if needed.
  • Logan: Fix bugs on the front end for dragging modules into the main area. Improve pose recognition by using multiple reference poses.
  • Erik: Implemented discarding of poses that only happened very briefly. Implement the response to a completed level (ie. music, some message). Implement the “cheat sheet” for puzzle mode (stationary text that reminds the user can reference -> move to instantiations of the hand prefab that the user can reference, maybe also a popup that shows up when user enters free play mode).

Upcoming Challenges

Puzzle UI might be kind of a weird experience for the user (we’re still working out exactly how the user will advance to the next level). We want this to be clear, but not disruptive.

Notes