Week 10

Final release

What We've Accomplished

This week is the final week of our development. Besides preparing for demos, we also made some improvements to our app. First, we merged survival game mode into the game. In this gamemode, players have 3 lives to begin with. Each time they miss a orb, their lives decrease by 1. If they touch a bomb or don't have any lives left, the game is over. Later on when we are testing the game, we found it would be too hard to give a fixed 3 lives, so a mechanism that gives the player an additional life for every 3 balls they catch is added to the game. We also finished the bug that spawn frequency in settings is inversed and the bug that orbs are not properly cleaned up after each game. Some UI polishment is also done to make the game looks more pleasing.

In the final version of the game, there are 3 gamemodes that can be changed in the settings along with other options such as ball speed and BGM volume:

  1. Unlimited mode: Player can play as long as they want without limitation of time. This is similar to a exploratory mode for user to get familiar with the game.
  2. Timed mode: The goal is to catch as many orbs as they can in a limited time that is adjustable in settings.
  3. Survival mode: This is the most challenging and fun mode of our game. In addition to trying to catch the orbs with no misses, players also need to dodge bombs.

The final version of our game is available here.

Contributions from each team member:
Yating: big help in debugging, setting up zoom sessions.
Yuxuan: bug fixing and improvements to survival mode
Ryan: demo video and demo presentation slides preparation, UI polishing
Ana: survival gamemode integration

Week 9

New gamemode and UI improvement

What We've Accomplished

Based on last week's progress, this week we refined game menu, added a new gamemode, and made several refine.

  1. UI Improvements
    • Change menu fonts and add logo banner to make it more visually appealing.
    • Add help text for different game sceniros.
  2. Timed gamemode: Players need to catch as many orbs as they can in designated time.
  3. Add more setting options such as an option to remap play space.
  4. Change start/end hole material for a consistent game play theme.
  5. Add controller vibration when orbs are caught.

Detailed information

Since we only have two weeks left to refine the game, we made some design choices on what to do or not to do. We did not implement a visual indicator (arrow on screen pointing to direction of the ball) because we think the orb path is already a clear indicator for player to see the orbs. This week we made a lot polish on UI elements so that it looks less serious. Lots of help message were added so that first time player can get their hands on the game quickly. A timed mode was added to increase the intensity for the game. Different from unlimited mode, a timer shows up on the primary wall of playspace instead of game summary, which is shown when time is up. We also incorporated advice from course staff to increase luminosity of orbs to make it more clear on demo videos. Adding a vibration to controller when a orb is caught also increases user experience drastically because of feedback.

Contributions from each team member:
Yating: First version demo video
Yuxuan: Timed gamemode
Ryan: UI improvement
Ana: Survival gamemode (releasing next week)

Plans for next week

The game is almost complete. We will finish ongoing changes to the game and focus on preparing slides and videos for demo day. Here are some of the things we want to do:

  1. Add a survival game mode
  2. Record a clear demo video with Magic Leap
  3. Decorate spatial room and prepare for presentation

Week 8

Visual and UI System

What We've Accomplished

This week we refined the game logistics, created a menu, and updated the game visual effects.

  1. Visual Effects: introduced particle system for ball and explosion effects. Changed material for path.
  2. Added Menu and Game Summary on the Primary Wall
  3. Used the primary wall to display the menu and game summary
  4. Introduced Settings: allowed players to turn the path or mesh visuals on/off, change BGM and sound effect volume, and adjust spawn rate and ball speed.
  5. Introduced Mesh Visual: added blue gradient indicator at the center of the viewp to indicate the furniture/wall mesh, could be turned on/off in the settings.

Past Obstacles and Working Solutions

The meshing system in the previous versions uses a black material which is claimed to be transparent in ML's display. However, after several tryouts, we figured that there's usually a black shadow on the meshing. The black color and occlution behavior are not ideal for our game environment. So we updated the material with a transparent base with blue pattern. This way the players could see the mesh visual in the center of their view to predict ball collision and could also turn off the function at their choice.

The UI system also posed great obstacles to the development process. In order to develop the main menu, the game logistics need to be clear and modular enough. Furthermore, the menu interaction is tricky as we need to combine controller behavior, raycasting, and custom UI elements together to trigger appropiate events. Some of the UI elements is particularly harder to manipulate in such a system, such as a slider. So we chose buttons for most of the UI elements, even for numerical values - clicking a button will just add or subtract certain value with a ceiling/floor in that case. In addition, the position of the menu involves a bit of consideration too. We face several problems regarding menu placement: 1) it is possible to be blocked by the mesh visual of the wall, 2) when the wall has decorations (such as frames, obstructive fire place, etc), the mesh may also block the menu. After experiments and discussion, we made the following adjustments: 1) move the UI canvas out from the wall for 10cm as MagicLeap did in their example, 2) prompt the players to choose a wall with flat surface in the center, 3) make the bottons in a slightly smaller but reasonable size so that the overall size of the menu/setting items is reasonable for the player to find a wall with those requirements.

Plans for next week

Moving forward, we are planning to do the following things:

  1. Explore Different Game Modes
  2. Explore Powerups
  3. Considering adding different exercises (maybe make the ball bounce?)
  4. Explore Visual Indicators

Week 7

Improvements based on MVP

What We've Accomplished

This week we made several improvements based on the advices we received when demoing to other people, refined the spawning algorithm and refactored code base to prepare fot the next round of updates.

  1. Refine ball spawning algorithm so that:
    • the ball does not appear outside of the playspace
    • ball path will not be too high for player to reach
    • ball path is less abrupt now: it wouldn't come out of the wall with a projection angle that is too small.
  2. Add a ring model to indicate the beginning and endding of a path
  3. Now the path can be a curve instead of just a straight line
  4. Add sound effect for ball spawning and despawning
  5. Refactor code base for a clean coding environment

Past Obstacles and Working Solutions

When we finished the MVP, the code is highly coupled, which is bad for future development. The score keeper script used for keeping track of score is referenced by other scripts a lot, and we decided to decouple it using the event/listener pattern to control the access to score keeper.

After adding start/end ring model for the path, we discovered that sometimes when the start/end point of path is on the floor, it can appear out of the playspace defined. This is caused by the fact that floor can be an irregular polygon, so the random point in a rectangular range defined by minimum/maximum x/z coordinate of the floor may not be in the placespace. We solved this by using a function provided by Magic Leap to examine whether the point we have chosen is inside the region, and if not, rechoose a point.

Plans for next week

Moving forward, we are planning to do the following things:

  1. Replace ball/trail with better model so that it is more attracting
  2. Add a menu for the game, so that we can have different game modes and custom settings.
  3. Make use of the primary wall of playspace to display informations like game summary.
  4. Use hand meshing to catch the ball.

Week 6

MVP Released

What We've Accomplished

This week we released the MVP and live-steamed the demo on Thursday's session.

New Release

MVP Repository : including the Unity project and prebuilt MVP.mpk

Code Update

  1. Added Playspace: Playspace.cs
  2. Added furniture meshing and collision: MLSpatialMapper.prefab
  3. Modified spawning algorithm: SpawnManager2.cs
  4. Modified exercise categorizing algorithm: ScoreKeeping.cs

Past Obstacles and Working Solutions

Plane Extraction: The game requires detecting planes (walls/ceiling/floor) for ball to spwan or end its path. The orginal plan is to use the existing cached planes built-in to Magic Leap. However, in the case that ML mapped the whole house/apartment, all the planes woule be activated and there is no effective way to narrow down the area to a reasonable range considering that players might move from where they wake up ML and each player's room size varies. Then we try to activly use the plane extraction functionality provided by ML at the beginning of each game, but the active plane extration runs into the same problems. Eventually, Playspace by ML seems to be the best solution, where players can define their own playing area and such an area can be cached for future usage.

Furniture Meshing: We envisioned the ball to crash on the furnitures if not catched before that. Such effect requires meshing for the furnitures, but the usual practice for collisions would cause the ball to crash on the spawning plane or something very close by (eg. a TV hanging the spawning plane), causing players to have lots of frustrations as such settings make some ball almost impossible to catch. Attempts to fix the issue including calculating the collision point distance from wall and exculding those that are close; categorize the meshing with tags and condition for collision crash effect. But none of them provides a satisfying effect visually or methematically. In the end, we added a timer for each ball. If a ball is spawned and collided with the mesh within some threshold (eg. 2 seconds), we do not crash the ball. This current working solution guarantees that no ball would collide on spawning wall or immediately after being spawned, but still allows the balls to collide on furnitures in the future, reducing the player frustration overall while made the visual effects possible.

Exercise Categorizing: The original exercise categorizing system utilizes the ball location to count if a ball-catch is a stretch-up or squat-down. But the ball location turned out to be very misterious, thus we are currently using controller position instead. If the controller is sustituted in the future, other Solutions are needed for this part.

Plans beyond MVP

Moving forward, we are planning to polish the MVP and add more features, indluding:

  1. Ball Spawning Algorithm: polish on ball spawning location to prevent balls from being uncatchable.
  2. Hand Meshing: explore hand meshing to replace one/both of the hand controls.
  3. Audio: add sound effects an dmusic to the scene.
  4. Ball Explosion Effects: add one more ball explosion effect to distinguish between catch and crash.
  5. Main Menu: add a main menu for player to choose between different mode and adjust settings.

Week 5

Individual Tasks Finished For MVP

What We've Accomplished

This week we divided tasks among the team members and finished those tasks for MVP. In specific, each team member is responsible for the following:

  1. Ryan: Meshing/Environment setup and spawning balls from walls
  2. Ana: Setup unity with Magic Leap, ball objects interaction with walls
  3. Yuxuan: Balls flying in the room and trajectories
  4. Yating: Scoring system

Demo

  1. Path/Ball Spawn and Collision Effects
  2. User Faciliating System: Score & Scan Texts
  3. User Faciliating System: Session Summary

Code Update

Codes for each task are uploaded into different branches of the repo

Plans for Next Week

Now that we have all the parts for MVP, we just need to incorporate those parts together to make a whole game.

Help Needed and Blocked Progress

Hardware/Software setup problems:

  1. Unity crash frequently on MacOS;
  2. Repo does not work for some teammates (asset missing, build failure);
  3. One teammate has difficulty connect simulator to Unity and has not received ML device yet.

Week 4

Setting up Magic Leap, working through tutorials

What We've Accomplished

This week we connected the Magic Leaps to our computers and started learning how to develop for the device. We watched a tutorial to get started. Learning how to develop for the magic leap before starting our actual app will certainly make development faster later on

New Release

No public releases this week

Code Update

No development of our app has begun, but we are working on a tutorial to make sure we are able to start soon

Plans for Next Week

The biggest goal is starting our actual app and getting a minimum version of the game working. Tasks to be finished next week:

  1. Game Implementation: Get wall/ceiling maps, spawn balls.
  2. Game Implementation: Show laser paths for balls.
  3. Game Implementation: Setup timer and score board.
  4. Add ball disappearing effect if the ball hits any obstacle.

Help Needed and Blocked Progress

The only block currently is that one of the team members is still waiting for the magic leap to arrive, making it harder to develop. Hopefully the device will arrive very soon

Week 3

Project Pitch, PRD, Environment Setup

What We've Accomplished

This week we refined the project idea in the Product Requirements Document and released the initial project pitch video explaining our ideas in the simple graphic/scratch form. The weekly milestones, evaluation metrics, and team member responsibilities are also refined for the next 7 weeks to ensure a smooth development cycle. Refer to the New Release section for more details.

New Release

Project Pitch Video

Product Requirements Document

Code Update

No actual development has begun, although we anticipate to finish setting up development environment for Magic Leap this weekend!

Ideas Update

MVP defined: stick with ball collection and time-based mode

  1. Path line for balls
  2. Balls from all directions moving along the path line
  3. If ball hits any other surface other than the one it came out of, disappear into particles
  4. Balls can be caught with both hands
  5. Scoreboard: Reward 1 pt per ball catched
  6. Timer
  7. Summary: how much time spent in the game, how many squat/stretch you got

Target product defined: visual indicator, more player control, and enjoyable environment

  1. Path lines can be turned on and off
  2. Ball speed can be adjusted by the user
  3. Interesting game environment
  4. Balls coming out of holes in the wall
  5. Visual indicator

Stretch goals defined:

  1. Intro instructions
  2. Obstacles
  3. Powerups
  4. Survival game mode
  5. Social media, post high scores

Plans for Next Week

The biggest goal for us moving forward is to get a simple scene up a running, and become more familiar with the magic leap development kit. Tasks to be finished next week:

  1. Dev Environment Setup: Setup unity with magic leap.
  2. Game Implementation: Get wall/ceiling maps, spawn balls.
  3. Game Implementation: Show laser paths for balls.

Help Needed and Blocked Progress

We currently have nothing blocking us, although we anticipate getting help from Joerg in order to help us through the initial setup and the various input modalities (gestures, control, etc.).

Weeks 1 + 2

Getting Started, Team and Device Setup, Project Requirements

What We've Accomplished

This week was anticipated to be slower than others, we spent a lot of time getting familiar with the course structure, our magic leap devices, and the development kits necessary to begin developing. The biggest thing we accomplished, as expected, was deciding on our project idea. We were initially interested in designing a medical exam, but ultimately decided that our knowledge and technical expertise would limit us there. Instead, we’ve decided to design a fitness game in hopes that it can help people stay engaged and active during these unusual times.

The current idea is to have users run around and collect balls falling from the sky in a timely fashion, while avoiding obstacles. It will be great exercise for the users legs, neck, and cardiovascular system as they are constantly moving, reacting, and changing directions. Depending on the difficulty of implementation, we plan to add more features or other fitness style mini-games.

Beyond the actual project idea, we’ve set up a slack channel and a google drive folder which houses all our documents. Working remotely has posed to be more difficult than expected, but we will begin to utilize more virtual zoom meetings in order to stay on the same page and delegate tasks as efficiently as possible.

Code Update

No actual development has begun, although we anticipate to begin within the next week.

Ideas Update

Here are the main elements we’ve begun to consider for our game:

  1. Different game modes and objectives: One mode could be to collect as many items as possible during some time frame, another could be to collect as many items without catching a bomb.
  2. Game objects with varying properties: Some will fall and disappear through the floor, while others will bounce around with different speeds and directions. Even something like the golden snitch could be a great addition.
  3. Various obstacles: We already mentioned bombs, but other things like black holes, items to avoid, and terrain to traverse will both increase the difficulty and physical activity of the game.
  4. Game props: We want to have power ups or props that give the player special abilities like doubling their score, slowing down time, or clearing all objects in the scene.
  5. Musical elements: Certain block types will have certain sounds, there may be background music, and some sort of indicator for a new record or even winning the game.
  6. Visual effects: This will help attract the players to certain objectives. For example, if an object hits the ground it could have a splash effect, or display a “-1” text.
  7. Varying game environments: The game could either be set up on the ground or in the sea. If it were to be in the sea, the items would float when they hit the water. We could even have a sports court to reinforce the fitness aspect of the game.

Plans for Next Week

The biggest goal for us moving forward is to get a simple scene up a running, and become more familiar with the magic leap development kit. Tasks to be finished next week:

  1. Environment setup: explore how to deploy the app to our devices or set up an emulator on our computers for debugging and testing purposes.
  2. Mentoring & Planning: figure out who is familiar with the features we want in our game and can help us establish a clear plan for development as the four of us haven’t done much AR development up to this point.
  3. Resource/Asset Discovery: Explore and gather some prebuilt assets (eg. scene, objects, animations, user interaction) that could be used in the game. If possible, use the assets to build a simple scene and identify the possible tech/non-tech challenges we need to solve in the future.

Help Needed and Blocked Progress

We currently have nothing blocking us, although we anticipate getting help from Joerg in order to help us through the initial setup and the various input modalities (gestures, control, etc.).