Participating in the NASA SUITS Challenge taught me a lot about working in a large team of developers and human factors researchers and how to navigate challenges with team productivity and meeting deadlines. I tried to step in and be there for the team for anything that we needed beyond the design and development side including securing a hololens for development, a travel scholarship to cover funding and representing the team at software reviews and test week. However, there was definitely a lot I could improve on such as setting up more coding sessions between teammates and ensuring that travel funding was secured earlier so that more teammates could have traveled to test week. Overall, it was very rewarding to see the project unfold from initial design and development to deployment and testing at Johnson Space Center with NASA engineers. I was also grateful that our project got exposure in the school news and we were able to publish 4 research publications and present at national conferences. Reflecting on the whole experience, I’m glad that I got to work with a talented and dedicated team as well as create a project that could influence future NASA missions/operations.
Navigation HMD
Egress/Ingress HMD Procedure
Egress/Ingress HMD Procedure
We tested our prototype in a mock extravehicular activity (EVA) scenario May 19-23 at Lyndon B. Johnson Space Center, showcasing our work to NASA as well as university and industry partners. During EVAs or spacewalks performed outside the spacecraft, astronauts wear spacesuits and exit the lander through an airlock to perform tasks such as exploring surface terrain, conducting research and interacting with various payloads and lunar assets, including life support systems, rovers and habitats. To emulate a realistic astronaut spacesuit, the testers wore a DCU with switches and buttons connected to the EV Telemetry stream backend data with an intercom receiving instructions from the local mission control center (shown on the top left image). The mock EVA scenario also included the physical UIA (top middle image) with buttons and switches and a cable repair repair task (bottom middle image) connected to the LMCC. The testers also had a physical scanner clipped onto their side for the geological sampling task (bottom left image). We connected our LMCC prototype to computers that were set up by the team (bottom right image).
A prominent notification will appear in the HMD when a sample is completely scanned and brief information will be provided for the sample (e.g. Scanned: Rock). There will be numerical data for displaying real-time spectroscopy data for the scanned sample, showing elemental compositions and concentrations. A message “Analysis in Progress” with red color will be displayed to indicate the LMCC is in progress to analyze the sample. This message will change to “Analysis Complete” with green color to indicate that LMCC is done with sample analysis. When the DE is ready to proceed to the next scan, a notification appears at the top of the display, indicating that the system is ready for the next sample (e.g.“Proceed to next sample”)
Breadcrumbs are automatically dropped at predefined intervals and are used to help users retrace their path. Users can toggle their visibility by activating Follow-Path Mode through the hand menu and entering the two POI’s that they are navigating between. Upon entering Follow-Path Mode, the map will appear and the DE can select two of the POIs to select the path. These breadcrumbs are stored sequentially based on the two destinations they were placed between in incase they lose track of direction. The most recent marker passed along with the total number of markers is displayed in the top right corner of the FOV. The farther they are from the DE, the smaller they appear. Breadcrumbs beyond 20 meters away from the DE are not displayed to minimize clutter.
The system is composed of software and hardware components, the Local Mission Control Centre (LMCC) Dual Display System and Head Mounted Display (HMD) system, designed to enhance the effectiveness and safety of Extravehicular Activities (EVAs). The system is equipped with a Main Screen and an Active Screen, both offering crucial information and task-specific interfaces for Design Evaluators (DEs) during missions. Key components of the system include EV Telemetry with advanced data visualization, a dynamic caution and warning system, mission timers, navigation tools featuring markers and breadcrumbs, and a rover control interface. Additionally, the system integrates interfaces for Egress, Utility Airlock, Equipment Diagnosis and Repair, Geological Sampling, and Ingress procedures, ensuring seamless communication and task execution.
The system is composed of software and hardware components, the Local Mission Control Centre (LMCC) Dual Display System and Head Mounted Display (HMD) system, designed to enhance the effectiveness and safety of Extravehicular Activities (EVAs). The system is equipped with a Main Screen and an Active Screen, both offering crucial information and task-specific interfaces for Design Evaluators (DEs) during missions. Key components of the system include EV Telemetry with advanced data visualization, a dynamic caution and warning system, mission timers, navigation tools featuring markers and breadcrumbs, and a rover control interface. Additionally, the system integrates interfaces for Egress, Utility Airlock, Equipment Diagnosis and Repair, Geological Sampling, and Ingress procedures, ensuring seamless communication and task execution.
The system is composed of software and hardware components, the Local Mission Control Centre (LMCC) Dual Display System and Head Mounted Display (HMD) system, designed to enhance the effectiveness and safety of Extravehicular Activities (EVAs). The system is equipped with a Main Screen and an Active Screen, both offering crucial information and task-specific interfaces for Design Evaluators (DEs) during missions. Key components of the system include EV Telemetry with advanced data visualization, a dynamic caution and warning system, mission timers, navigation tools featuring markers and breadcrumbs, and a rover control interface. Additionally, the system integrates interfaces for Egress, Utility Airlock, Equipment Diagnosis and Repair, Geological Sampling, and Ingress procedures, ensuring seamless communication and task execution.
Head Mounted Display (HMD)
Local Mission Control Conter (LMCC)
Pins are essential markers used for various purposes, including hazard identification, specimenmarking, and general point demarcation. Specimen pins are employed to mark out specimens, which can be dropped by the rover. They can be treated as a POI if the DE needs to navigate to a specimen to retrieve it. General Pins: These versatile markers, deployable by both the DE and the LMCC are a quick way to demarcate objectives and focus. Operators can choose to toggle their visibility on or off. Hazard Pins: Hazard-based markers serve to highlight potential hazards.
The size of pins decreases as DE moves farther from the pin, optimizing visual clarity and preventing map clutter. Pins are not displayed beyond a certain distance, often 20 meters, ensuring that only relevant markers are visible. Operators can toggle the visibility of pins on or off, enhancing interface customization to focus on specific mission requirements. Pins come with different icons and colors, allowing for easy categorization and identification of marked points.
The goal of this component of the extra-vehicular activity (EVA) procedure is to create a design that allows the DE to navigate in their environment to complete tasks. To successfully navigate the lunar surface, the DEs will have the ability to drop markers in the environment. The design implements four types of markers: pins, breadcrumbs, points of interest (POIs) and live location markers. Each marker allows the DE to safely, effectively, and efficiently navigate their environment.
The HMD interface design features a mini diagram of the UIA on the side which lights up the switch corresponding to the current task, assisting the DE with locating the switches on the panel more easily. Further, The current task to be completed by the DE will be displayed with the instruction and the relevant progress bar which tracks the relevant metrics for each task (eg. When UIA Supply Pressure is > 3000 psi, proceed). The HMD egress interface features a progress bar which tracks the status of the whole egress procedure which provides the DE with greater understanding of the task.
The Geo Sampling LMCC Active screen is designed for real-time spectroscopy data for the scanned sample which shows elemental compositions and concentrations. For the HMD display, there will be a notification bar to appear when a sample is completely scanned and brief information provided for the sample. The design for the navigation LMCC Active Screen contains a minimap that provides a comprehensive overview of the surroundings within a 20-meter radius and systematically assesses the landscape for hazards, automatically or manually marking hazards using dedicated markers such as general pins, points of interest, hazards, specimens and the locations of the two crewmembers and rover. For the HMD display, the markers menu can be accessed through a small hand menu activated by gazing at the left hand.
The LMCC interface will prominently display an ordered list of procedures required for the egress task. Each step will be clearly outlined and updated in real-time as the DE progresses through the task. When a task is complete, the LMCC user will click on the next task and the previous task will be grayed out. Further, the LMCC interface will implement bar metrics to provide at-a-glance information on critical metrics that updates for each task. There will be scales on bar graphs indicating the levels for all the necessary parameters. When each task is initiated, the corresponding bar will have a highlighted segment representing the target level that must be reached.
The LMCC interface will prominently display an ordered list of procedures required for the egress task. Each step will be clearly outlined and updated in real-time as the DE progresses through the task. When a task is complete, the LMCC user will click on the next task and the previous task will be grayed out. Further, the LMCC interface will implement bar metrics to provide at-a-glance information on critical metrics that updates for each task. There will be scales on bar graphs indicating the levels for all the necessary parameters. When each task is initiated, the corresponding bar will have a highlighted segment representing the target level that must be reached.
The spacesuit diagram will present the status of the spacesuit using a detailed diagram with different components highlighted. When a problem arises the color of the affected component will change on the diagram. A panel will display a live video feed from the crewmember's HMD on a dedicated screen. An overlay system will be used for caution and warning indicators. For example, if there's an issue with the DE's visor or camera, that portion of the video feed will be highlighted. Also, when a caution or warning is triggered, the panel window could flash and change its border color. Shown below are intial LMCC user interface mockups designed in Figma.
The EV Telemetry provides a comprehensive overview of the EVA situation, allowing LMCC to quickly identify and respond to any issues. The dashboard within the LMCC interface displays crewmember’s biomedical data of vital signs such as heart rate, blood pressure, oxygen levels, and body temperature using color-coded visualizations. In this dashboard, dynamic graphs would be used that change color or shape to alert LMCC in case of abnormal readings.
The EV Telemetry provides a comprehensive overview of the EVA situation, allowing LMCC to quickly identify and respond to any issues. The dashboard within the LMCC interface displays crewmember’s biomedical data of vital signs such as heart rate, blood pressure, oxygen levels, and body temperature using color-coded visualizations. In this dashboard, dynamic graphs would be used that change color or shape to alert LMCC in case of abnormal readings.
Our team’s software development plan involved using a Zello board to keep track of all the existing tasks and who was in charge of them. We held weekly working sessions and team meetings for updates and progress. We also utilized our team’s github with branches for each of the EVA tasks such as Geological Sampling and Navigation and labeled them based on front-end or back-end and the task abbreviation such as F-NAV1. For keeping all of our documents together, we used an easily accessible google drive with folders for user manuals, research and presentations.
Our team’s software development plan involved using a Zello board to keep track of all the existing tasks and who was in charge of them. We held weekly working sessions and team meetings for updates and progress. We also utilized our team’s github with branches for each of the EVA tasks such as Geological Sampling and Navigation and labeled them based on front-end or back-end and the task abbreviation such as F-NAV1. For keeping all of our documents together, we used an easily accessible google drive with folders for user manuals, research and presentations.
