Virtual Physical Exam Platform

Cross-Platform Diagnostic Cardiology Physical Exam App

Last semester, I worked in a team of four engineers to develop a mobile and web physical exam plaform for a local clinic, Kelsey-Seybold to streamline workflow for physicians, provide step-by-step guidance for medical technicans and enhance patient engagement.

Last semester, I worked in a team of four engineers to develop a mobile and web physical exam plaform for a local clinic, Kelsey-Seybold to streamline workflow for physicians, provide step-by-step guidance for medical technicans and enhance patient engagement.

Client

Kelsey-Seybold Clinic and Rice University

Services

Visual Design UI & UX Design

Industries

Healthcare

Date

August 2023 - May 2024

Testing and results validation are integral to ensuring the functionality, security, and user experience of Virtual Physical. General feature testing involves walkthroughs by five testers, evaluating usability and responsiveness across all platform functionalities. Security tests confirm data access restrictions, while Amazon S3 storage capacity tests ensure scalability and reliability. Data quality testing focuses on preserving the integrity of uploaded files, including audio, video, and image formats. Specialized tools are employed to compare pre- and post-storage data, confirming minimal quality loss. User experience testing encompasses user surveys to gauge the intuitiveness, visual appeal, and informativeness of the platform. Initial feedback from medical technicians and cardiologists highlighted positive ratings and valuable suggestions for improvement, such as implementing data quality checkpoints and incorporating text messaging for patient-cardiologist communication.

Testing and results validation are integral to ensuring the functionality, security, and user experience of Virtual Physical. General feature testing involves walkthroughs by five testers, evaluating usability and responsiveness across all platform functionalities. Security tests confirm data access restrictions, while Amazon S3 storage capacity tests ensure scalability and reliability. Data quality testing focuses on preserving the integrity of uploaded files, including audio, video, and image formats. Specialized tools are employed to compare pre- and post-storage data, confirming minimal quality loss. User experience testing encompasses user surveys to gauge the intuitiveness, visual appeal, and informativeness of the platform. Initial feedback from medical technicians and cardiologists highlighted positive ratings and valuable suggestions for improvement, such as implementing data quality checkpoints and incorporating text messaging for patient-cardiologist communication.

In summary, Virtual Physical represents a transformative solution in cardiology healthcare, driven by a user-centered design approach and close adherence to regulatory standards. While this platform was developed with the Kelsey Seybold Clinic in Houston, Texas in mind, further work in commercialization of the platform, confirmation of our platform’s adherence to regulations and standards, and marketing would be required before broader implementation. As we continue to iterate and refine Virtual Physical based on user insights, we remain committed to our mission of making specialty healthcare accessible for all and expanding the scope to any specialty medicine.

In summary, Virtual Physical represents a transformative solution in cardiology healthcare, driven by a user-centered design approach and close adherence to regulatory standards. While this platform was developed with the Kelsey Seybold Clinic in Houston, Texas in mind, further work in commercialization of the platform, confirmation of our platform’s adherence to regulations and standards, and marketing would be required before broader implementation. As we continue to iterate and refine Virtual Physical based on user insights, we remain committed to our mission of making specialty healthcare accessible for all and expanding the scope to any specialty medicine.

With the goal of revolutionizing cardiology healthcare accessibility and efficiency, our team developed Virtual Physical, a robust web platform tailored for the unique user needs of medical technicians and cardiologists. Guided by HIPAA compliance and user-centered design, our project integrates data from 4 medical devices foundational to cardiology, provides guidance for medical technicians completing a comprehensive cardiology physical exam, and enables remote patient evaluations by cardiologists. Our design strategy prioritized user feedback and adherence to healthcare regulations. While out of our scope, we still recognized the importance of aligning our efforts with existing healthcare frameworks, such as insurance reimbursement policies.The final design of Virtual Physical encompasses a JavaScript React frontend, a Python Flask backend, and AWS S3 cloud storage, offering a cohesive solution for seamless interaction and data management. For medical technicians, the platform features step-by-step guidance through physical examinations, facilitating data collection and storage for cardiologists' assessment.

With the goal of revolutionizing cardiology healthcare accessibility and efficiency, our team developed Virtual Physical, a robust web platform tailored for the unique user needs of medical technicians and cardiologists. Guided by HIPAA compliance and user-centered design, our project integrates data from 4 medical devices foundational to cardiology, provides guidance for medical technicians completing a comprehensive cardiology physical exam, and enables remote patient evaluations by cardiologists. Our design strategy prioritized user feedback and adherence to healthcare regulations. While out of our scope, we still recognized the importance of aligning our efforts with existing healthcare frameworks, such as insurance reimbursement policies.The final design of Virtual Physical encompasses a JavaScript React frontend, a Python Flask backend, and AWS S3 cloud storage, offering a cohesive solution for seamless interaction and data management. For medical technicians, the platform features step-by-step guidance through physical examinations, facilitating data collection and storage for cardiologists' assessment.

Summary: Successful Creation of Proof-of-Concept Workflow & Web Platform for Efficient Cardiology Care

Physicians have a unique tab compared to the medical technicians, the summary tab. This tab summarizes all information from the physical exam and includes the physician’s notes. It provides a quick overview of the physical exam. Furthermore, abnormal values are marked with an asterisk in red to attract the eye of the cardiologist. To select a different patient chart, physicians can navigate back to the patient search page and select a new patient to view. Patient charts can only be viewed one at a time unless physicians log into the website from another browser/platform.

Physicians have a unique tab compared to the medical technicians, the summary tab. This tab summarizes all information from the physical exam and includes the physician’s notes. It provides a quick overview of the physical exam. Furthermore, abnormal values are marked with an asterisk in red to attract the eye of the cardiologist. To select a different patient chart, physicians can navigate back to the patient search page and select a new patient to view. Patient charts can only be viewed one at a time unless physicians log into the website from another browser/platform.

From the patient search page, physicians can search through stored physical exams by patient name, email, or the date of the exam. Upon search, the page will be populated with a table of relevant stored exams. Physicians can then select which patient chart to view. The unique combination of email, name, and date form the folder in which data about that particular physical exam can be found. This combination is added as an additional claim in the original JWT login token stored in local storage. After selecting a patient chart to view, physicians are directed to the patient chart pages. There are 8 tabs in this section: demographic, general, lungs, pulses, heart, abdomen, legs, and summary. These tabs can be freely navigated to by means of a tab navigation component which contains all of the necessary links. Data is retrieved from the appropriate folder as designated by the stored access token. Prior to rendering the data on the page, the react frontend checks for any abnormal or concerning metrics and displays all abnormalities in red to bring the physician’s attention to them. Physicians can also add notes to sections of the physical exam which will be stored in S3.

From the patient search page, physicians can search through stored physical exams by patient name, email, or the date of the exam. Upon search, the page will be populated with a table of relevant stored exams. Physicians can then select which patient chart to view. The unique combination of email, name, and date form the folder in which data about that particular physical exam can be found. This combination is added as an additional claim in the original JWT login token stored in local storage. After selecting a patient chart to view, physicians are directed to the patient chart pages. There are 8 tabs in this section: demographic, general, lungs, pulses, heart, abdomen, legs, and summary. These tabs can be freely navigated to by means of a tab navigation component which contains all of the necessary links. Data is retrieved from the appropriate folder as designated by the stored access token. Prior to rendering the data on the page, the react frontend checks for any abnormal or concerning metrics and displays all abnormalities in red to bring the physician’s attention to them. Physicians can also add notes to sections of the physical exam which will be stored in S3.

Speciality Physician User Interface

Medical Technician Guidance System Features

In our last cycle, some major milestones were deploying our website to virtual-physical.com and ensuring that it is HIPAA compliant with secure login information. The medical technician guidance system was also reworked for more intuitiveness, by reimagining the position of image references and making sure there was an image reference for each level of the scale, region of the body and detailed instructions. We also included a notes section for every tab in case there was an extreme condition outside of the options provided. There’s now a next input button that scrolls through the page, making the process faster and a save button to allow all the information to remain after navigating away and returning back to the tab. To ensure that all data entered is accurate and reliable, there are warnings and flags on the input if the numerical value is out of range and prevents entering letters for numerical entries. When the medical technician is finished with the physical exam and has input all the required data, there will be a submission page that they can use to confirm all the entries.

In our last cycle, some major milestones were deploying our website to virtual-physical.com and ensuring that it is HIPAA compliant with secure login information. The medical technician guidance system was also reworked for more intuitiveness, by reimagining the position of image references and making sure there was an image reference for each level of the scale, region of the body and detailed instructions. We also included a notes section for every tab in case there was an extreme condition outside of the options provided. There’s now a next input button that scrolls through the page, making the process faster and a save button to allow all the information to remain after navigating away and returning back to the tab. To ensure that all data entered is accurate and reliable, there are warnings and flags on the input if the numerical value is out of range and prevents entering letters for numerical entries. When the medical technician is finished with the physical exam and has input all the required data, there will be a submission page that they can use to confirm all the entries.

In the patient chart pages, there are two main types of files to be uploaded, either json files or any other type of file (pdf, wav, jpeg, mp4). The json files are dictionaries, mapping keys to values and include both text data and any numerical metrics. These dictionaries cannot be uploaded from the mobile platform. For other files, the user can choose to upload it either from mobile platform (after logging in on mobile) or from the laptop platform. For both platforms, the file type is checked depending on the file prompt, if the file is of the correct type then the file is sent to S3 to be stored in the appropriate patient folder. The validity of all medical technician inputs is checked. If the input is not valid, the medical technician will be shown an error prompt. Furthermore, the system checks whether metrics fall within a reasonable range/are normal. If they are abnormal, it is marked in red. User input on the React frontend triggers requests to the Flask backend. This Flask backend comprises a series of API endpoints primarily tasked with retrieving and storing data in the S3 database. Additionally, it interfaces with the Google API for functions such as email transmission and calendar event viewing. Many of these API endpoints necessitate JWT token authentication, ensuring usage only when users are logged in. To facilitate this, the JWT token is stored in the browser's local storage. The cardiologist will be able to view all the input data and notes in the summary page or close inspection.

In the patient chart pages, there are two main types of files to be uploaded, either json files or any other type of file (pdf, wav, jpeg, mp4). The json files are dictionaries, mapping keys to values and include both text data and any numerical metrics. These dictionaries cannot be uploaded from the mobile platform. For other files, the user can choose to upload it either from mobile platform (after logging in on mobile) or from the laptop platform. For both platforms, the file type is checked depending on the file prompt, if the file is of the correct type then the file is sent to S3 to be stored in the appropriate patient folder. The validity of all medical technician inputs is checked. If the input is not valid, the medical technician will be shown an error prompt. Furthermore, the system checks whether metrics fall within a reasonable range/are normal. If they are abnormal, it is marked in red. User input on the React frontend triggers requests to the Flask backend. This Flask backend comprises a series of API endpoints primarily tasked with retrieving and storing data in the S3 database. Additionally, it interfaces with the Google API for functions such as email transmission and calendar event viewing. Many of these API endpoints necessitate JWT token authentication, ensuring usage only when users are logged in. To facilitate this, the JWT token is stored in the browser's local storage. The cardiologist will be able to view all the input data and notes in the summary page or close inspection.

Virtual Physical is a robust HIPAA-compliant web platform designed for two main stakeholders with separate account types and user interfaces (UIs): medical technicians (MTs) and cardiologists. During a patient's physical examination at their nearest clinic, our platform facilitates the comprehensive collection and storage of medical data utilizing four essential medical devices for a cardiologist's examination: a stethoscope, ECG, sphygmomanometer, and pulse oximeter. While this data collection resembles that of established electronic medical record (EMR) systems such as Epic, our solution stands out due to its interactive guidance system. This system aids the medical technician (MT) throughout the cardiology physical exam by providing detailed step-by-step instructions and diagrams for each maneuver. This guidance ensures consistency among MTs, reduces the learning curve for platform usage, and enables any MT to effectively utilize it in a short period.

Virtual Physical is a robust HIPAA-compliant web platform designed for two main stakeholders with separate account types and user interfaces (UIs): medical technicians (MTs) and cardiologists. During a patient's physical examination at their nearest clinic, our platform facilitates the comprehensive collection and storage of medical data utilizing four essential medical devices for a cardiologist's examination: a stethoscope, ECG, sphygmomanometer, and pulse oximeter. While this data collection resembles that of established electronic medical record (EMR) systems such as Epic, our solution stands out due to its interactive guidance system. This system aids the medical technician (MT) throughout the cardiology physical exam by providing detailed step-by-step instructions and diagrams for each maneuver. This guidance ensures consistency among MTs, reduces the learning curve for platform usage, and enables any MT to effectively utilize it in a short period.

Medical Technician User Interface

Our platform is engineered to tackle the intricacies of administering physical examinations, securely storing exam data, and streamlining communication among medical technicians, specialty physicians, and patients. Comprising three primary elements - a JavaScript React frontend, a Python Flask backend, and AWS S3 cloud storage - the platform offers a cohesive solution. The frontend, developed with JavaScript React, delivers user-friendly pages, guiding medical technicians through exams, capturing pertinent patient data, and facilitating seamless interaction with the backend in real-time. On the backend, powered by Python Flask, the platform handles the majority of internal logic and processes, including data processing, analysis, and communication with the storage infrastructure. This ensures efficient management of exam data and enables seamless integration with other components of the platform. The AWS S3 cloud storage plays a crucial role in securely storing all sensitive information related to patients and registered users. This includes patient health records, examination findings, and communication logs between medical professionals. Leveraging the scalability and reliability of AWS S3, the platform ensures data integrity and confidentiality while facilitating easy access for authorized users. Overall, Virtual Physical aims to streamline the physical examination process, enhance data management capabilities, and foster collaboration among healthcare professionals, ultimately improving patient care and outcomes.

Our platform is engineered to tackle the intricacies of administering physical examinations, securely storing exam data, and streamlining communication among medical technicians, specialty physicians, and patients. Comprising three primary elements - a JavaScript React frontend, a Python Flask backend, and AWS S3 cloud storage - the platform offers a cohesive solution. The frontend, developed with JavaScript React, delivers user-friendly pages, guiding medical technicians through exams, capturing pertinent patient data, and facilitating seamless interaction with the backend in real-time. On the backend, powered by Python Flask, the platform handles the majority of internal logic and processes, including data processing, analysis, and communication with the storage infrastructure. This ensures efficient management of exam data and enables seamless integration with other components of the platform. The AWS S3 cloud storage plays a crucial role in securely storing all sensitive information related to patients and registered users. This includes patient health records, examination findings, and communication logs between medical professionals. Leveraging the scalability and reliability of AWS S3, the platform ensures data integrity and confidentiality while facilitating easy access for authorized users. Overall, Virtual Physical aims to streamline the physical examination process, enhance data management capabilities, and foster collaboration among healthcare professionals, ultimately improving patient care and outcomes.

We initiated concept generation through a series of five brainstorming sessions, including individual, sister team, and three group sessions. The most effective brainstorming sessions involved the creation of detailed flowcharts illustrating our software. This approach obviated the need for morph charts and allowed us to make informed decisions regarding the intricate aspects of our platform's user inputs and interface. These decisions were directly influenced by the user needs identified during our research and informational interviews with specialty physicians and sponsors. Subsequently, we progressed to refining and prioritizing our specifications, completing screening matrices, and outlining a clear path for prototyping beyond Cycle 1. Of particular significance were the decisions regarding the features of our user interfaces (UIs) for different user groups, and the protocols for storing, modifying, and retrieving patient data by physicians and medical technicians.To more clearly define the rank of importance of each of these design criteria, the team completed a pairwise comparison chart (PCC) of our specifications as shown above.

We initiated concept generation through a series of five brainstorming sessions, including individual, sister team, and three group sessions. The most effective brainstorming sessions involved the creation of detailed flowcharts illustrating our software. This approach obviated the need for morph charts and allowed us to make informed decisions regarding the intricate aspects of our platform's user inputs and interface. These decisions were directly influenced by the user needs identified during our research and informational interviews with specialty physicians and sponsors. Subsequently, we progressed to refining and prioritizing our specifications, completing screening matrices, and outlining a clear path for prototyping beyond Cycle 1. Of particular significance were the decisions regarding the features of our user interfaces (UIs) for different user groups, and the protocols for storing, modifying, and retrieving patient data by physicians and medical technicians.To more clearly define the rank of importance of each of these design criteria, the team completed a pairwise comparison chart (PCC) of our specifications as shown above.

Heart disease is the number one killer in the United States, responsible for 1 in 5 deaths in 2021. This pressing public health issue is exacerbated by a significant shortage of cardiologists, with only one available for every 14,699 cardiology patients. This imbalance not only results in delayed diagnoses for time-sensitive treatments but also increases the likelihood of preventable complications. The root cause of this strained cardiology healthcare landscape is due to both the numerical scarcity of cardiologists and the high geographical distance between cardiologists and patients.

Heart disease is the number one killer in the United States, responsible for 1 in 5 deaths in 2021. This pressing public health issue is exacerbated by a significant shortage of cardiologists, with only one available for every 14,699 cardiology patients. This imbalance not only results in delayed diagnoses for time-sensitive treatments but also increases the likelihood of preventable complications. The root cause of this strained cardiology healthcare landscape is due to both the numerical scarcity of cardiologists and the high geographical distance between cardiologists and patients.

Current telemedicine solutions offer the benefits of convenience, improved healthcare access (especially for those in rural or underserved areas), and cost savings for patients and physicians. However, telemedicine physicals have key limitations, including their focus on primary care as opposed to specialty medicine, a lack of physical examination by a medical technician rendering virtual exams a non-substitute for in-person examination, technical difficulties, and varied insurance coverage. Traditional telemedicine is effective in fields such as primary care and psychology as subjective symptom descriptions given over a video call suffice to achieve a diagnosis. However, telemedicine application in cardiology necessitates a rethinking of the typical model as cardiology diagnosis relies heavily on objective physiological measurements such as blood pressure and heart sounds. Therefore, the incorporation of a comprehensive physical examination, including specialized maneuvers and essential medical tools such as ECG, digital stethoscope, sphygmomanometer, and pulse oximeter, is an unmet need for effective telecardiology.

Current telemedicine solutions offer the benefits of convenience, improved healthcare access (especially for those in rural or underserved areas), and cost savings for patients and physicians. However, telemedicine physicals have key limitations, including their focus on primary care as opposed to specialty medicine, a lack of physical examination by a medical technician rendering virtual exams a non-substitute for in-person examination, technical difficulties, and varied insurance coverage. Traditional telemedicine is effective in fields such as primary care and psychology as subjective symptom descriptions given over a video call suffice to achieve a diagnosis. However, telemedicine application in cardiology necessitates a rethinking of the typical model as cardiology diagnosis relies heavily on objective physiological measurements such as blood pressure and heart sounds. Therefore, the incorporation of a comprehensive physical examination, including specialized maneuvers and essential medical tools such as ECG, digital stethoscope, sphygmomanometer, and pulse oximeter, is an unmet need for effective telecardiology.

Proposed Solution: Leveraging Medical Technicians

Given that the four essential cardiology tools (ECG, digital stethoscope, sphygmomanometer, and pulse oximeter) are not commonly available at home and require specialized training for accurate usage, an in-person component involving medically trained professionals at local clinics becomes necessary. Thus, our novel and strategic innovation addresses this challenge by delegating cardiology-focused physical exam tasks to qualified medical technicians, including nurses, physician assistants, and nurse practitioners. This delegation not only allows cardiologists to focus their time on diagnosis rather than in-office consultations but also enhances accessibility to cardiology care for millions of patients across the United States, particularly those in rural or underserved areas. This streamlined approach significantly expands the number of patients each cardiologist can attend to while alleviating the burden of long travel times for patients seeking specialized care. To do this, we developed a proof-of-concept web platform that integrates 4 cardiology medical tools to record physical exam data and facilitates subsequent patient-cardiologist virtual consultations, thereby providing convenient access to timely and quality cardiology healthcare, saving lives.

Given that the four essential cardiology tools (ECG, digital stethoscope, sphygmomanometer, and pulse oximeter) are not commonly available at home and require specialized training for accurate usage, an in-person component involving medically trained professionals at local clinics becomes necessary. Thus, our novel and strategic innovation addresses this challenge by delegating cardiology-focused physical exam tasks to qualified medical technicians, including nurses, physician assistants, and nurse practitioners. This delegation not only allows cardiologists to focus their time on diagnosis rather than in-office consultations but also enhances accessibility to cardiology care for millions of patients across the United States, particularly those in rural or underserved areas. This streamlined approach significantly expands the number of patients each cardiologist can attend to while alleviating the burden of long travel times for patients seeking specialized care. To do this, we developed a proof-of-concept web platform that integrates 4 cardiology medical tools to record physical exam data and facilitates subsequent patient-cardiologist virtual consultations, thereby providing convenient access to timely and quality cardiology healthcare, saving lives.

Given that our system operates as a website platform, both inputs and outputs manifest as various forms of data or signals. The inputs can be categorized into four groups: general user input, medical technician user input, specialty physician user input, and patient user input. The medical technician user input has one main category: physical exam data. There are four outputs from our system: registered accounts, patient charts, scheduled virtual calls, and after-visit summaries. The initiation of new patient charts is reserved for cases where patients have not undergone any physical exams through the platform. Upon commencing a medical exam, the platform offers standards and references for accurate measurement techniques. Following this guidance, the technician inputs exam data, which is then validated based on data type and provided prompts. Any user type can create an account and their account details (personal information as well as email and password combination) will be stored in the database along with a marker for user type. Medical technicians and specialty physician users (cardiologists) will be able to view all patient charts stored in the database. Specialty physicians can append comments to charts and flag them for high-risk cases. They can also send links directly to patients to schedule a follow-up virtual consultation at their earliest convenience.

Given that our system operates as a website platform, both inputs and outputs manifest as various forms of data or signals. The inputs can be categorized into four groups: general user input, medical technician user input, specialty physician user input, and patient user input. The medical technician user input has one main category: physical exam data. There are four outputs from our system: registered accounts, patient charts, scheduled virtual calls, and after-visit summaries. The initiation of new patient charts is reserved for cases where patients have not undergone any physical exams through the platform. Upon commencing a medical exam, the platform offers standards and references for accurate measurement techniques. Following this guidance, the technician inputs exam data, which is then validated based on data type and provided prompts. Any user type can create an account and their account details (personal information as well as email and password combination) will be stored in the database along with a marker for user type. Medical technicians and specialty physician users (cardiologists) will be able to view all patient charts stored in the database. Specialty physicians can append comments to charts and flag them for high-risk cases. They can also send links directly to patients to schedule a follow-up virtual consultation at their earliest convenience.

Problem Decomposition

Physical examinations are a key component for cardiologists to accurately diagnose patients and provide subsequent care. Patients are first asked to list their medical history, the prescribed and over-the-counter medications they are currently taking, as well as all the symptoms they are having, when they began, their intensity and duration, if they are triggered by anything, and what the patient has done to make them better. For cardiovascular disease, important indicators include chest pain, palpitations, shortness of breath, syncope, fatigue, and edema. In a physical exam, inspection, palpation, and auscultation of the heart and vessels are conducted, during which the patient rests on a bed. The inspection involves subjectively observing and recording the general state of the patient, the temperature of the skin, the color of the nails, and the appearance of the pulse on the neck.

Physical examinations are a key component for cardiologists to accurately diagnose patients and provide subsequent care. Patients are first asked to list their medical history, the prescribed and over-the-counter medications they are currently taking, as well as all the symptoms they are having, when they began, their intensity and duration, if they are triggered by anything, and what the patient has done to make them better. For cardiovascular disease, important indicators include chest pain, palpitations, shortness of breath, syncope, fatigue, and edema. In a physical exam, inspection, palpation, and auscultation of the heart and vessels are conducted, during which the patient rests on a bed. The inspection involves subjectively observing and recording the general state of the patient, the temperature of the skin, the color of the nails, and the appearance of the pulse on the neck.