PathSense
Freshmen URD Track; Entrepreneurship and Innovation
There are more than 43 million people with completely blind vision and more than 295 million others living with visual impairment. Since vision impairment creates more obstacles for the individual who experiences it we want to help them by improving their quality of life.
Since commuting and transportation are some of the most important aspects of daily life. Being vision-impaired creates a much harder and time-consuming process for the individual. The cane or guide dog is one of the most effective and common options out there at the moment. But the cane and guide dogs have their limitations which is the distance of the area that they cover. Thus, we want to create a device that tries to push that limit of distance.
Project Solution Approach
Our approach involves developing an environmental sensing system for visually impaired individuals. We have designed a unique bag equipped with a camera on its strap to record the user’s path in real time. The captured images are then processed by an artificial intelligence system to determine whether the path is safe for the user to traverse. The results of this analysis are communicated to the users through their chosen audio output.
Project Objectives
One of the most important deliverables for this project is Path Analyzing AI, or developing an AI model that can interpret visual data, identify obstacles, and understand surroundings for real-time navigation assistance. Another objective is portability by designing a lightweight and compact product that is comfortable to wear and easy to carry, thus enhancing the user's mobility. Finally, the product should also be affordable so that more affected people can have access to the technology.
Methodology
The first step made in designing the UI was to decide on a color template that considers the importance of light perception. To achieve this, the colors used must be contrasting enough with a combination of, based on the Ecological Society of America, darker text on a white background as the most accessible combination.
Creating Training Data
When creating the training data, we have 2 objectives that we want to achieve. The quantity of the training data must be sufficient and the variety of conditions the training data must also be robust to allow for accurate predictions under different conditions. The first objective was met to a certain extent with a total of 4000 images that we managed to collect around the CMKL campus that we then used to train the model.
Results
There were 3 components to this project when we worked on it, the AI model hosted in the cloud, the physical cross-body bag with all the hardware components implemented, and the mobile application that will allow the user to connect to the cross-body bag.
The AI model was successfully implemented by building a model using the PyTorch library in Python with the AI model taking the form of the CNN architecture which is well suited for our use case. The model is successfully hosted in the Google Cloud Run service where our microcontroller in the cross-body bag can reach. The AI has been trained with 4000 images that have been augmented to increase the robustness of the training data such as brightness, rotation, etc. The physical product was successfully implemented with 3 main components, the camera module, microcontroller, and power supply.
Overall, all of our objectives were met. The AI model boasts a decent accuracy considering the limited variety of data we could collect, this however needs to be improved in the following semester to encompass other locations. The portability of the device is addressed with the bag only having an additional microcontroller, camera module, and power supply which doesn’t significantly increase the weight. Finally, by using a cheaper alternative to the Raspberry Pi microcontroller, OrangePi. We can significantly cut the cost of the physical product.
Summary of Accomplishments
The successful outcomes of our project started with the completion of the initial data collection and data labeling, allowing us to have a working AI model that is hosted on a cloud server with an authentication and authorization system, hardware CAD design, and complete assembly of the hardware components. Overall, these accomplishments allowed us to develop a functioning mobile application and working product prototype.
Future Directions
In the next upcoming semester, we aim to extend our project and add additional useful capabilities to the product. Firstly, we plan to collect more data and increase model accuracy to ensure its effectiveness in various locations. Secondly, we would like to ensure images are discarded immediately after the analysis and implement robust security measures to prevent unauthorized retrieval of the images as part of improving security and privacy procedures.
