my Academic Projects
largely pertaining to machine learning and artificial intelligence
HARRY POTTER'S VANISHING SPELL
Combining Object Dectection/Masking with Image Inpainting and a NLP model provides the eerie effect of objects vanishing when a magic word is said!
First, a Google word detection algorithm is applied to the audio which detects when the word "abracadabra" is said. The next word said is the passed to the rest of the system as the object to be masked. Then MaskRCNN, an Object Segmentation neural network, looks through the frames and picks out any instances of the target object. Finally, an Image Inpainting neural network attempts to fill in the section containing the segmented object with a plausible background.
The end result is a video where it appears the user is making objects vanish from sight just by uttering some magic words!
DEEP LEARNING FOR VIRAL IDENTIFICATION
Metagenomic viral identification is a critical field of research due to the prevalence of viruses in the biosphere. In recent years, deep neural networks have begun to outperform traditional reference-dependent methods on this task. In this work, Jackson Lightfoot and I survey developments in the field over the last decade and conduct an evaluation of a state of the art model, DeepVirFinder
We conclude that although deep learning has made profound advancements to metagenomic viral identification, quality dataset composition is a hugely important factor for progressing the field to ensure generalizability to novel viruses and usability in real world applications
EYE DISEASE RECOGNITION USING DEEP LEARNING
This project sought to predict the presence of a disease and categorize it by applying deep convolutional neural networks to images of the eye. Using fundus (inside) images of the eye, we were able to we were able to accurately categorize whether an eye was healthy or had one of six eye related diseases:
| • Myopia | • Glaucoma |
| • Cataracts | • Age-related macular degeneration |
| • Diabetic retinopathy | • Hypertensive retinopathy |
After using several popular convolutional neural networks including VGG19, AlexNet, and ResNet50, we trained models and made predictions. When classifying each disease individually, we achieved an accuracy of 80-95% depending on the disease. In a multiclass setting, we achieved an overall accuracy of 87%.
MEDICATION-ADHERENCE TRACKING
smart PILL BOTTLE
We sought to address the problem of medical non-adherence, both among seniors and those with mental illness. To this end, we attached an inertial sensor along with a number of cameras to a pill bottle. We then had 5 participants take 10 pills each. After collecting data from all devices, a number of machine learning technique were applied to predict whether or not a pill had been taken. Using a Random Forest Classifier and a Convolutional Neural Network, we were able to reach an accuracy of 95% and 94% on those models respectively. We also employed a state-of-the-art real time object recognition system to detect both the hands and face of anyone using the pill bottle. With the assumption that these two objects overlapping indicates a pill-taking moment, we were able to achieve an accuracy of 80% when classifying whether or not a pill was taken. Finally, we propose a system by which these models could be integrated to solve the problem of non-adherence and discuss the limitations of our work.
