Pokedex ML - Object Detection
A personal project building a real-time Pokémon object detection model from scratch utilizing computer vision, custom dataset creation, and neural network training on consumer hardware. A fun way to strengthen my Machine Learning Knowledge.
Tools & Stack: Python 3.10 · YOLOv8 · CUDA · LabelImg
The project started as a binary image classifier running on an Arduino Nano 33 BLE Sense, then pivoted to a full YOLOv8 architecture after hitting the ceiling of what a microcontroller-class model could do. Training runs on a Windows machine with an RTX 4080 using a custom Anaconda environment. The first trained model targeted Smoochum as a single-class proof of concept, achieving strong precision and recall. Smoochums was picked due to its similarity to other cute and pink objects the model could easily confuse. It also happens to be my girlfriends favorite Pokemon.
Project Origin
Dataset construction is entirely manual. Attempts to download preconstructed data set resulted in poor training. The adage garbage in garbage out is never more relevant than ML training. Images are sourced and annotated by hand using LabelImg, with bounding boxes drawn for each class. This keeps the training data clean and consistent, but is slow.
Dataset Generation
Original Smoochums identified data set
Original NOT Smoochums identified data set to strengthen model correctness
To validate the model in real time I ran the ML on my laptop using the webcam as a life feed and having it put a bounding box around any object it believed to be Smoochum, as well as its live confidence reading. Model V1 had too low of a confidence level before putting a bounding box, and the “NOT Smoochums” data had not been properly implemented so a false positive was the norm. This was not a problem for Madelyn who enjoyed when we were “both Smoochums”.
Refinement of confidence threshold and correct implementation of false trigger information resulted in a consistent Smoochum Detector.
Model Testing and Iterations
My very first ML model running session, V1 too sensitive
Version 2 with corrected training and detection threshold
Future Work
The longer-term goal is a working Pokedex replica: point a camera at a Pokemon (card, figure, or screen), get an ID back in real time.
Potential future products include:
A themed pokedex which only identifies a persons favorite pokemon, aesthetically based upon said character.
Leaning into the trading card game aspect and identifying cards along with their current market value.