Yeah I think the results turned out pretty well. I indeed went with Euclidean RGB distance, as per your suggestion. I spent some time researching other options before finalizing that bit, but this approach ended up being both the easiest and fastest, so went with that.
I was kinda surprised it worked as well as it did, since in the past other RGB-related comparison operations I've worked with didn't seem to have nearly as nice an effect as in HSV space. I'd be interested in comparing RGB vs HSV in terms of color matching for this purpose, too (weighted, naturally), but I don't really have a good enough collection of sample material to work with, so that would take too long...