Jon,
Right off hand, you could use tags to accomplish this. Filtering functionality is in the PhotoSmash Extend extension. There is a module called extended navigation that lets you build a navigation block that can have a search box and/or tag drop downs. You can have multiple tag dropdowns that can work together (images have to be tagged with all the selected tags or optionally, only one tag). The search is actually pretty cool...only thing is that it doesn't search custom fields...only tags, captions, contributors, and maybe another field.
To do the jQuery you could build a layout that would let you put a custom field into the class on the div or li or td (whichever you're using to segregate your images...you'd want to hide/show the whole block, not just the image). To do this, you would: div class='[my_custom_field]'
Ideally you could put the tags in there, but I don't think I've got a code for just showing the tags unlinked. I'll add at some point since it makes good since.
My recommendation is to always use tags where possible since you get all the WordPress functionality behind them, in addition to the PhotoSmash functionality.
Once you've got your classes, I think you know how the jQuery would look. That would be pretty cool.
BB