Cumberland House Natural History Museum
(Eastern Parade, Southsea, adjacent to Canoe Lake. Free Admission)
Some examples of taxidermied animals can be found here (upstairs) as well as live fish, crickets, there's also a butterfly house at the back (although they're asleep at the moment).
There are quite a few birds on view, also a fox, mice, rabbit, badger, weasels, seals and insects.