It has always amazed me how accurate a fish's sense of what it looks like and its size are. They always seem to know when another fish is bigger than themselves. How ? Also, how do they know who to breed with ? I have little evidence to quote you but suspect that this is done largely by sight for many species.
Also, with territorial species (eg cichlids etc) it is so clear that they are waaaaaay more likely to display territorial behaviour to other fish who have the same shape/colour as them.
Thus I am convinced most fish have a strong identity of their own size, colour and shape. How ? Well thats another question entirely. More and more evidence is coming out in the scientific world about how fish actually do have some intelligence (something any experienced aquarist already knows) so maybe they are smarter than even we give them credit for ?
An interesting perspective is that of Rupert Sheldrake's Morphic Resonance.
Video here
Broadly, the more similar species, sub species and variants are, and the longer they've co-habited in an environment, the more they're capable of resonance and mutuality on the one hand, and threat identification on the other. Doesn't mean they'll interbreed, or even go through the motions of breeding behaviour (though some do), but on another level they're similar enough to co-exist and I suspect a significant proportion of the stress (a sense of 'wrongness' or dissonance) that would be induced by isolation in a prey species, is at least moderated.
Shoaling and schooling fish species are in this sense quite similar to other prey species, at least behaviourally. They'll search out their own specific sub-species for maximum reassurance and to breed, (perhaps we mgiht call it a sense of 'rightness' or harmony,) but - for example - zoologists have discovered fertile hybrids between plains and Grevy's zebra in the wild, so there are exceptions to the rule.
There are lots of herbivores that graze together: various gazelles, zebra, wildebeest etc. They couldn't look more different, but are often found together, except under certain circumstances, such as breeding, migration etc. When one of these species is alerted to the presence of a hunting predator its alarm signals are perceived by other species, because they're similar enough to be on the same menu even if they're visually as different as giraffes and gazelle.
Personally in my current aquarium I have Corydoras Julii, Trilineatus and Schwarzii, (which all look pretty similar, unless you're a Corydoras nut) and also Corydoras Panda. The first three species generally hang out together, while the Pandas stick together. However were I to try (and inevitably fail a few times) to net a Panda this 'alarm' seems to trigger pretty immediate evasive behaviour from all the other corydoras. Meanwhile the guppies chase the net or my hands about the tank generally getting in the way. Is that because they've never co-habited with corydoras? Because it's been decades and thousands of generations that guppies have been captive bred? Maybe they're just thick?
😉