The thing is, it's not a "few weird edge cases". But this seems like an odd "problem" to me anyway - I must admit I've never been in the situation of having to trudge through vast numbers of definitively fantasy books to find SF books anywhere...
The majority are really not that hard to categorise.
In the UK at least, fantasy and sci fi occupy the same shelving. Takes me ages pulling books out of the shelf, and immediately rejecting because they are fantasy.
The majority of the books are fantasy, not sci fi. Fantasy seems to have a much bigger audience in the UK anyway.