In all likelihood, I imagine that affinity summoning is the norm, rather than the exception in this context. Since it wouldn't make much sense to try to force the hand of fate if you aren't like a super fan of a certain specific legendary figure, or have a relationship with them in some way.

Like say, Bazett and Cu.

For people who aren't that close to a specific Servant, letting fate roll the dice s likely the most sane choice.

Which, now that I think about it... did Hinami summon Gil just because she's an old lady? Hmm...