There's also the part where a number of people were already inactive and hasn't logged in the Oldguild for a long while (some even years). So when the Guild falls and the list refreshed, those inactive people are wiped from the member list.
This is something I have no idea but, if banned member is still counted, then there's that and various alt accounts.