Further, by the same logic, there shouldn't be skills for picking up on lies or the like. Players are already capable of trying to figure out when something another character says is a lie based on their own skills. Further, this also restricts you from making characters that are more intelligent than yourself too.

I heavily disagree with the notion of physical abilities (magic, sword play, w/e) having stats and things tied them while other important things don't.

Just because they may not be rolled as often, doesn't mean they shouldn't still be there as a mechanic.