I must agree - build, height, et cetera can have significant impact, psychologically and in practice. My sheets always contain height, even when not asked, and although I probably would not put exact weight, build will be there. And I will probably also note whether the character's hair is just an inch long or reaches back to their tailbone, or whether they are pale as snow or near-black, or anything in between... 

Physical effects left aside, it all helps to visualize the scene. I always visualize the scenes I am writing, and the more I have on appearance, the easier it is to do. Everything else is either filled in at random or just left at the "vaguely anthropomorphic blob" stage. 

Just imagine two guys: one is scrawny and stands at 5'3'' or so, and the other is 7'1'' and rather strongly built. Perfectly realistic difference. Knowing noting else about them, and other things about their appearance being similar, would you approach them in the same way? Or would you be intimidated by the latter more than the former if it is just the two of you? ("He was well over two meters tall and 125 kilograms of pure muscle" is most likely enough to conclude that you probably do not want to get into a physical fight with that guy. Heck, I've heard such physical attributes noted often enough in casual descriptions by other people...) One could also imagine that the latter may have to watch out for lower doorframes, whereas the former may need a chair to get that box from the upper shelf. And so on and so forth.