Madgar started off in the spot to the Red's right (or above on the picture). However, he jugged over to join the Dragonborn for a minute or two