EJ gives up waiting and heads towards the entrance of the house