The Doctor was looking at him "Its not easy Jack, because just looking at you... your wrong, A fixed point that was never meant to happen, I can't help it, I'm a time lord, Its instinct."
He knew that was harsh and would probably hurt Jack's feelings, He did feel that way. It wasn't really against who Jack was as a person. He saw the hurt look on Jack's face when he said that.
"Thanks!" Jack remarked bitterly. He looked rather hurt. After awhile he went inside. The Doctor following, It was better to have got such words out then keep them to themselves. He hugged Jack and whispered he was sorry, He saw Jack just cry harder, He felt awkward, but he knew Jack needed a hug.