The human inside the cave was scared and twice as nervous. The tall tree-like beings were the ones that scared him off from the Pond of Jewels...with a jewel in hand. It was against the forest law to steal or tip the nature off balance. Taking a jewel from the Pond of Jewels was one of them. The boy in the cave didn't believe what Loki was saying. He doesn't trust them enough to just come out and see them attack him. 

"No! They mean to kill me. And besides, they're hideous to look at", the boy from the cave responded as he didn't move an inch in te cave nor did he come out.