To be fair, Crow's only impression of Gavin was when he followed Penelope out to the tavern without her knowing xD
He didn't come off like the most trustworthy person at that time

[s]Also, the thing with Rikki is still fresh in his mind and has him on edge[/s]