Yeah, the one down by the ocean works for the fisherman. 
The inn just needs to be near everything else. Apart from that, it doesn't really matter where you put it.