Well mainly they're all holed up in Cracked Tusk Keep in Falkreath, they don't often leave that one hold, and never in very large numbers. But when Brannus ventures out himself beyond the walls, he'll generally take one or two men with him, one never can be too careful out in the wilds.