My thoughts exactly. I know caine like absolutely craves everyone enjoying his missions but with a larger cast I feel like he might be more lenient on people occasionally sitting one out. 

"If it's ok, can I skip this one? Me and blank were going to check out the lake" 

"Caine, I NEED to sleep" 

Stuff like that, where the characters aren't harping on the adventure or consistently refusing might slide more for Caine than just outright denying it, especially if everyone else is on board. Or maybe he gives them an in house adventure or other situation we just don't focus on, etc. I know Caine is actively overbearing and might not get that in canon, but for our sakes, it would make sense if we occasionally got to skip one or two adventures.