Unless a baby brain is too undeveloped for an adult to properly possess. Like I know they have trouble seeing and stuff so he might be plagued by the same issues unless possessing significantly alters the brain.