It's common knowledge. He made a point to brag about it. Part of the reason why Billie and Alex hate him. They don't think that that is something to flaunt. Especially since they try to keep under wraps who their father is.