Not unless you and your brother both stand under the direct command of the same entity. But if you do, then totally yes.
If Sleazy Inc owns companies A and B, then yes, A doing something evil does reflect poorly on B, and it makes total sense to (for example) consider B without moral standing, based on the actions of A.
It makes sense because the same single board controls both A and B.
If your left hand engages in aggression towards me, I'm totally going to consider your right hand a potential threat: because I'm aware that both of your arms are controlled by the same entity.