That's what refs and linos are for New - and dare I say it VAR at the top level.
Yeah you are right it isn't always obvious, but it often is as well. When you've got a situation where a defender is facing the wrong way and blocking an attacker with his hands - penalty. If you can see an attacker pulling a defenders shirt, free kick. If they gave what they saw every time it would stop it dead in a few weeks - guaranteed. Football was never meant to be played with the hands. At no point should a player have his hands on another player, it's a foul.