"Officiate the defense" also means primarily watching the defender movements. You must watch the defense before judging whether the offensive player or defensive player committed a foul.

Obviously, "officiate the defense" doesn't mean you can watch only the defender. You must also watch the offensive player for violations and fouls. By maintaining good angles and establishing proper spacing, you can watch both players.

Think like the offense
Former players and students of the game usually make good officials. Why? They have learned as players to recognize defenses and they know what to do to beat those defenses. Good officials do the same thing. Once you've learned to recognize defenses and understand defensive tendencies, you must think about what the offense is likely to do and adjust accordingly. Gaining that knowledge allows you to anticipate correctly and move to get proper angles and spacing.

The more you know about the game, the less chance you have of getting surprised. Studying the rules and mechanics isn't enough. A complete official knows what's going on from the 'players,' 'coaches' and 'officials' perspectives.