For multiple games I see things like:

Windows Installer v1.0.0
Universal Update v1.0.3

What's the deal? Am I supposed to first install the game and then the update? Why isn't the installer itself updated to install the latest version of the game, leaving the "Universal Update" for those who might have installed from an older version?

Yes. In some instances, the updater may be installing a patch that changes things from their original version, but most of the time I think they just haven't gotten around to it... and likely won't. If anyone's got more details than me, I'm open to them.

For bigger games it's obvious - updating the installer would require people to redownload entire game to have updated version and reinstall the whole game, not just a patch.

There's nothing preventing them from having both (an updated installer and a match patch), though, unless I'm misunderstanding some key point along the way. Indeed, many of the releases have done this.

Universal updates usually are the ones that work on all versions of games. For example a few weeks back IL 2 Sturmovik had a patch released, this patch was compatible with the GOG version and also the disc version of the game.

I know in at least some cases the game is updated as well and the "update" file is there for people who already downloaded the game and don't want to redownload it.

Not sure how you tell which were updated and which were not, though. I always download the updates to be sure.

The version number next to the downloads seems to be the only way of even guessing. If the update's small, of course, there's no harm in downloading and running it as well!