If I set a custom window resolution such as 800x600 and am using aspect ratio correction, is that worse than, for example, using "original" window resolution and a normal3x scaler? Do I get stretched pixels with the former vs the latter or anything like that?

Look here: http://www.dosbox.com/wiki/Scaler
Got all you need to know about resolutions and scaling with images for example. You could look here as well: [url=http://www.dosbox.com/wiki/Configuration:RENDER]http://www.dosbox.com/wiki/Configuration:RENDER[/url]
Got some info on how the render effects image quality and aspect ratio.

It gets a bit complicated with render and all, but you'll probably get less distorted pixels with aspect=true and scaler=normal3x.

Aspect ratio correction doesn't do what you probably think it does. Many games from the DOS era are in 320x200, the pixels of which aren't square (are taller than they are wide). aspect=true will just try to simulate this. It has no effect in square pixel modes. It does mean some lines are taller than others. Adding normal3x will make the difference smaller and less noticeable. Scaling all the way to 1600x1200 would give the best results but that's a bit much.

The renderer may still affect things and in some circumstances there are reasons for not using aspect=true, but unless you like your game characters short and chubby, you will want the same end result produced by something.

With a custom window resolution, your game will always use the original aspect ratio (usually 4:3). DOSBox will resize the window to the closest fit (by changing the width or height). The scaling is done by the video system, so make sure a scaleable output is set in DOSBox, like ddraw, opengl or openglnb (output=surface won't allow scaling the game to your window size). output=openglnb will give you crisp pixels, while the other video systems usually blur the edges.

Using DOSBox' scalers doesn't have any real advantage, except that scaling exactly x times the original resolution with normal mode (e.g. normal3x) keeps the pixels crisp and clear and avoids blurry edges (a good thing for old games in my opinion), but output=openglnb does the same and is usually hardware accelerated. If you want smooth edges, then it's best to use one of the various DOSBox scalers available.

Long story short, no matter what method you use, there is no pixel stretching. Aspect=true forces non 4:3 games into 4:3, like Rixasha said above. It's all about smooth edges versus sharp pixelated graphics, whichever you like best.