Someone recently wanted me to answer the question which is the title of this post.
I'm sure my answer is based in the training from my sewing, pattern drafting and tailoring background. My preferred garment is one with a set in sleeve. I consider the armhole area to be the most important critical part to be fitted correctly to the body. It is equally important in raglan style garments. All areas of fit are subject to design, ease and personal preference variations that change over time according to fashion. Even the armhole varies greatly according to fashion. Remember the over-sized garments with dropped shoulders that were popular in the 80's?
At this fashion moment we are using a relatively close fitting silhouette so that's going to be the assumption in this post. As an aside, I'm seeing a lot of over-sized coats trending on runways so we may be seeing a shift in the future.
Garments intersect in many places and are interdependent where they don't intersect. As an example, the width of a neckline is limited by the width of the shoulders. It must be a smaller measurement. Seams must be joined and while we can gather or pleat one edge to make it fit to another there are technical limitations to making those joins function and flatter the body. So ultimately each part of a garment has an impact on another cascading down to every section.
In my own experience I have found that the armhole depth measurement has made the biggest visual difference in the fit of my garments. See this post on a method to get an accurate measurement.
The shoulders of a garment act like a hanger. Every garment hangs from the shoulder so I think the fit at the shoulder point is critical. The shoulder edge impacts the armhole. The armhole impacts the length of the sleeve. A deeper armhole means a shorter sleeve from the wrist to the upper arm. The longer the armhole the shorter the body to maintain the same total length to the hem.
Shoulder slope has an impact. If it's ignored you can get bunching or a hem that does not hang straight at the outer edges. See this post for more on shoulder slope.
A too deep armhole creates a sloppy upper torso. It makes your arms look shorter and creates unflattering bunching around the lower part of the armhole just above the bust line. If the armhole does not intersect at the correct point with the shoulder it visually widens the torso and again shortens the arm.
In the photo at the top of the post, my garment is shown on a mannequin that is a little smaller than me and has a slightly more sloped shoulder. It fits me very well. In the photo collage below the shoulders are too wide and the armholes are too deep. Note the gathers that appear at the shoulder line or under the armhole. See how some of the garments fold vertically at the edge of the shoulder line? There is nothing wrong with these garments, it is just a mismatch between the garment size and the model's size.
I think this area is critical because visually it falls into the picture frame that others see when they engage with us and look at our faces. It's what they see when we sit down for a conversation.