Modern knitters tend to put a high priority on the softness of the yarns we choose to work with. I believe it is up to every knitter to decide for themselves which yarns they work with. I choose yarns for a variety of reasons related to cost, colour, texture, intended use and wear-ability. Every yarn has a place in the knitting world. It is up to us to decide if the yarn aligns with our needs.
Often the best way to determine if the yarn is worthy of our time and effort is a pros and cons list.
I'd like to address an interesting pro of hard-wearing yarns. Most knitters know a yarn in this category. Usually 100% wool, it feels a little rough to the touch and is often used for garments that are not worn next to the skin. These yarns are tightly spun and provide excellent stitch definition. Items made from these yarns last for years and rarely pill. They create a somewhat stiff fabric which is warm, usually worn in the winter or on chilly days without a coat and with a soft layer of clothing next to the skin. The stiffness of that fabric has an additional benefit in a well fitting garment. The fabric tends to skim past the body and can be surprisingly slimming, in that it does not cling to the jiggly bits that some of us wish to disguise. If you would like to confirm this for yourself try on and compare two garments that have positive ease of about 2 - 4 inches. One should be in a soft yarn, perhaps a superwash or an alpaca or cashmere yarn. The other should have a fabric hand that is much stiffer.
I've included photos of two of my sweaters, the grey one is a worsted weight wool, the burgundy is a superwash DK. Photos are static, they give the knitter no information about the fabric being created. I suggest that you actively work towards understanding how a particular knitted fabric interacts with your body and the appearance you wish to achieve. Both garments are the same size and include waist shaping but only the burgundy one clings. It does that even when I add the same oxford shirt under it that I wear under the grey sweater.