Some Knitters don't wear their best work because they are afraid that it is not fashionable. Fortunately lots of my friends are Knitters so my knitting does get worn out in public. When I worked at my LYS it was easy, I wore a sweater or shawl every day and I got a total kick out of all those knitters coming in and asking about whatever I happened to have on.
I think that what is going on is due to the judgments that we all make about others. We know that the knitting that we like to think of as special and unique might be viewed by others as just a little on the other side of "weird". We think people may stay away from the crazy lady with pointy sticks as weapons clutched in her hands.
So what's the solution? I think we need to consider context, where we are when we choose to be seen in our knitting and also the garments we choose to wear with it. I often choose yarns knowing exactly what other garments the end product will be worn with. I usually pick simple conservative pieces as underpinnings and I think carefully about the colour matchings. I also tend to be a little overdressed (as my husband constantly points out), so I think that might keep me out of Weirdsville.We Knitters are a special subculture and I think we need to remember that. The garments we produce are special and that's the reason we don't see more of them in main stream fashion. The reason is that they are special is that no one can produce what we do in a cost efficient way. What we are crafting is done as a labour of love. If you can find a comparable hand knit like this http://www.pringlescotland.com/ or this Oscar pictured below, priced at $1,690.00, it's on the fashion runways not at our local ready to wear stores. So rejoice, savor your unique skills and wear your knitting proudly.