Why is this not a race to the bottom? Is there a reason to think that Gildan has cemented their position and that HBI or Fruit of the Loom will not have an equipment upgrade cycle in the near future that allows them to undercut and/or produce better products than Gildan?
Morningstar rates Gildan "no moat" and your concerns are certainly justified.
On the other hand, Gildan has 70% market share in the printwear market. That clearly wouldn't be possible if Gildan was truly "no moat". As one example, the printers who buy Gildan products are reluctant to switch. Not only does Gildan have a strong brand but different "blanks" react differently to different printing processes. If company B offers you a $0.05 off your blank you would still be reluctant to switch. I don't think Fruit is a worrisome competitor. If Buffett wanted to double-down on this industry, he would have done it earlier when Fruit had the strongest brand.
So it would likely be a foreign competitor, say a manufacture who has scale in China and decides to dump product in the U.S.
But I've learned to be comfortable investing in companies whose moats are not obvious. Equity investment is risk capital. There are always risks. Gildan is the hunter not the hunted, so I think comparisons with BH are wrong. If Gildan stops gaining market share, then I think you should be concerned.
I will say that the scenario of a "scale player" popping up overnight is not likely. Just look at the history of Gildan. It was a slow methodical process. A bigger short-term concern would be a trade war, cotton shortage, or coup.