The same applies to women's sport. If female athletes believe that competing against some transgender athletes creates an unfair competitive advantage, then their concerns should also be taken seriously. I don't see why it's unreasonable to explore alternatives, such as creating a separate category for trans athletes where practical.
As things stand, many female athletes say they don't feel able to express their views publicly for fear of backlash, including the potential loss of sponsorships or damage to their careers. Whatever someone's position on the issue, surely we can agree that it's unhealthy if athletes feel unable to voice their opinions on a matter that directly affects their profession and livelihood?