Because it's the most high profile thing bearing the label libertarian, which in the USA ought to mean the political philosophy of Mises, Rothbard and Hoppe.
The LP should be seeking allies in, and defending the interests of, all sources of civil order outside the state.
Religion, fraternal societies, charities and businesses that lack government ties.
The party is instead defending interests similar to the Democrats: pushing for cheap labor via open borders, defending high time preference activities from any checks imposed by the state.
Sarwark inconsistently pays lip service to property rights while bowing down before the social justice altar. By acting this way the party scrambles the signal, jamming the real libertarians.
I'm not proposing a purge, but if the libertines and squishy Republicans all took up fishing instead of activism the party would be 3000% more effective.
The real question isn't "why aren't you a member?" It's "why does the party bearing the name libertarian just so happen to consistently alienate every libertarian that reads books?"