"The Free Market Would Have Taken Care of It"

But it had to exist first. It didn't.

David Bernstein

If segregation and discrimination in the Jim Crow South was simply a matter of law, federal legislation that would have overturned Jim Crow laws would have sufficed. But, in fact, it involved the equivalent of a white supremacist cartel, enforced not just by overt government regulation like segregation laws, but also by the implicit threat of private violence and harassment of anyone who challenged the racist status quo.

There's a simple proof for this: In Greensboro, N.C., Nashville, Tenn., and other places, lunch counters were integrated by agreement between store managers and sit-in organizations after months of pressure. How could that have happened if segregation had been mandated by law? They would have had to lobby for repeal of the relevant Jim Crow statute.

Further, we know that some businesses, particularly those that catered to interstate traffic, wanted to desegregate (bigotry generally is bad business) but feared private/government harassment and worse.

Richard Epstein (same link):

In 1964, every major public accommodation that operated a nationwide business was in favor of being forced to admit minorities.

Wanted to be forced? Sort of. Actually, they wanted others to be forced. In that way they couldn't be singled out for doing what they wanted to do: serve blacks. It's from game theory. X wants to do something, but only if everyone else does it, because if X does it alone, X will suffer some consequence X doesn't like. So X favors everyone's being forced to do it. (Do I have to say that I am not in favor of force?)

The State is the source of many bad things. But it's not the source of all bad things.

I realize that Bernstein and Epstein use these facts to justify Title II of the Civil Rights Act of 1964. I don't follow them there, but they do make the best, most libertarian case for it.