Left to 'ourselves' I'm sure many states and municipalities would also vote to ban interracial marriage... or other 'activities' the majority didn't particularly approve of, but that are (or should be) protected as individual liberties.
Thankfully, we have a constitution that guarantees equal protection -- or at least it says that.