When you are with a group of friends, you feel you can talk about anything, right? You share the same views on most subjects, so there is no dangerous territory. Nobody will become upset because everybody implicitly agrees. It can be different if you are the odd man out, though.
Suppose you are the only person at the dinner table that favors, say, securing permits and inspections before building or remodeling. Someone else deliberately starts a conversation on that subject, and it quickly becomes a multi-person rant about the evils of county regulations and building inspectors. What do you do? If you defend your viewpoint, you will start an argument. You will also become the target for three other people who want to vent their negative experiences.
You could get up from the table and walk away, but that is almost like demanding to be let out of the car while your brother lectures you on all the mistakes you have made in your life, while he is driving down the Interstate highway in the middle of the night. The pain of enduring the “conversation” is probably less than making an escape, so you sit it out, watching the time, waiting for the torture to end. Eventually it does end, with nobody the wiser, everybody secure in their personal bias, with essentially no communication or learning having taken place.
In our increasingly polarized society, we seek out homogeneous groups that align with our own world view. We feel secure in most topics of conversation because we know the other person will share our bias. We can talk about politics and religion because we all know each other to be liberal atheists, or conservative fundamentalists. All ideas slide smoothly down the pipe. If someone raises an objection, dares to differ, suddenly there is an obstruction and the flow backs up, and can get nasty.
The polite thing to do in such a situation when you are the odd man out is to remain silent. As unpleasant as it is for you, it would be worse if you created an argument or made a dramatic exit. That could make four other people uncomfortable. Your own intense discomfort may be four times the magnitude of theirs, but it is still localized and contained, within your own tense chest muscles and pounding head.
Such are the perils of not belonging to an ideological group nowadays. Outsiders don't seem to fit into any group, and constantly encounter clashing viewpoints. It is hard to mix when people gravitate to hot-button topics in social gatherings. You know you will spoil your welcome if you express yourself in public.
What can you do? You can look for people like you and band together, forming another tribal enclave, or you can become even more isolated and excluded. There seems to be no middle ground anymore. One third possibility, however, is forming a band of social misfits, but that, my friend, is an oxymoron.