I see the points your making but I don't think it's this simple. We still barely understand anything about the gut microbiome, in terms of what's "optimal" and it probably varies wildly depending on the individual.
I don't know if I buy the idea that we need to eat a super large variety of different foods. How do you explain people who go on a carnivore diet and seemingly cure all their GI and immune problems?
They have greatly increased their understanding of it over the last decade and have found repeatedly that diversity is a key component...which is created by eating a wide variety of foods...
That's simple, and I've talked about it many times. There are foods that are considered "healthy" that don't work with individual people's body chemistry and cause low grade chronic inflammation and reactivity. Some of the foods that are almost always considered "healthy" are high on this list and include concentrated dairy(whey, Greek yogurt, etc), salmon and even oatmeal.
If people eat those foods regularly and they cause inflammation, it's no wonder if they stop eating them it helps their issues go away.
But that doesn't mean they should stop eating all other foods, they simply need to do more to understand what works with their body and what doesn't.