The brainworms will come for us all
Thanks for the link. I've been chewing on this problem since at least early 2018 and I think there's simply no way to fight it, other than to destroy the internet. Best I can come up with is to develop and propagate a new religion with a "thou shalt not use smart phones" commandment in it somewhere.
And further, I cannot see how this problem won't get worse, because of the feedback loops you're describing. Even if machine learning were not applied to the problem, human learning does a bang up job of creating it on it's own. See: reddit moderation practices. It's not just the algorithms. At it's root, this is just the ordinary mechanic of a culture interpreting its reality (as has been done for thousands of years) amplified and bifurcated and accelerated by a time factor of thousands. We've kind of always been doing this, but never this fast or this fractured.
As with much of my material, the solution as best as I can see is not to try and avoid Very Bad Things, it's to buy bullets and find some place to hide out for when the thing pops. Print out all those The Prepared articles and put them in a hardcopy binder.
This and another post you wrote about this gamification i really want you to be wrong. Its a pretty bleak vision. I hope that Cory Doctorow is right and that ad tech is actually not very effective. On the one hand there is evidence that many are using ad tech which makes one think it must be valuable, on the other hand Doctorow does seem to list some case studies that show it is not. I'm hoping he is right.
I think you could make the same argument more simply, without the hidden advertising algorithms. Any writer or politician can do something like this, if they come up with something the audience really likes. You don't need fancy algorithms, you just need to get people to subscribe to your email list or show up at your rallies. Or go to your church. The people who like that sort of thing clap for what they like and the performer learns from their audience what to do to please them.
It seems to me that there are two opposing incentives for marketers here: growth and retention. The companies that benefit from Wyrmism don't necessarily gain from market fragmentation per se -- they want everyone to convert! So there is a tension between the incentive to "mainstream" Wyrmism and make it palatable to everyone and the incentive to make Wyrmism more engaging to retain existing converts, which would often involve making it more distinctive/extreme/isolating. I wonder what factors make one effect dominate.