"But this is exactly what ML is intended to do. ML models are tools for prediction that rely on statistical correlations and "process data that supposedly capture people's behaviours, actions, and the social world, at large."
I don't know if I entirely agree. I don't think all ML models are designed to be tools for prediction, or 'capture people's behaviours, actions, and the social world'. There are plenty of uses for ML that don't involve any of that (generative ML models, summarization, processing large datasets to look for anomalies, generation of novel structures, proteins, techniques, conversation)
None of those prediction or capturing people's behaviours, etc. So, Birhane is right in the sense that use of AI/ML models for such things would be a net-bad and would, in fact, stagnate society. In some respects, that's what @desystemize was on about as well. That is one area where I think law can be effective (using AI for any sort of prediction with legal or similarly significant effects should be prohibited, full stop).
But I do think it's important to remember nuance and that one terrible use of ML is not necessarily indicative of all uses.
"ML systems end up maintaining the very social orders that they were merely designed to model and predict. They reinforce what they observe via one big feedback loop."
This is such an important point. LLMs and generative AI are based on data from the past and cannot easily adapt to new data. Thus they keep reinforcing stereotypes, social orders, mainstream opinions, etc. in an eternal feedback loop to the detriment of culture .
I am currently reading "Filterworld: How Algorithms Flattened Culture" by Kyle Chayka where he makes the same point about recommendation algorithms.
I agree, not all ML use cases are subject to Birhane’s criticisms. Though I think certain use cases (eg. recommendation engines, law enforcement use of facial recognition, and I would even say generative AI since this is producing “new” content based on the learnings from training data which consists of data created by humans) have (the most?) significant impacts on society and so in essence the cumulative effect of these use cases is stagnation.
"But this is exactly what ML is intended to do. ML models are tools for prediction that rely on statistical correlations and "process data that supposedly capture people's behaviours, actions, and the social world, at large."
I don't know if I entirely agree. I don't think all ML models are designed to be tools for prediction, or 'capture people's behaviours, actions, and the social world'. There are plenty of uses for ML that don't involve any of that (generative ML models, summarization, processing large datasets to look for anomalies, generation of novel structures, proteins, techniques, conversation)
None of those prediction or capturing people's behaviours, etc. So, Birhane is right in the sense that use of AI/ML models for such things would be a net-bad and would, in fact, stagnate society. In some respects, that's what @desystemize was on about as well. That is one area where I think law can be effective (using AI for any sort of prediction with legal or similarly significant effects should be prohibited, full stop).
But I do think it's important to remember nuance and that one terrible use of ML is not necessarily indicative of all uses.
"ML systems end up maintaining the very social orders that they were merely designed to model and predict. They reinforce what they observe via one big feedback loop."
This is such an important point. LLMs and generative AI are based on data from the past and cannot easily adapt to new data. Thus they keep reinforcing stereotypes, social orders, mainstream opinions, etc. in an eternal feedback loop to the detriment of culture .
I am currently reading "Filterworld: How Algorithms Flattened Culture" by Kyle Chayka where he makes the same point about recommendation algorithms.
It’s also similar to the point made in this piece: https://open.substack.com/pub/erikhoel/p/here-lies-the-internet-murdered-by?r=1urdan&utm_medium=ios
I agree, not all ML use cases are subject to Birhane’s criticisms. Though I think certain use cases (eg. recommendation engines, law enforcement use of facial recognition, and I would even say generative AI since this is producing “new” content based on the learnings from training data which consists of data created by humans) have (the most?) significant impacts on society and so in essence the cumulative effect of these use cases is stagnation.