Tarleton Gillespie explores the controversy over Twitter Trends and the algorithmic ‘censorship’ of #occupywallstreet.

[…]

The workings of these algorithms are political, an important terrain upon which political battles about visibility are being fought (Grimmelmann 2009). Much like taking over the privately owned Zuccotti Park in Manhattan in order to stage a public protest, more and more of our online public discourse is taking place on private communication platforms like Twitter. These providers offer complex algorithms to manage, curate, and organize these massive networks. But there is a tension between what we understand these algorithms to be, what we need them to be, and what they in fact are. We do not have a sufficient vocabulary for assessing the intervention of these algorithms. We’re not adept at appreciating what it takes to design a tool like Trends – one that appears to effortlessly identify what’s going on, yet also makes distinct and motivated choices. We don’t have a language for the unexpected associations algorithms make, beyond the intention (or even comprehension) of their designers (Ananny 2011). Most importantly, we have not fully recognized how these algorithms attempt to produce representations of the wants or concerns of the public, and as such, run into the classic problem of political representation: who claims to know the mind of the public, and how do they claim to know it?

Can An Algorithm be Wrong?
Tarleton Gillespie in Limn issue 2: Crowds and Clouds.

Also very much worth a read:
Data Reveals that Occupying Twitter Trending Topics is Harder Than It Looks by Gilad Lotan (@gilgul) on the SocialFlow blog, 12 October 2011.

hautepop:

“One company getting buzz is ConnectEDU, sometimes described as an eHarmony for college matchmaking. Its founder, Craig Powell, dreams that students won’t even have to apply to college “because an algorithm will have already told them and the schools where they would fit best,”

The algo dream: to replace human agency and choice.

We must start to ask, what is so bad about making a non-“optimised” decision? Do we not learn things - do we not sometimes learn more - from the things we do that don’t quite work?

[…]

We regret the roads not travelled. Let’s be careful about letting the gospel of “efficiency” too near these choices: rational “optimisation” may potentially be distinctly sub-optimal for a sense of life satisfaction.

mortari:

Agree with this entirely. While optimisation may be very important for decisions in business or resource allocation, it cannot allow for that which we struggle to quantify: life satisfaction. Education does not just convey information or skills, it also fills out our world view by exposing us to new ideas and other people who have different priorities. The astronomy course which I took in my undergraduate studies has been of zero practical use to me in my academic career, but has left me with a true sense of wonder at the possibilities of science.

In the past 10 years I’ve hoped from studying philosophy, to psychiatry, to neuroscience, to psychoanalysis and now to psychology, and I don’t regret any of these choices. This could be seen as inefficient in terms of material learned which was not essential to understand my current field of study. But it has provided an understanding of the much broader context in which my research is situated, and the ability to analyse and incorporate insights from other fields.

Creativity and original thinking requires a degree of flexibility and openness, and in my experience this is best encouraged by a broad and varied education, in which people are exposed to new ideas from different topics. My fear over the optimisation of education is that it encourages the opposite of this: that students focus purely on the subjects which align closely with each other and reinforce each others’ assumptions. Efficiency is just another word for stagnation.

Thanks for the response Georgie - eloquent.

Two more points to flag from the article’s comments take the critique in a different direction - in fact, the socio-economic, slightly Foucauldian lens I tend to use. Worth summarising:

1. The real world doesn’t put you in a meeting or a job interview with people who’ve been pre-selected for optimised learning. Success requires developing the capabilities to handle a challenge and people / situations you might not get on with very well. Avoiding this in college doesn’t do the student a favour.
2. The economic argument: the UK has now joined the US in the vast, near-mortgage-sized debts 18-year-olds are expected to take out to render themselves “employable”. Under this circumstance, it may be more rational to continue on an economics or science degree you don’t like and get a 2:2 than switch to art history, get a 1st, and face much more constrained employment choices.
3. Admin increasing the role for itself in preference to teaching staff; creating technocratic “solutions” to the problem of student performance and uncertainty which don’t address the real problem - don’t make a structural critique of the university and its relationship to capital.

Two more, from further thought and discussion:

• Students already have plenty of data on their own performance - their marks, plus their own self-knowledge. This is enough to make a good decision about what to study. Outsourcing this to a piece of software abnegates responsibility and delays maturity.
• From @megblight: Feedback loops. The new data would over a few years become wholly based on its own predictions. Random effects in initial data would surely be replicated?

The algo dream: to replace human agency and choice.

We must start to ask, what is so bad about making a non-“optimised” decision? Do we not learn things - do we not sometimes learn more - from the things we do that don’t quite work?

This quotation is taken from an article, Colleges Mine Data to Tailor Students’ Experience, where “analytics” is posited as the saviour of everything from college choice, to course choice, to who you work with on a calculus problem.

I offer myself as a counter-example: applied to university to read Philosophy. Realised it didn’t interest me, applied for Social Anthropology and started a course at LSE. Realised I actually needed to find out what I should be doing with Mathematics: left, went to Bristol, started a maths degree. Realised my heart was in Anthro, returned to LSE, finished that degree.

Any salesman with an algorithm would think they could “optimise” that journey. But for me at least, the chance to make those choices on my own and get empirical experience of studying both subjects was essential to be at peace and commit to my final decision.

We regret the roads not travelled. Let’s be careful about letting the gospel of “efficiency” too near these choices: rational “optimisation” may potentially be distinctly sub-optimal for a sense of life satisfaction.

A couple of days ago I was thinking through the question, what’s important now? Or rather, what is now - what are the currents shaping the way the world is going over the next 10 or so years?

Perhaps this was inspired by Jon Henley’s article on September 11th, which argued that it wasn’t actually the “day that changed everything”, and many of the geopolitical events seen as consequences of the attack may have happened regardless.

One key trend is clearly black box algorithms:

In a speech at the technology conference TEDGlobal this summer, computer scientist Kevin Slavin argued that a profound shift is taking place: maths is undergoing a “transition from being something that we extract and derive from the world to something that actually starts to shape it”.

The maths Slavin is talking about, and Harris is writing about, is algorithms. We are, he says, living in an “algo-world”. If Slavin is right, algorithms are shaping everything from the goods we buy to the value of the money with which we buy them.

[…]

The thing is, as systems of algorithms get more complex and take control of ever greater areas of everyday life, concerns are being raised over how much we’re able to track what they’re up to. The answer is: not all that much. As Slavin puts it: “We’re writing these things that we can no longer read.”

[Welcome to the algoworld, Sam Leith, Evening Standard, 12th Sept 2011]

Second, and bigger point - the stagnation and decline of the middle class standard of life.

Take a story that appeared in the Wall Street Journal Monday. The tale is nominally one about marketing strategy and it looks at how giant firm Procter & Gamble sells its household goods to its customers. But the picture that emerges is terrifying. P&G, it transpires, is cutting back on marketing to the disappearing middle classes, instead selling more and more to either high-income or low-income customers and abandoning the middle. Other big firms, like Heinz, are following suit. The piece reveals there is even a word for this strategy, helpfully coined by Citibank: the Consumer Hourglass Theory – because it denotes a society that bulges at the top and bottom and is squeezed in the middle.

The story contains some scary figures, such as the fact that the net worth of the middle fifth of American households has plunged by 26% in the last two years. Or that the income of the median American family, adjusted for inflation, is lower now than in 1998.

Or look at a story in the New York Times Tuesday. It starkly shows how the plight of the American working person has worsened. Solid jobs that once provided a secure grasp on middle class aims (a house, college for the kids, a retirement) have changed to become low-wage ones. It looks at the situation of some Detroit auto-workers, pointing out that new hires can find themselves working opposite long-term colleagues who do similar jobs yet earn twice as much. The system is called a “two tier” wage structure.

Perhaps that system can be justified as an emergency measure to keep Detroit’s auto-industry alive and help it survive the current tough times. But, like the Consumer Hourglass Theory, it actually looks far more like the permanent shape of things to come. American society is bifurcating, squeezing the middle class out of existence. The ranks of the poor and low-income earners are growing and the rich are doing just fine – and no one is talking about it, much less doing anything about it.

[The decline and fall of the American middle class, Paul Harris, Guardian, 13th Sept 2011]

More thinking on these later…