This year, the public narrative around Facebook has switched – the company feels on the defensive in lots of ways. I think it deserves to be – with billions of users, it is long past time for them to spend their energy on reducing harm, rather than on more growth.

There’s a bit less talk about YouTube (owned by Google), and the problems with its recommendation algorithm.

The problems

Here are an article and a video which show the span of problems – from causing political radicalisation in every direction, to creating vast farms of weird, abusive videos targetted at children:

1. YouTube, The Great Radicalizer by the excellent Zeynep Tufekci.
2. The nightmare videos of childrens’ YouTube by the great James Bridle.

This is causing pain for content creators too. For example, a board games reviewer I like called Actulol gives some idea of the mental health issues caused by the algorithm in Why Actualol Went Quiet. You can find other examples – ask YouTubers what they think of the recommendation algorithm.

Why it happens

I wrote a Quora answer last year on How does YouTube’s recommenation algorithm work. If you’re technical, definitely read the full paper from Google.

I find two basic problems with the algorithm:

1. Populist. It first of all uses a crude criteria to find videos watched by people similar to you. This means it is pre-filtering for the popular. This is the opposite of what has made Google search a success – where some users mine dozens of pages into search results, and Google uses that signal to slowly increase the rank of good sites. (We watched it find PDFTables and rank it highly by those means).
2. Short-term. After getting those few hundred candidates, it then uses a proper super smart neural network algorithm with thousands of factors fed in to rank the videos based on which you’re most likely to watch for longest. This is extremely simplistic – the idea that the best videos for you to see are the ones you’ll watch the most. Naturally, things that appal or deceive, or that make you unhappy in the long term, will bubble to the top.

Other ideas

It seems that even at a basic level, Deep Mind (the division of Google’s parent company which made the algorithm, and whose offices are but a mile from where I sit now in London), could come up with a better criteria for the algorithm.

As well as a better criteria, Google would really have to want to use it – YouTube is vast in both size and speed of change, making it hard to run whatever algorithms the company wants over the corpus. That feels like an excuse though – they manage to do similar scale work in search. It feels possible, it just needs the budget allocating, so in turn needs the pressure from us.

At Newspeak House the other evening, the topic drifted to ideas for improving the recommendation algorithm. We were just coming up with criteria to train it – I’m sure the boffins at Deep Mind can come up with better ideas, and more interesting technical implementations of them. Here are two of them:

• Feel happiest in 6 months time. It could explicitly ask you – do you feel happy / does using YouTube make you feel good or bad? And train the algorithm on whether that signal improves months later. As well as improving videos, this would be good for the long term brand of YouTube.
• Become higher value to advertisers in the next year. This is interesting, as it sounds like it could make more money for Google. It would naturally tend to push aspirational videos – or videos that lead to aspirational videos – onto people. So they are more likely to get promoted, get a better job, want to buy more expensive things that are advertised on YouTube. It’s not clear this would be good overall but it would be interesting.

Some more radical ideas, going beyond just tweaking the training criteria:

• Get rid of automated recommendations. Instead, they could be curated, perhaps by the person who makes each video. For a more AI based version of this, something like the Spotify model based on the crowd-curation of playlists could help. We used to watch TV stations curated by amazing people like David Attenborough (director of BBC2 in the 1960s). Could YouTube help me do that in a more modern way? If things get bad enough, we could regulate to simply ban robot recommendations outright, and see what innovation human curation leads to.
• Agent on own phone I own and tweak to make recommendations. This is perhaps too demanding on the user, it feels like the way things are going though. The work done on TensorFlow Light to get the clipboard AI to run on Android phones shows that with the right engineering this kind of solution is possible. The end game is like the movie Her. If the customisation is too hard I could follow someone else’s meta-ruleset – maybe a famous brain training coach.
• Split up YouTube. Create competition. Right now I have no choice of recommendation algorithm. All the videos are on YouTube, so I have to go there. I guess I could learn Chinese and move to China and find a different system, but that’s the limit of my choice. How to split up, or regulate the new generation of big tech companies isn’t clear yet – we could find a smart way to do it well.

What do you think, how would you improve YouTube’s recommendation algorithm? What would a smarter criteria be?

Zenrainman’s house is like a tree.

Grown out of the earth of its own basement on the edge of Bangalore, pressed into bricks. 

Nurtured entirely by water and light from its own roof. 

It breeds, symbiotically, large primates to maintain it. 

Treating their sewage. 

To water a rooftop rice paddy ready to grow enough to feed them. 

The spaces are full of art and light. 

Architecture students come and go, borrowing books.

They’re a means of reproduction, working nearby to design hundreds of similar houses.

The press call to ask a question on water. Bangalore is short of it, a chance at last for change. Sustainable water use. 

An itinerant British computer programmer appears for dinner.

They talk about technology and politics, about when opening data harms the common man, about dark mountains, about stamps, about China, about villages which aren’t on Google maps, about wasted tomato harvests, about building a helpful information society. 

All find Zenrainman in his chair. 

The house/tree attracting the world to one room like flowers to bees.

Just as our cells and our bacteria are equal in number… Is the organism the house or the rice or the human?

I imagine a million years passing, the methods maturing, the earth scattered with these strange complex constructs. 

An alien passes. What are these novel trees? 

I like John Redwood. I started reading his blog in 2006 while I was involved in the Save Parliament campaign, trying to stop the Government pass a Bill whose craziness you’ll have to read about by following the link. John spoke prominently on the Internet and in Parliament against the Bill.

Since then, I’ve seen eye to eye with him on issues such as David Davis’s resignation over civil liberties and the lack of quality in Parliament’s law making process.

But for some time, I’ve been distressed by his view on climate change. Unlike David Cameron, John Redwood seems to be hiding from reality. Rather than accepting our predicament, and using his other political principles to work out how to fix it, he uses weasel words to avoid saying whether he believes or denies.

For a while I accepted this, and tried to point him to evidence he might listen to in the comments of his blog. For example, Confederation of British Industry reports on climate change (“Our changing climate is a threat to the way we live and work. Building a low-carbon economy requires government, businesses and consumers to work together, but we are not doing enough quickly enough.”).

Finally I was tipped over the edge, and challenged him to say whether he thought climate change was happening or not. It seems others have asked him the same thing, as he replied in full.

Here’s the core of John Redwood’s reply:

The warmists and their Ministers need to set out in detail their case to the public. They need to show that

1. The world is warming. Some temperature series show no warming in the last decade, and a cooler period after the war until the 1970s.
2. That warming comes from rising CO2 levels
3. That past periods of warming prior to industrialisation in both historical and geological time were caused by processes and events that do not apply today
4. That the man made element of increasing CO2 is the bit that matters and will cause unacceptable warming
5. That it makes more sense to try to stop the CO2 increases and the warming, than to invest in ways of handling the adverse consequences
6. That taxing and regulating is a better way to change human behaviour than incentives and technology

Even in the full article, John Redwood still doesn’t set out his view. He doesn’t say what he thinks on any of points 1 – 6, or why. I think that he is a climate change denier, but that he isn’t very confident about it. He doesn’t want to admit it in public, then find out he was wrong later in life.
No matter, his old blog posts are still a smoking gun, that will show that he was indecisive, that he didn’t lead when it mattered.

Yes, John is correct that warmists need to show 1 – 6 above. They’ve done so in numerous scientific papers and fat books from the IPCC, as well as evidence you can see with your own eyes. However, it is also the case that denialists need to show the opposite:

Actually John, the denialists and their supports in the old energy companies need to set out in detail their case to the public. They need to show that:

1. The world is not warming. When the north sea is melting, glaciers are retreating.
2. That warming comes from somewhere other than rising CO2 levels
3. That past periods of warming temperature prior to industrialisation in both historical and geological time were caused by processes and events that apply today
4. That the non-man made element of increasing CO2 is the bit that matters and is causing unacceptable warming
5. That it makes more sense to invest in ways of handling the adverse consequences than to try to stop the CO2 increases and the warming
6. That “taxing and regulating” is what is being proposed, when it isn’t, it is creating incentives for the market to deploy existing technologies and create new ones.

I’ll give John that the warmists have to prove all of their 1 – 6 are true, whereas the denialists only have to prove one of theirs is true. However, the denialists also have to show:

b. That allowing for the growth of China and India, and the fundamental physical limit of easily accessible fossil fuel reserves, we will be able to continue using existing sources of fuel indefinitely, at cheap prices.
c. That in an unstable world, where Russia have cut of gas supplies to customers, and there is terrorism, we can continue to rely entirely on a complex supply chain of imported energy every winter.

The burden of proof is just as much on John’s shoulders to show we shouldn’t act to reduce carbon emissions, as mine to show that we should. It’s a risk based analysis.

The sad thing is that John’d be pretty good at working out what action to take to help British businesses make money by getting the world a new zero carbon energy system. He says sound things on related subjects, such as the need for flood prevention (the photo at top of this blog post is John discussing flooding with his constituents) and on making Government buildings energy efficient. Heck, he has recently posted on energy security.

So near, John.

All you have to do is admit to the evidence that climate change is a risk to our country, just as insurance companies do, and help us take out the ultimate insurance policy.

A zero carbon energy system.

China has a new great wall, which blocks parts of the Internet from its citizens. This is partly done by absolute blocks, for example banning the BBC because it has a Chinese language news site. But it is also done by more subtle means – letting the companies who run forums know that they might be shut down if they don’t remove unacceptable posts, but never quite defining what unacceptable is. That way the companies have to err on the side of caution, and require less direct supervision and enforcement.

Sites I couldn’t access from within China: BBC News Online, Google News UK, Chinese Google News on google.com, Wikipedia, any WordPress.com blog. An expat told me in seconds how to get round some of these. For example, by searching Wikipedia at A9.

In contrast, I was surpised I could access: CIA World Factbook and EastSouthNorthWest (a Hong Kong run blog about China). The restriction really does seem to be only on Chinese language content.

For all that, the most physical censorship I found was of English, in the Economist I bought at Xi’an airport. There was an article about the Cultural Revolution. Curiously, I could read the leader which said “this editorial will probably not be read in China”. Then I turned to the main article – suddenly in shock. Pages 29 and 30 excised with a neat tear. As well as the Cultural Revolution article, one on the back about Japan.

Visions of factories full of censors, desperate to ship the Economist to educate businessmen into booming the economy, eager not to see China insulted by revision of Mao. Carefully turning the pages of the foreign magazine, one by one removing the offending page.

(For background, read my posts Chinese family history and Child of the atom bomb first)

In Shanghai, Rosemary and I went to a few more places related to our family history. Shanghai has undergone massive development, knocking down of whole areas, building of new skyscrapers. Amazingly, everywhere we went was still there. A hundred year old colonial buildings, with quite different architectures to those surrounding them. And still used in modern China. It was also great that the guards and porters would let us in. When Rosemary went 20 years ago, all she managed to do was peer from a distance. China is opening up, and relaxing.

The red building on the right of the left photo is the school that my grandmother used to teach in. She was a Physical Education teacher, and it was a school for Eurasians, people with one Western parent and one Chinese parent. The date above the entrance says 1893/1894, and it is now the Shanghai Installation Engineering Co, Ltd. The men on the left were arc welding some metal railings together.

The photo on the right is of the back of the hospital where my mother was born. It’s still a hospital. At first they wouldn’t let us in, I think because I’d been taking photos too obviously. Rosemary insisted she wanted to go round the back, I wasn’t sure why. Eventually a kind Doctor who spoke English was summoned, and escorted us round. The gardens were beautifully kept, and the rear of the hospital (photo) better architected. You can see the tall tower of the new wing rising up on the right behind the old building.

The bottom left photo is of the former Jubilee Court, where my grandparents, mother and aunt lived before they were interned by the Japanese. There was a sign on it saying “Monument under the Protection of Shanghai Municipality”. It’s still residential, we chatted to a woman whose mother lived in one of the other flats.

The right hand picture is of Yuyuen Rd camp. It’s where they were first interned before being taken to the Yangtzepoo camp at the end of the war. The camp used to be a school, and now is again. It’s called the Shi Xi High School, and had very well kept grounds. There were some kids playing basketball, even though it was the weekend.

(Just to explain, I’ve been back from China for a couple of weeks now, but still have blog posts to make about it, which I’m gradually catching up on)

When we were in Chengdu we went on a trip a couple of hours south to Emei Shan. This is where the religious thread of the holiday met the family history one.

My grandmother went on a journey up the Yangtze river to Emei Shan in 1936, with her friend Gracie. She’d been in Shanghai for six years, and in her summer holidays had already visited Peking, the Philippines, Saigon, Angkor, Bangkok, Korea and Japan. So by 1936 there wasn’t much left – they had to go to inland China. At the time, this was a very adventurous thing for two unaccompanied English ladies to do.

This verdant mountain is an ancient place of pilgrimage, both for pious Buddhists, and for seemingly millions of Chinese tourists. But as soon as you take a hard route away from the temples near the road, it becomes quiet and calm, you can watch the dragon flies, and be at peace.

We walked up the long route, but only went half way up. Even so it was very steep, so for some sections Rosemary got herself carried (photo). This was following her mother – who went up the mountain carried by 7 men! “3 to carry me – 2 Gracie and 2 baggage”

We stayed the night on the mountain in a Buddhist temple, in a surprisingly luxurious room. I’d hope for something a bit more down with the monks (I’d accidentally been to a service with some at the monastry in the town at the bottom of the mountain the night before), but Rosemary needed as much luxury as we could find. It was peaceful wondering round the glen in the twilight, with lakes and bridges. I went up onto the roof garden, watched and briefly played cards with a monk and some of the lay workers.

On the way back down (via foot, cable car and bus) in the morning we saw two donkeys struggling their way up the steps from the valley below. They were both loaded with panniers containing fine rocks for building. Panting all the time, they would sometimes stumble. Their two minders had spades, which I saw one beat his donkey with when it had fallen down once too often.

That afternoon we went to see the largest Buddha in the world as Leshan, carved in a rockface by the river. I climbed down the steps to his feet, and looked up in awe (photo). The Chinese tourists hardly looked, as ever busy snapping souvenir photos of themselves in front of him. I put my camera away. I thought about impermanence. He looked down at me and told me that life is change, and that I would never see him again. In the end, that’s the way it is with everything in this fragile world.

And my grandmother? She stayed five days at the top of the mountain, sleeping in a temple on their own camp beds, and eating tinned beans and pork they’d brought with them. The weather completely failed to clear, so she never saw the rarely visible fantastic view, and had to set off on the weeks of journey home.

She never mentioned the Leshan Buddha in her letters, even though her boat must have gone past it. This had puzzled my mother. But now we know. It is actually quite hard to see the Buddha from the river, you’d have to happen to look when the boat was right next to him.

Even though I’m an atheist, it’s fun and moving to seek out pious people and watching them going about their business. A few years ago I used to do this in the UK via Cambridge Interfaith, who organised such things as a visit to the synagogue on a Friday evening, and a day trip to the Hari Krishna temple George Harrison paid for in Hertfordshire. And I saw lots of religious places with Phil in Burma (see parts of this post and all of this one).

China’s religions are in resurgence, as things continue to loosen up following the Cultural Revolution. Numerous temples have been rebuilt recently. Just before leaving Shanghai, Rosemary and I went on a two day trip to Putuoshan. It’s a Buddhist holy island, a few hours journey by bus and boat south east. We spent the first afternoon going up to the top of the mountain and exploring temples. There were pious people there, lighting incense (photo left), and praying to Guanyin, goddess of mercy. Some pilgrims were even climbing the 1200m path of steps up the hill by prostating themselves forwards on the ground. They would stand up, step forward to where their head had been, and bow forwards flat, again and again.

The photo on the right is of a food area in one of the temples. You could pay a few yuan to a monk at a counter on one side of the temple, and get led off down side passages to the food area and dining room at the other side. There a lay assistant would fill a bowl with excellent vegetarian food, tasty aubergines and mushrooms and more strange Chinese vegetables. You help yourself to rice from a big vat of it, eat as much as you need. I think this is the same food the monks and their assistants ate, made available for anyone.

As well as Chinese pilgrims, there were also lots of Chinese tourists, most of whom we found in the morning down by the coast, coming to look at the brand new (well, 1998) shiny statue of Guanyin. She is golden, holding a small ships wheel in one hand to bring luck to fisherman. She towers what must be 30m above you, you climb up steps to her, flanked by guardians. She rests on a tower of two halls, on a promontry with almost 360 degree views of the sea. One hall contains amazing detailed wood carvings, and another 400 statues of her past lives. A stone carving frieze encircles the halls, skillfully carved with scenes of Guanyin helping locals. I’ve seen quite a few spectacular Buddhist statues, and this one was pretty good. Not to be diminished because she was new, she was awesome.

What I really want to know is how she was funded. The Chinese government have, I remember being told a few years ago, paid for lots of temples to be rebuilt, if only to attract tourists. But are there also rich Buddhist benefactors, paying millions for new statues? I haven’t found the answer, but here is a description of the Guanyin statue and some photos of her.

Everywhere in the People’s Republic of China there is building. In Shanghai, deep foundations of new skyscrapers. In Songpan, new apartments paid for by the government to relocate people from slumy areas. And north of Songpan, a newly upgraded highway across into Gansu province. The only problem is, it isn’t finished yet.

The bus left at about 7am. The first part was easy, and after one last stop to pick up passengers in another small town, we entered the area where they’re rebuilding the road. Rebuilding it, that is, as we drive along it.

Kilometers of drying road surface, covered with a white material held down with rocks, while we drive along the other bumpy incomplete carriageway. Small, tortuous detours round unready bridges, the bus lurching over rough earth. Cement factories, with stores of different granularities of stone powder. Folorn clusters of individuals, miles from anywhere, slowly pounding and fixing verges in place with rocks, just clinging to the surface of the earth. Tractors carrying metal girders, gangs locking them down to hold the edge of the carriageway. Tent villages, signs telling new arrivals to report to the office first, nomad shops. Winding precarious tracks down round and up valleys, us looking up enviously to the skeleton of a dramatic new bridge that will take the new highway straight through. New tunnels boring into the fabric of our world.

The bus stopped for a while, the driver’s assistant refilling water in the radiator. I’m glad, because I needed a piss. We drive on for a bit longer. And then the bus stops again for an age. Everyone urinates again.

Two ages.

I chat to the other three foreign tourists on board, from Sweden, Switzerland and France. There’s a field of yaks. It’s cold, desolate weather. I get a second jumper out of my main luggage.

Three ages.

The shack we’re next to, built of brieze blocks, a corregated metal roof sloping in one direction, turns out to be a restaurant. Well, she has fresh vegetables to stir fry. The Frenchman, an entertaining rogue of whom more in a separate post, helps ourselves to tea from the hot stove. She doesn’t even charge me for my noodle soup, when I find a spider in it. The yaks get taken up the hill to graze elsewhere.

The Chinese bus passengers play a strange game, with long thin cards I’ve never seen. They play another with normal playing cards, and I spend a while trying to work out the rules. I get out the novel I’m reading, which I haven’t touched for weeks. The sun comes out, the place seems cheery. (Later I find my nose flaking, having not accounted for the extra power of the sun at high altitude). We start trying to hitch lifts onwards to Zoige from the meagre passing traffic.

Eventually a minivan with a mechanic or two and some tools arrives, and they start working on the engine. These buses have the engine at the front between the driver and the door, you get to it by lifting up the cover that normally has luggage and extra people on it. Several people get very oily hands, and more hours pass.

Only the worst buses get put on roads like this. The new ones are being saved for a year or two, when the tarmac is good, the surface sweet. This morning (a couple of days later) I took a bus from Langmusi to Hezuo, along a beautiful finished new road. It was a lovely shining white bus, the TV flat screen, the windows large and clean. I almost wept each time we passed over the smallest of bridges without detour. The new highway even has roadsigns at junctions, and giving distances to places. In Chinese characters, and also, a pleasant surprise, in Tibetan. (And not always in the Roman alphabet).

Back by the spider restaurant shack, none of the other passengers really seem to mind or complain about the delay. A few have managed to catch lifts on to their destination, so by the time the bus starts again there’s a bit more room in it. We’re broken down maybe 4 or 5 hours, and arrive at our destination of Zoige at 7pm. Far too late for the afternoon bus onwards to our actual destination Langmusi, or even to take a taxi between us.

All this turned out to be a bonus in the end, as an unexepected stay in Zoige is much more interesting than the guidebooks imply. The roads approaching it are straight, along endless grasslands at 3000m above sea level. Grassy hills protrude out of the planes, and people herd black yaks. There’s a fascinating village we quickly pass through, of unusual wooden houses with moss rooves, each with a Tibetan prayer flag in the garden.

Zoige itself is indeed a grid of concrete buildings, but lovely ones painted delicately with Tibetan art. The people seem friendlier, I buy water and snacks in a small family shop, and the man gets his young girl to give me the change. Some adults are playing outside. School finishes, strangely at 8pm (maybe something to do with International Children’s Day) and hundreds of friendly kids burst out, practicing their excellent English on me. (Later I learn that their teacher is Tibetan, educated in exile in India, so has good English.) Small children guide me to an internet bar, but it’s full. I eat home style tofu, attended carefully and courteously by the owner of a new Chinese restaurant.

When the new road opens it will bring new trade, new tourists, new immigrant Han Chinese. To Zoige and to Langmusi. Everything changes.

At the moment I’m in Tibet.

No, I don’t mean the Tibetan Autonomous Region (TAR), I mean historic and ethnic Tibet. TAR is the province occupied by China which contains Lhasa, and which we lazily often refer to as “Tibet” in English. Tibet proper also includes all of Qinghai province, and parts of several other Chinese provinces. Tibetan people making a political point refer to themselves as living in Tibet, even if they don’t live in TAR. As a tourist this is good news – it is much easier to visit the non-TAR parts of Tibet, than suffer the complex web of permits and travel restrictions in TAR itself.

I took an all day bus ride north of Chengdu, a dramatic drive up into the mountains along a river lined with electrical pylons and filled with dam after dam, arriving in Songpan (population 70,000). Immediately the contrast with large Chinese cities was clear. It is nestled amongst low green mountains, with old city walls still giving it shape. I felt immediately much happier.

The thing to do in Songpan is to go horse trekking, which basically means a 3 day hike with camping, only all the uphill bits you sit on a horse. It’s too dangerous on steep downhill parts. This is hard work, back or knees aching after 6 hours riding in a day. Guides did all the real hard work, like erect tents and cook us dinner. The photo (right) is of a horse loaded up with feed, near a Tibetan prayer for peace, which involves putting arrows on the top of hills and mountains.

On the way we saw a Tibetan prayer water wheel (photo below). A small stream was channeled by rocks into a thin open pipe, where it landed on a simple and inefficient water wheel. This was housed in a little hut, and spun a prayer wheel. That, in turn, was octagonal, and had a chant written round it in the Tibetan alphabet. I assume that the idea, which is slightly mad if quite sweet and fun, is that by this means the river says your prayers for you.