By Purity Mukami, Joy Kirigia, Gabriel Geiger, Tomas Statius, Naipanoi Lepapa
SHA’s AI system for predicting income was guaranteed to overestimate the earnings of the poorest and underestimate the earnings of the richest. This meant that the poorest would pay more than they should for healthcare, while the wealthiest paid less.
A previously unseen report by international consultants tasked with assessing the AI prediction system said it was “inequitable”, based on flawed data and “highly likely” to miscalculate. Low-income households would be hardest hit.
Insiders described how, at every turn of the system’s design and implementation, warnings about the tech were ignored as the changes were “bulldozed through”.
An exclusive audit of SHA’s algorithm exposes its biases in detail.
As protests over the rising cost of living swept across the country, the Ruto government prepared to roll out AI inside the public health system.
To get more Kenyans to pay for health insurance, a new system would ask questions about how people lived and use the answers to predict their wealth. The idea was that the poorest should pay as little as possible based on their actual income rather than a flat rate as was the case with the old system.
A new investigation, by Africa Uncensored, Lighthouse Reports and The Guardian, can show that the government was told that the rollout would end in disaster. The system was biased in a way that would hammer the poor, while letting the wealthy off.
Consultants warned that the income prediction algorithm was essentially unfixable, while experts in Kenya and in international organisations cautioned that it should not be used.
The government launched it anyway.
This investigation looks deep inside the choices that were made to understand how the set-up of the new Social Health Authority (SHA) gambled with people’s lives.
Ruto announced that SHA would extend affordable healthcare to all. “No Kenyan will be left behind,” he said in 2023.
The big idea was that by asking dozens of questions about what people own and how they live, and comparing their answers to a set of survey findings taken a few years before, it was possible to train a computerised system to make an estimate of their income.
The team building the system were faced with a decision, however. The formula they used was guaranteed to overestimate the income of the poorest people and underestimate the income of the richest. This meant that the poorest would end up paying more than they should, while the wealthiest paid less.
We spoke to David Khaoya, one of the first people to work on the system.
Faced with that choice, Khaoya said, the government decided to prioritise correct assessment of the wealthy, even if that meant overcharging the poor.
“If you identify a richer person as poor and ask him to pay less, this person will never own up and say look, I’m actually supposed to be paying more,” he said.
Poor people would have an incentive to contest the sum of money they were asked to pay and get the mistake corrected, he suggested.
But when the international consultants hired to examine the system saw it, they sounded alarm bells. We obtained their report, written on the eve of SHA’s launch. Its contents have not been seen by the public until now.
The consultants said the income prediction system was “inequitable, particularly for low-income households”. The survey used as a basis for teaching the system to make predictions was flawed because it had too much data from middle-income households and too little from the poorest. It was also out of date, especially given the “multiple economic shocks” that Kenyans had suffered over the last four years.
The consultants warned that, unless new training data was used, the system was “highly likely” to miscalculate how much people ought to be paying. The poorest would be hardest hit.
No new training data was available, however. All that could be done was adjust the system to try to avoid the worst outcomes. When they tried to do this, they managed a “slight overall improvement”, with predictions for low-income households getting more accurate. But only by 7%.
Last year we requested SHA disclose to us the inner workings of the AI prediction system. When the agency refused, we made a complaint to the Ombudsman. Forced to comply, SHA sent us information on how it makes its calculations.
We used this information, along with the consultants’ report, to build our own version of SHA’s prediction algorithm and test it to see how it performed.
With or without the adjustments proposed by the consultants, we made one consistent finding. The poorest were paying too much.
As we dug into the choices that were made, interviewing insiders and experts about SHA’s set-up, we found how at every turn people had warned against using this type of technology. But they were overruled.
“It was bulldozed through,” one insider told us.
Add comment