Build What Stays Hard

What's scarce when expert work is abundant

The digital products we want to build for the world's poorest countries are going to get easier and easier to make. And in many of the most important areas, the AI labs (OpenAI, Google, Anthropic and others) and the biggest funders are already putting serious money into making great tutors, safe clinical reasoning and AI that works in local languages.

I think that a year from now, a frontier model won't just be a high-quality tutor. I think it'll build a pedagogically sound tutoring product (with AI inside), add safety and learning guardrails, ship it, and run the data and experiment loop to improve it every week. If that's right, building a great product stops being the hard part.

But a good model and product aren't enough if they don't get used and don't change the outcome, as a trial at Penda Health's clinics in Kenya showed.

This is the question I came to next. If we really believe the models will keep getting better, and building an excellent digital product will become easy: what will limit impact then, and what should we do about it now?

I think this requires thinking about what a service needs beyond just intelligence and code:

Which of these factors limits impact the most depends on the area. There are three major patterns across the seven highest-impact opportunities from Chapter I.

1. Delivery

In clinical decisions, teaching and mental health, the thing that remains scarce once we've got great AI doctors and teachers is delivery. That means figuring out how to actually have the AI deliver care and instruction.

Getting there will take two things. First, we need to prepare for a world where we get better results as the AI does more of the work. Then, we need to do the much harder job of actually redesigning the service to support that — including how we help humans (staff and users) to use it.

At Penda, high-quality AI advice wasn't enough. Unfortunately, this is not limited to one trial. Once the AI is better at a task than the person, adding the person tends to make the result worse.² In medicine, that point seems to be arriving, as a recent JAMA piece suggests.

The consensus in rich countries is that AI should support the doctor or teacher, never make clinical decisions or take on a large share of teaching. Because service quality is already pretty good, we don't lose too much upside from that decision.

I don't think it's obvious that we should apply the same heuristic in developing countries. We know that the quality of instruction and care is often poor, because teachers and clinicians are often working without much training or support. So this raises a question: if AI can deliver much better learning and care for nearly free, how far should we take it? This doesn't have an obvious answer, but I think we have a responsibility to have the conversation — and to raise our ambition for what AI can do.

But except for some areas like mental health or farmer advice, AI cannot actually deliver the service on its own. And we know that a great tool doesn't always get used (Penda had to do seven weeks of coaching before clinicians started responding to AI alerts).⁵ So the harder part is redesigning the service and all its human messiness so we actually get the benefit of AI.

This is a really hard problem, and mostly a service + organisational one, not a technical one:

• How do people access the service? e.g., Do we hand a phone with a voice agent to someone who walks into the clinic? Put the voice agent in the clinician's ear and walk them through the consult? Or put AI-enabled software on a clinician phone?
• How do we make sure it gets used? This is partly about making the product so good that staff/users want to use it, but mostly about creating the right org, incentives, and supervision
• Where is the model good enough and not good enough, and how do we iterate fast as things change?

It's very hard to figure these things out from outside, by selling a tool to someone else's clinic or school. But I think we can learn some lessons from the private sector's work on this question. Increasingly, we're seeing that getting value from AI is an organisational problem more than a technical one — it's far faster to rework a firm you own than to persuade someone else to rework theirs. So companies are converging on this understanding: the best way to iterate fast and deliver better results with AI is to own the full service including the outcome.

Some development organisations already work this way, and Penda is the clearest case. Because the trial ran in its own clinics, Penda can keep iterating on the system and testing again until it finds what works.

I'll call these lighthouses: organisations or sites where we can redesign the service around AI, improve outcomes, measure every test, and show others what works. Running your own sites isn't new. What would be new is doing the hard organisational and system-design work to rethink the service to deliver higher quality, greater reach, and/or dramatically lower cost with AI. This might look like existing operators choosing a few lighthouse sites, or new orgs launching to start (or even acquire) lighthouse clinics and schools, rather than focusing on tech.

OpenAI points to Penda as its example of closing that gap. In teaching, Rising Academies has started: it tested Rori, its AI maths tutor, in its own schools before taking it into government ones.

In mental health, I suspect a lighthouse also has to build the product, because the labs have chosen not to act as therapists.¹¹ Counselling programmes like Friendship Bench, StrongMinds and Shamiri are natural places to build it.

Of course, most poor children go to public schools, and many poor families use public clinics. So a lighthouse only matters if what it proves becomes a public service, and that means working with government from the first day rather than arriving with results at the end. There are three routes to scale, and organisations are already on each. But I am still very uncertain about how we can iterate fast in lighthouses to build exceptionally cost-effective services, without falling into the trap of creating something which fails at national scale for political and implementation reasons.

2. Permission

In tax and courts, there's no lighthouse to run, because only the state can audit a taxpayer or hear a case. What's scarce is often permission to try. Although for tax, the product itself is likely also scarce: the models are getting much better at financial work, but this hasn't yet been pointed at the state.

Here, I think there is a really interesting opportunity to massively scale the impact of something like Tax Inspectors Without Borders (an OECD and UNDP scheme) with an AI-native tax inspection service.

In courts, the product already exists: Adalat AI runs in thousands of Indian courtrooms, and Kerala has made it mandatory for recording evidence.^{22, 23} Here, permission is earned by easing one of the biggest pain points for judges, and the opportunity is to use that trust to expand across the court's work and into new judiciaries.

3. Physical inputs

In drug discovery and agricultural R&D, capital and effort are beginning to pour into generating new drug and seed candidates with AI. The timeline over which this bears fruit is uncertain, but impact will almost certainly be limited by the other parts of the process.

So the useful work there is building trial sites, speeding up regulators' reviews, and getting new seed multiplied and released — likely not another AI discovery organisation. Trials are the clearest gap: Africa hosted 1.1% of the clinical trials started worldwide in 2023.²⁶

Area by area

Next in the series: III. Forge →