I am highly grateful to Alexey Morgunov and Adam Casey for reviewing and commenting on an earlier draft of this post, and pestering me into migrating the content from many emails to a somewhat coherent post.

Will Crouch has posted about the Centre for Effective Altruism and in a follow up post discussed questions in more detail. The general sense of the discussion of that post was that the arguments were convincing and that donating to CEA is a good idea. Recently, he visited Cambridge, primarily to discuss 80,000 hours, and several Cambridge LWers spoke with him. These discussions caused a number of us to substantially downgrade our estimates of the effectiveness of CEA, and made our concerns more concrete.

We're aware that our kind often don't cooperate well, but we are concerned that at present CEA's projects are unlikely to cash out into large numbers of people changing their behaviour. Ultimately, we are concerned that the space for high impact meta-charity is limited, and that if CEA is suboptimal this will have large opportunity costs. We want CEA to change the world, and would prefer this happens quickly.

The key argument in favour of donating money to CEA which was presented by Will was that by donating $1 to CEA you produce more than $1 in donations to the most effective charities. We present some apparent difficulties with this remaining true on the margin. We also present more general worries with CEA as an organisation under these headings:

Transparency
    Cost effectiveness estimates
    Research
80,000 hours
    Impact of 80,000 hours advice
    Content of 80,000 hours advice
    The 80,000 hours pledge
Scope and Goals
    Speed of growth
    Ambition

Transparency

It is worrying how little of the key information about CEA is publicly available. This makes assessment hard. By contrast to GiveWell, CEA programs are not particularly open about where their money is spent, what their marginal goals are, or what they are doing internally. As presented online, the majority of both 80,000 hours' and GWWC's day to day activity is maintaining blogs. These blogs are not substantial by comparison to, say, OB in terms of their frequency of content or their frequency of insight. Concretely, it does not seem that CEA is being tranparent in the sense of GiveWell.

Qn: How does CEA think its programs would score on a GiveWell assessment?
Qn: Does CEA think that GiveWell’s assessments systemically go wrong?

Qn: Does CEA consider the blogs to be a substantial source of impact? What external assessments or objective data support a claim of impact from the blogs?

Cost effectiveness estimates

As presented online and in person, CEA does not present as having credible models for their future impact. The GWWC site, for example, claims that from 291 members there will be £72.68M pledged. This equates to £250K / person over the course of their life. Claiming that this level of pledging will occur requires either unreasonable rates of donation or multi-decade payment schedules. If, in line with GWWC's projections, around 50% of people will maintain their donations, then assuming a linear drop off the expected pledge from a full time member is around £375K. Over a lifetime, this is essentially £10K / year. It seems implausible that expected mean annual earnings for GWWC members is of order £100K.

Qn: On what basis does GWWC assert that its near 300 members are credibly precommitted to donating £72.68M?

Looking at valuing marginal impacts, it would be hoped CEA's programs are better. For example, it has been stated that GWWC has an internal price of around £1700 for new pledges. This does not appear to extend to new programs, or to portions of 80,000 hours. In recent conversation with Will Crouch, he was asked what marginal value was placed on having a new intern in Cambridge (UK). There was no numerate response. Indeed, the assorted estimates do not cohere. If new pledges are worth £10K / year in expectation, and even 10% of the donations flow into CEA, then an intern generating 20 marginal pledges is a winning proposition for CEA at their stated wage level. If the horizon time for 20 pledges from one worker is larger than CEA can afford to wait, then it is not clear that CEA has an effective program for using their interns.

Qn: What does GWWC or 80,000 hours see as the marginal impact of one additional grad student in full time labour?
Qn: What is the horizon against which CEA programs are acting?

On research

The primary less visible activity of both GWWC and 80,000 hours is research. For GWWC, there are questions to be resolved about how best to Earn to Give, whether there are other activities which are less immediately fiscal but of high impact, and broadly how to identify near optimal opportunities for donation. For 80,000 hours, there is a need to establish how to optimise career paths for a broad set of potential terminal goals. Neither project appears to be bearing visible fruit. In conversation with Will Crouch, he observed that 80,000 hours don't know much about the burnout rates of various careers, the wage progressions or the likelihood of career progression.

At present, this means that 80,000 hours is not publicly presenting things which are better than other sources of advice. There is a need for the best current knowledge to be available quickly; there are people who are deciding careers now who are unlikely to do reliably better than average on the basis of the information that 80,000 hours has made public. It seems implausible that new results are reliably coming in so quickly that the time spent publishing the internal state of the art will substantively slow down further improvements. There is a strong sense in which they are being graded on their speed, with publication being the mediator of impact. It also seems plausible that the publishing and research are substantially orthogonal, and would use different people. Hence from the outside, the lack of published concrete advice seems to be a substantial reason to think that there is no internal art.

Qn: What is 80,000 hours producing with their current research time? What is the planned schedule? What constitutes success?

There is a similar concern with the output of GWWC. Of their listed papers, only one (by Toby Ord) is substantive. The remainder are not written as if there is a pressing need to have results that are concise, clear and better than other available materials. For example, there is an extended article on investing vs. giving, and another on the distinction between income and happiness. The former does little more than list factors that might be relevant, with no attempt to discern which of these effects are largest or a sense of what ranges of reasonable looking assumptions give. The second observes correctly that the impact of monetary loss on donors may be overestimated, but then doesn't even question how impacts on recipients should be converted into hedonic terms. As a document, it seems to have been written to convince rather than elucidate truth. Neither paper drives an update to a belief that the current researchers at GWWC are effectively seeking to identify close to optimal opportunities or to reason coherently about the impact of interventions.

More worrying is the absolute lack of material. Whilst the number of active researchers is difficult to discern from the website, it seems plausible that GWWC has had at least 6 people researching for it for at least the last year. There is no matching level of output; in academe one would expect to see several papers per year per person, and the primary claim of GWWC is that there are low hanging fruit in terms of the optimisation of donation and the ability of people to donate. So a priori, if GWWC was efficiently researching, one would expect it to be finding and publicising their results.

Qn: What is GWWC producing with their current research time? What is the planned schedule? What constitutes success?

80,000 Hours

Impact of the 80,000 hours advice

In conversation with Will, he asserted that on the basis of self-reports, something like 20-25% of those involved in 80K have changed or substantially rethought their career choice. This implies immediately that 75-80% haven't, and in practise that number will be higher care of the self-reporting. This substantially reduces the likely impact of 80,000 hours as a program. Indeed, it seems to be a near fatal problem for GWWC, in that if the 80,000 hours population is representative of pledges, then most of the GWWC pledges are earning in line with typical post grads, which makes it much harder to raise the mean value of each pledge to £250K as is required.

Methods of achieving this impact do not seem to be well attested. Will was asked what the internal value of a paid worker in Cambridge might be. A broad response was that it might improve the ability to give advice, but it was not suggested that this was based on hard data. This is a little troubling, because it implies that the effectiveness of 1-1 Skype interventions or 1-1 in person interventions are not known on a per hour basis. Absent this kind of data, it's difficult to see how 80,000 hours can be effectively optimising their impact.

Qn: Does 80,000 hours have data on the relative effectiveness of their activities?
Qn: How does CEA square a lack of reported career changes with GWWC's numbers, given background over-life earnings?

Content of 80,000 hours’ advice

In conversation, earning to give was suggested as being the baseline to measure against. Will noted explicitly that it's hard to know what kinds of careers are substantively better than others in a data-driven way. He was then very quick to hedge that by saying that of course research was valuable, and of course political activism could be valuable, and of course being a program manager at the world bank could be valuable (which would naturally require you do a PhD first), and of course being an entrepreneur could be valuable. It was not suggested that clearly at most one of these was optimal, or that people might ultimately be in a position where they trade off what they would have chosen to do in isolation against world optimising goals. We came away from this with the concern that 80,000 hours is not being epistemically vicious, and so is not willing to say things that might cause people to be unhappy. In particular, it seemed that there was more pressure to preserving the fuzzies that people were getting out of being affiliated to 80,000 hours than there was to make the advice good, and so most potential career paths were deemed to be OK.

Qn: Does 80,000 hours offer information that causes a substantial reduction in the space of careers that are considered optimal?
Qn: How does 80,000 hours square a lack of reported career changes with their advice being good?

The 80,000 hours pledge

It was noted that the pledge had been substantially weakened, to "I intend, at least in part, to use my career in an effective way to make the world a better place.". My recollection says that it used to be more like "I will use my career to most effectively reduce global poverty". There wasn't any particular defence of the choice of wording or any indication that there had been deep thought about precisely what that pledge should constitute.

The core mechanism by which 80,000 hours or GWWC will achieve long term impact has to be maintaining people's desire to act over a long period. In turn, it seems that the primary intermediate goal is to build a strong social structure to encourage adherence to these pledges. The pledges are then the key totems around which a community will be built, and so there should be massive pressure to optimise these and the surrounding social structures. This does not seem to have occurred.

Qn: What are the design decisions behind the pledge, and what motivated the change in pledge?
Qn: To what extent is the wording of the pledge thought to be important?

Scope and goals

Speed of growth

It was stated that around 1/3 of the Oxford undergraduate population (~4000 people) are on the mailing lists. Of that, there are around 300 members and a few dozen are coming to each event. By comparison, enterprising college societies in Cambridge (TMS, TCSS) have well in excess of 1000 undergraduates on their mailing lists, and get 80-100 people to their talks. When TCSS advertised an event to 1/3 of Cambridge, upwards of 600 people attended. From some organisational point of view, 80,000 hours Oxford could probably extract another factor of 5-10 out of its talk attendance. Whilst that won't factor through directly to the pledges, it seems unlikely that there would not be substantial growth there. In both of the Cambridge societies, the operating scale of the society has been doubled in a single year, by ensuring a reliable stream of events and getting networks in place to advertise widely.

It does not seem like the organisations are optimising for growth and retention of a population of attendees. This would provide a pool of people broadly on board with the aims of the organisations, and substantially enriched for likely pledges. It is very plain that such optimisation has not been codified and sent to other new chapters; the Cambridge GWWC chapter does not behave as if such guidance exists.

Qn: What optimisation has GWWC / 80,000 hours attempted in terms of the structure of their chapters?

Ambition

Taking a larger scale view, lots of these concerns ultimately cash out in a concern that a large fraction of the people involved with 80,000 hours or GWWC behave like dilettantes. There is an apparent desire to feel comfortable about career choice, think about dealing with poverty and get involved with 501(c)(3)'s/NGO's/UK charities. However as organisations, they are not behaving as we would expect for a bunch of people that seriously expect to vector hundreds of millions of pounds over the next decade, which is what continued linear growth would imply.

Nor do they seem to act as if they wish to seriously optimise the world. For example, the world bank throws ~$43B/year around. Which is easier: To upscale GWWC by a factor of ~17000, or double the mean effectiveness of the world bank? This should not be a hypothetical question; it should be answered. There doesn't seem to be an acceptance that large social structures are going to be needed to support GWWC style donation for a lifetime, in the fashion of say the rotary clubs.

Qn: Where does CEA see its projects in 10 years? 20? 40?