Cross-posted here on the EA Forum.

Transformative AI and Compute - A holistic approach - Part 4 out of 4

This is the Appendix (part four) of the series Transformative AI and Compute - A holistic approach. You can find the sequence here and the summary here. This appendix attempts to:

Provide a list of connected research questions (Appendix A).
Present common compute metrics and discusses their caveats (Appendix B).
Provide a list of Startups in the AI Hardware domain (Appendix C).

Previous Post: Compute Governance and Conclusions

You can find the previous post "Compute Governance and Conclusions [3/4]" here.

Appendix

This appendix lists research questions, some thoughts on metrics related to compute, and a list of AI hardware startups.

These paragraphs ended up in the Appendix, as I have spent less time on them and it is more of a draft. Nonetheless, I still think they can be potentially useful.

A. Research Questions

This is not a research agenda or anything close to it. This is me doing research and thinking “Oh, I’d be curious to learn more about this *scribbling it down*”.

The questions rank from specific to nonspecific, from important to “I’m just curious”. There are a couple of questions which I think are more important, I’ve marked them with a ★.

This list is also published as a Google Doc, so one can add comments to individual items.

In general, this domain of research requires more:

Data Acquisition: Collect data in a machine-readable format on compute trends, hardware performances, algorithmic efficiency,
- Parameter, Compute and Data Trends in Machine Learning (Sevilla et al. 2021) is such an example. Work with us!
Data Analysis: Data analysis is a crucial ingredient: from trend graphs and doubling times up to new measurements of AI systems’ capabilities and AI hardware.
Data Interpretation: Once we have available data and analyzed general trends, we require interpretation for informing forecasts and potential policies.

Figure A.1: Sketch of research domains for AI and Compute.

Compute Governance

For a list of research questions, see “Some AI Governance Research Ideas” (Anderljung and Carlier 2021).
What is the current budget of public research organizations for compute?
- Should we lobby for an increase in compute budget?
  - If yes, how?
- How does this compare to corporate AI research labs?

Scaling Hypothesis

GPT-3 is a leading language model i

...