Differential Intellectual Progress

• Standardized AGI lexicon.terminology. Research is continuing to establish formal definitions and language, thereby forming a framework by which researchers can communicate effectively and thus more efficiently evolve AGI safety development.

• Secure AGI construction.confinement. Incorporating physical mechanisms which limit the AGI can prevent it from inflicting damage. Physical isolation has already been developed (such as AI Boxing) as well as embedded solutions which shut down parts of the system under certain conditions.

Technological advances -— without corresponding development of safety mechanisms -— simultaneously increase the capacity for both friendly and unfriendly AGI development. Presently, most AGI research is concerned with increasing its capacity rather than its safety and thus, most progress increases the risk for a widespread negative effect.

The above developments could also help in the creation of Friendly AI. However, Friendliness requires the development of both AGI and Friendliness theory, while an Unfriendly Artificial Intelligence might be created by AGI efforts alone. Thus developments that bring AGI closer or make it more powerful will increase risk, at least if not combined with work on Friendliness.

Differential intellectual progress was defined by Luke Muehlhauser and Anna Salamon as "prioritizing risk-reducing intellectual progress over risk-increasing intellectual progress". Muehlhauser and SalamonThey discuss differential intellectual progress in relation to Artificial General Intelligence (AGI) development (which will also be the focus of this article):

Muehlhauser and Salamon also note that differential technological development can be seen as a special case of this concept.

Differential intellectual progress describes a scenario which,was defined by Luke Muehlhauser and Anna Salamon as "prioritizing risk-reducing intellectual progress over risk-increasing intellectual progress". Muehlhauser and Salamon discuss differential intellectual progress in terms of human safety, risk-reducingrelation to Artificial General Intelligence (AGI) development takes precedence over risk-increasing AGI development. In Luke Muehlhauser's Facing(which will also be the Singularity, he defines it accordingly:focus of this article):

• Intelligence Explosion: Evidence and Import by Luke Muehlhauser and Anna Salamon
• The AI Problem, with Solutions from Facing the Singularity
• Why We Need Friendly AI by Eliezer Yudkowsky
• Safety Engineering for Artificial General Intelligence by Roman Yampolskiy and Joshua Fox
• Leakproofing the Singularity by Roman Yampolskiy

• Increased computing power. Computing power continues to rise in step with Moore's Law, providing the raw capacity for smarter AGIs. This allows for more 'brute-force' programming, leading toincreasing the probability of someone creating of an AGI without properly understanding it (and thus, being less capable of controlling it).it. Such an AGI would also be harder to control.

• More efficient algorithms. Mathematical advances can produce substantial reductions in computing time, allowing an AGI to be more efficient within its current operating capacity. Since machine intelligence can be measured by its optimization power divided by resources used, this hasThe ability to carry about a larger number of computations with the same amount of hardware would have the net effect of making the machineAGI smarter.

• Extensive datasets. Living in the 'Information Age' has produced immense amounts of data. Not only hasAs data storage capacity has increased, butso has the medium on which itamount of information that is stored has decreasedcollected and stored, allowing an AGI immediate access to massive amounts of knowledge.

• Advanced neuroscience. Cognitive scientists have resolveddiscovered several algorithms used by the human brain which contribute to our intelligence, leading to a field called 'Computational Cognitive Neuroscience.' This technology has already led to significantdevelopments such as brain implants that have helped restore memory and motor learning in animals, algorithms which might conceivably contribute to AGI progress (such as neural networks).development.

While theThe above developments could loweralso help in the riskcreation Friendly AI. However, Friendliness requires the development of creatingboth AGI and Friendliness theory, while an Unfriendly Artificial Intelligence (UAI), this ismight be created by AGI efforts alone. Thus developments that bring AGI closer or make it more powerful will increase risk, at least if not the case presently. For example, an AGIcombined with access to massive datasets has the ability to use that information to increase its capacity to serve its purpose. Unless specifically programmed with ethical values respecting human life, it would inadvertently consume resources needed by humans in pursuit of this goal. This same paradigm applies for all risk-increasing progress.work on Friendliness.

• Standardized AGI terminology.Computer security. Research is continuing to establish formal definitions and language, thereby forming a frameworkOne way by which researchers can communicate effectivelyAGIs might grow rapidly more powerful is by taking over poorly-protected computers on the Internet. Hardening computers and thus more efficiently evolve AGI safety development.networks against such attacks would help reduce this risk.

• Friendly AGI goals. Embedding an AGI with friendly terminal values limitsreduces the actionsrisk that it canwill take with regardsaction that is harmful to human safety.humanity. Development in this area has lead to many questions about what should be implemented. However, precise methodologies which, when executed within an AGI, would prevent it from harming humanity have not yet materialized.

• Intelligence Explosion: Evidence and Import by Luke Muehlhauser and Anna Salamon
• The AI Problem, with Solutions from Facing the Singularity
• Why We Need Friendly AI by Eliezer Yudkowsky
• Safety Engineering for Artificial General Intelligence by Roman Yampolskiy and Joshua Fox
• Leakproofing the Singularity by Roman Yampolskiy

While the above developments could lower the risk of creating an Unfriendly Artificial Intelligence (UAI), this is not the case presently. For example, an AGI with access to massive datasets has the ability to use that information to increase its capacity to serve its purpose. Unless specifically programmed with ethical values respecting human life, it would inadvertently consume resources needed by humans in pursuit of this goal. This same paradigm applies for all risk-increasing progress.

Differential intellectual progress describes a scenario which, in whichterms of human safety, risk-reducing Artificial General Intelligence (AGI) development takes precedence over risk-increasing AGI development. In Luke Muehlhauser's Facing the Singularity, he defines it accordingly: