I think it might be true that substrate convergence is inevitable eventually. But it would be helpful to know how long it would take. Potentially we might be ok with it if the expected timescale is long enough (or the probability of it happening in a given timescale is low enough).
I think the singleton scenario is the most interesting, since I think that if we have several competing AI's, then we are just super doomed.
If that's true then that is a super important finding! And also an important thing to communicate to people... (read more)
I am interested in the substrate-needs convergence project.
Here are some initial thoughts, I would love to hear some responses:
An approach could be to say under what conditions natural selection will and will not sneak in.
Natural selection requires variation. Information theory tells us that all information is subject to noise and therefore variation across time. However, we can reduce error rates to arbitrarily low probabilities using coding schemes. Essentially this means that it is possible to propagate information across finite timescales w
Thanks for the reply!
I think it might be true that substrate convergence is inevitable eventually. But it would be helpful to know how long it would take. Potentially we might be ok with it if the expected timescale is long enough (or the probability of it happening in a given timescale is low enough).
If that's true then that is a super important finding! And also an important thing to communicate to people... (read more)