on chaining the AI’s Prometheus


Prometheus brings Fire to Mankind: image by Wikipedia

Dr. Aubrey de Grey: SIAI Interview Series

“The real big deal about the Singularity is the negative side rather than the positive side.

* * *

“The possibility that when, if, we develop these machines that are recursively self-improving and become extraordinary intelligent very quickly indeed, they may not like us terribly much. They may decide we’re not very important. That would be a bad thing from our point of view, at least; maybe from an objective observer’s point of view, it wouldn’t be such a bad thing but we are not objective.

” So the question is can we develop machines that can be repetitively self-improving but which maintain themselves reliably within a sort of … a space … a subset of potential machines of this sort that carry on thinking of human beings as important and, indeed, is important as their main retondic. In order to make sure this happens, one has to do some very careful mathematics and some very careful planning to make these machines in a way that retains that property.

It’s worse than that.

The Singularity Institute takes the view that we’ve got to get there first. If someone comes along and invents recursively self-improving systems that do not have this, what we like to call this ‘friendliness’ property, and that happens first, they will get rid of humanity rather quickly if we’re unlucky. So we better develop friendly AI before anyone else ‘accidentally‘ develops unfriendly AI.

“Now, whether friendly AI is even possible is unknown at this point, whether it is possible to invent machines which you can give the freedom to improve themselves without giving them the freedom to become unfriendly; we just don’t know whether that’s possible. But it’s worth trying.”

Video: Dr. Aubrey de Grey Interview . Dr. de Grey advises The Singularity Institute and co-founded The Methuselah Foundation.

Source: SIAI Interview Series

+ + + + +

Newcomb’s commentary: “Where there are two desires in a man’s heart he has no choice between the two but must obey the strongest, there being no such thing as free will in the composition of any human being that ever lived.” Mark Twain, Eruption


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: