The idea of AI overthrowing humankind has been talked about for decades, and in 2021, scientists delivered their verdict on whether we'd be able to control a high-level computer AI
The catch is that controlling a super-intelligence far beyond human comprehension would require a simulation of that super-intelligence which we can analyze.
Once a computer system is working on a level above the scope of our programmers, we can no longer set limits.
A super-intelligence poses a fundamentally different problem than those typically studied under the banner of 'robot ethics', wrote the researchers.
The problem centers on knowing whether or not a computer program will reach a conclusion and answer, or simply loop forever trying to find one.
That brings us back to AI, which in a super-intelligent state could feasibly hold every possible computer program in its memory at once.
The effect, of this makes the containment algorithm unusable, said computer scientist Iyad Rahwan from the Max-Planck Institute for Human Development in Germany in 2021.
Alternative to teaching AI some ethics and telling it not to destroy the world something which no algorithm can be absolutely certain of doing is to limit the capabilities of the super-intelligence
A super-intelligent machine that controls the world sounds like science fiction, said computer scientist Manuel Cebrian from the Max-Planck Institute for Human Development, also in 2021.
But there are already machines that perform certain important tasks independently without programmers fully understanding how they learned it.
The question, therefore, arises whether this could at some point become uncontrollable and dangerous for humanity. The research was published in the Journal of Artificial Intelligence Research.