Critique of Strong AI Research
If research into Strong AI produced sufficiently intelligent software, it might be able to reprogram and improve itself. The improved software would be even better at improving itself, leading to recursive self-improvement. The new intelligence could thus increase exponentially and dramatically surpass humans.
Hyper-intelligent software may not necessarily decide to support the continued existence of mankind, and would be extremely difficult to stop. If its utility function were written carelessly, it might become a destructive amoral monster without regard for human values.This topic has also recently begun to be discussed in academic publications as a real source of risks to civilization, humans, and planet Earth.
One proposal to deal with this is to ensure that the first generally intelligent AI is 'Friendly AI', and will then be able to control subsequently developed AIs. Some question whether this kind of check could really remain in place.
Read more about this topic: Strong AI
Famous quotes containing the words critique of, critique, strong and/or research:
“Wagner’s art is the most sensational self-portrayal and self- critique of German nature that it is possible to conceive.”
—Thomas Mann (1875–1955)
“... the outcome of the Clarence Thomas hearings and his subsequent appointment to the Supreme Court shows how misguided, narrow notions of racial solidarity that suppress dissent and critique can lead black folks to support individuals who will not protect their rights.”
—bell hooks (b. c. 1955)
“He is a strong man who can hold down his opinion. A man cannot utter two or three sentences, without disclosing to intelligent ears precisely where he stands in life and thought, namely, whether in the kingdom of the senses and the understanding, or, in that of ideas and imagination, in the realm of intuitions and duty.”
—Ralph Waldo Emerson (1803–1882)
“The research on gender and morality shows that women and men looked at the world through very different moral frameworks. Men tend to think in terms of “justice” or absolute “right and wrong,” while women define morality through the filter of how relationships will be affected. Given these basic differences, why would men and women suddenly agree about disciplining children?”
—Ron Taffel (20th century)