Measuring Intelligence
How does one measure intelligence? Common answers include standardized general metrics like IQ tests, Weschler scales, and things of that sort. However, such tests cannot be administered to artificial intelligence. How, then, can we measure the intelligence of AI? How can we know that we are moving forward in the field, growing ever closer to the day that computers can think and relate the same way humans can?
Such is the thesis of Francois Chollet, the creator of Keras. Keras is an open source library that provides a Python interface for neural networks. In his publication, On the Measure of Intelligence, published in 2019, he makes several points about AI and the way that we view the intelligence of AI today that are rather thought provoking.
Current AI are not intelligent
We are in a narrow AI phase right now. AI and machine learning can only do specified tasks, and narrow ones at that. An AI built and optimized to play chess, for example, may be very good at playing the version of chess it is taught, with the specific rules that it is optimized for, but it will not be able to apply its chess knowledge to other similar board games, like shogi or xiangqi, which are the Japanese and Chinese versions of chess, with somewhat similar rules and strategies, but different in their approaches and rules.
First, let me define narrow AI, general AI, and super AI as is understood by the greater artificial intelligence community:
Narrow AI:
Narrow AI is AI that performs specified tasks. This is the artificial intelligence that we see in the world today. It is programmed to perform that specified task, and the data it is fed is also purely for the sake of advancing its ability to perform the task it is designed to do. No matter how intelligent these systems may seem to the outsider, it can only operate within the pre-determined range that its programmer set for it.
Narrow AI cannot create or “think” outside of what it is programmed to do. Let’s take Siri as an example. Siri is considered an AI built on neuro-linguistic programming. Siri cannot inherently understand what we say; it can only process the sounds it hears, feed it into a search engine, and return results to the user. We cannot ask Siri to solve problems that cannot be found in a website.
Narrow AI is far less intelligent than humans. As humans, we have the ability to correlate and apply our experience, or, in terms of data science, our “training data”, to other similar situations. Humans can see a red stop sign, and then correlate that response to a red light, somewhat understanding that while driving, both mean stop. However, a self-driving car cannot make that correlation unless specified by its programmer or data.
General AI:
General AI is AI that shows human-level intelligence. This level of AI can think correlatively and learn the way humans can. More importantly, the difference between narrow AI and general AI is that general AI should not need a specific data set to train it specific things. When AI reaches this level, it should be able to take in any data and, like humans, think and connect different concepts on its own.
The difficult part about creating general AI is that humans have the ability to think abstractly and correlate emotionally to each other. Teaching a computer concepts that we don’t even fully understand ourselves is a problem that professionals and researchers are still looking to solve.
Super AI:
Super AI is AI that has surpassed human intellect. At this stage, AI has surpassed human intelligence in pretty much all facets — from creativity to logic. This level of AI is what some people, such as Stephen Hawking, worry will lead to the end of the human race. However, luckily, we are nowhere near even general AI.
What is Intelligence?
“Intelligence measures an agent’s ability to achieve goals in a wide range of environments.” — Shane Legg and Marcus Hutter
We typically define machinated intelligence by its ability to complete a specified task, or its ability to pass a Turing test, but Chollet uses a broader, higher level definition: in the same way we deem humans intelligent or unintelligent, AI should be able to think logically and abstractly through a wide range of environments to be deemed “intelligent”.
What Chollet argues, in a nutshell, is that intelligence is defined by the capabilities of an AI to solve problems it was not expressly designed to solve. An AI that plays chess well will not know how to play checkers or shogi. Thus, all it shows is the intelligence of the programmers behind it, rather than its own inherent intelligence. (supervised learning)
True higher-intelligence should be able to apply their intelligence across a series of at least similar tasks (an intelligent and generalized MOBA AI should be able to excel in League of Legends, DOTA, Smite, etc.).
Important concept: Generalization
Generalization: “ability to handle situations (or tasks) that differ from previously encountered situations.”
“Solving any given task with beyond-human level performance by leveraging either unlimited priors or unlimited data does not bring us any closer to broad AI or general AI, whether the task is chess, football, or any e-sport,” — Chollet
Sources and Additional Material:
