I’ve recently become quite interested in the idea of the technological singularity, which is basically where artificial intelligence becomes more intelligent than human intelligence. What form this takes, and how we get there is not known, but it is not uncomprehendable that we accidentally or purposefully build an artificial general intelligence which evolves itself beyond the level of its creators intelligence.
That aside, I have watched a few of the talks from the Singularity Summit of 2012, and stumbled across one talk by Julia Galef (of CFAR) on “Rationality and the Future“. Rationality is an important on its own, but it has a special relationship with singularity theory. It seems to me (and those of you in this particular field, please do feel free to correct me), that rationality is important in singularity theory for the following reasons:
- Machines are programmed to be rational. Programming languages are based on mathematics - such as algebra, calculus, geometry and proof. It is this “proof” theory which allows us to test, and be confident that an algorithm (or whole software) will act in a certain way.
- Rationality allows us to define beliefs, intentions and desires (BDI). As humans, this has, or at least should have, an implication on the decisions we make and the actions we perform thereafter. The same stands for an artificial intelligence - in machine learning algorithms the results may or may not match up with reality or even rationality, and those decisions will lead into action for an intelligent agent. PEAS (Performance measure, Environment,. Actuators, Sensors) theory also comes to mind.
- Also from what I’ve seen in singularity topics, there is plenty of opinion. Some opinion is based on reasonable speculation, and some is based on pure guesswork. (Although it sounds as if expert opinion and non-expert opinion when it comes to singularity is somewhat similar in its estimations on when singularity will occur. See talk on How we’re predicting AI by Stuart Armstrong). This means that rational thinking is essential for humans to sort through the strong theories, and the weak theories. Having assumptions is something necessary as we don’t know everything, and those things that we do know exhibit levels of uncertainty and vagueness, but the important thing is to actually specify for any particular statement that you are taking such an assumption.
So the problem with the above is that almost every human is at least sometimes irrational. There are very few people that are able to live completely rationally. Uncertainties and vagueness permeates our understandings and our communications, not to mention that we do things wrong because of physical limitations (temporal or permanent). This is not necessarily always a bad thing - for example, when we fall in love (could be with a person, or a place), we might have our reasons for falling in love, but these reasons might not necessarily match up with science and mathematics, if they do then scientific and mathematical reasoning is not necessarily at the front of the mind of the human.
The talk by Galef, mentioned (and I am paraphrasing here) that one of her students came to her saying that he did not know whether to move away from family and friends in order to take a much higher paid job. To which Galef rephrased the problematic decision to being if you were already in that job, would you take a very big pay cut in order to move back to your family and friends. To which the answer was apparently “no”. Galef said that this rephrasing of the decision got around the problem of the Status Quo, in that people prefer to stay in a situation than move from it - even if it is the irrational option.
It is a good example, and rephrasing a decision can allow for more reasonable decision making. It also depends on how much we hone in on one or the other forms of decision. For example, in the decision about moving for a job, there could be an element of risk involved - the what-if’s could creep in, for example what if I don’t make friends, what if I lose the job, what if I am not comfortable in the place where I live. The level of risk might be too much for a rational move. In other words the level of risk is greater than the level of pay increase. Likewise risk can creep in to the inverse - if I stay where I am, then what if I lose my job, what if I lose my friends or upset my family, and what happens if my environment changes dramatically. The level of risk might be too much for a rational stay. We could also go into much more depth of reasoning, and actually give value to staying or going. This is turning the irrational into the rational… but do we always need to go into such depths of reasoning? Particularly as we’re sometimes irrational anyway, can we not hone our decisions without becoming so rational?
At the moment I don’t know the answer to this final question, or even know whether it is very important. What I do know is that this irrationality, or at least just the uncertainty and vagueness, is the reason why I became involved in and continue to be interested in Fuzzy Set Theory and Fuzzy Logic. Fuzzy attempts to model these shades of grey, allows for them to be reasoned, and does not have the requirement of definitive input or output. Probability theory is another area which helps with uncertainties, and I am very convinced that there is use for Fuzzy Probabilities and Possibility theory in Artificial Intelligence. Particularly if we combine such reasoning systems with knowledge bases (and that is where my knowledge of Semantic Web / Linked Data and Databases comes in handy).
These are just my initial thoughts on rationality for this blog, as I go along in my research into fuzzy theory and artificial intelligence I’m sure I’ll have more. Plus, I’m sure they’ll develop the more I consider singularity too.
Please feel free to comment.