Introduction to Artificial Intelligence
People have been fascinated with the concept of artificial intelligence for as long as we have talked about intelligence. Some milestones in the discussion of AI
Turing Test: The Chess-playing robot of the 19th century The ELIZA software program. The AI FAQ Section 7 "Bots" or other agents that work over o ver the Internet What defines AI? - Discuss.
Is the game of Life AI? (cellular automata) Why or why not? What is the role of randomness? Some examples include chess playing systems, pattern recognition (voice, vision etc.), irritability, reproduction (software worms), software psychology (ELIZA1), machine learning, imprecision, cooperation. All these, when we study them in detail seem to be just interesting machines, not truly intelligent, except perhaps to the uninitiated. What is the difference between a sophisticated mathematical model and a learning system? Can we make a mail delivery mobile unit (robot) that could operate in a normal work environment with minimal programming? What is humanity's Frankenstein Complex? - Discuss
"I want to build a robot that is proud of me." The intersection of robotics and AI. (muscle and mind). The Robocup contest and all the other autonomous robot contests all focus around aspects of this problem. AI applies to autonomous robots because all other robots by definition have a human operator providing the intelligence. What constitutes an intelligent system? How about the Turing Test? Suggest your own test for intelligence. Can an author write a book about a protagonist that is more intelligent than the author? Are all our efforts so far merely stepping stones or is this an activity that is doomed from the outset? Discuss :
Can an author write a book about a character that is more intelligent than the author? What principles for developing intelligent systems can we p ostulate from this discussion?
To dig a little deeper http://cogsci.soton.ac.uk/~harnad/Papers/Py104/searle.comp.html What is the current state of AI research and application?
What's the difference between "classical" AI and "statistical" AI? (From the FAQ) Statistical AI, arising from machine learning, tends to be more concerned with "inductive" thought: given a set of patterns, induce the trend. Classical AI, on the other hand, is more concerned with "deductive" thought: given a set of constraints, deduce a conclusion. Another difference, as mentioned in the previous question, is that C++ tends to be a favorite language for statistical AI while LISP dominates in classical AI. A system can't be truly intelligent without displaying properties of both inductive and deductive thought. This lends many to believe that in the end, there will be some kind of synthesis of statistical and classical AI. Demonstration of learning . The 20 stick game. How did you arrive at a solution? What are the characteristics of the solution? What kind of learning is needed in each of the following circumstances: 1. A man is looking at a photograph of a person and says. "Brothers and sisters have I none. This man's father is my father's son. Who am I." 2. A little Indian and a big Indian are hiking through the desert each armed with a bow and arrow. The little Indian is the big Indian's son but the big Indian is not the little Indian's father. Who is the big Indian? Problem 1 - Deductive; problem 2 - Inductive References
Turing test ( http://cogsci.ucsd.edu/~asaygin/tt/ttest.html ) ELIZA http://i5.nyu.edu/~mm64/x52.9265/january1966.html Weizenbaum, J., "ELIZA -- A computer program for the study of natural language communication between man and machine", Communications of the ACM 9(1):36-45, 1966. ELIZA is a classic program that parses English sentences and responds with psychobabble generalities to keep the conversation going. Some people became convinced that they were talking to a real psychologist and even after the program was explained to them continued to believe that ELIZA's psychological he lp was meaningful and helpful.
AI FAQ http://www.faqs.org/faqs/ai-faq/general/part1/ The Robocup page http://www.robocup.org/02.html Appendices: Comments on the Artificial Intelligence discussion above
A. * What is the difference between a sophisticated mathematical model and a learning system? A model will only reflect your current understanding of the problem. A learning algorithm will discover new insights or structures within the problem that you were not aware of. A generic ANN will model a process without your knowing the eq uations to describe it. Note that most ANNs cannot then describe their "reasoning" they can only give answers based on their learning. B. Can an author ... * Need a standard of measurement. Note that optimization algorithms require us to measure success without necessarily being able to model or define the process by which the output is obtained. IE the measurement of the output is (and should be) independent of the process that creates the output. * The standard may be multi-faceted ("Seven Kinds of Smart" etc.), or, more pragmatically, may be measured only in dimensions relative to the problem. IE we produce "idiot-savants" * A math genius can solve a problem (E.G. what is the square root of 173579) instantaneously. I will take longer and will probably use a calculator, but I will still get an answer. How do you suppose Conan Doyle wrote the Sherlock Holmes mysteries? Can we define (create) a human-like intelligence that is an idiot-savant . IE has enormous capability in some limited areas of human intelligence but is severely handicapped in others. C. When will computers overtake human beings? * Problem: Some expressions of Moore's Law suggest that integrated circuit density (and by implication, computing capability) in a computer d oubles every 18 months. From our current understanding of physiology human neural networks have about 1010 nodes and 10100 interconnections. The current state of the art for artificial ANNs is about 106 nodes. Assuming that intelligence is simply a function of the number of learning nodes and that Moore's Law applies, how soon will computers have the same learning power as human beings? Assuming that each interconnection is represented by one unit of memory (say one byte) and that Moore's law applies to memory size, how soon will we be able to connect up sufficient interconnections? AI - Techniques
Artificial Neural Networks and Fuzzy logic Machine Intelligence a.k.a. artificial intelligence, a.k.a. computational intelligence
What constitutes an intelligent machine? What are the limitations on machine intelligence? We have seen at a very simple level how difficult it is to make a simple fork-lift cope with a controlled environment. How can a system cope with a much less controlled environment? How do systems deal with approximation and learning? Definition of AI: The ability of a system to perceive ch anges in its environment and to develop a plan and produce appropriate reactions. A system that exhibits learning and elements of reasoning (generalization, discovery, association, abstraction) Generalization: From several examples of birds can identify a generic creature as a bird or not-bird. Creativity: Can produce new meaningful artifacts from raw materials. Association: Shown one thing (car) can find another thing in memory that is similar (truck) Abstraction: From specific examples can generate a rule or method. EG watch objects falling => Law of gravity. OK now let's switch from philosophy to engineering. Some notes on Fuzzy Logic and Artificial Neural Networks Artificial Neural Networks (ANN) a.k.a. Machine Learning (CS ???) ANNs are either an artificial intelligence human-like learning system or a non-linear, statistical, parametric, multi-dimensional optimization algorithm, or both
Many models and algorithms - favorite occupation of researchers - develop a new algorithm. The famous ones Perceptron learning/back propagation Artificial Neural Networks (ANN): ANNs imitate human thinking. They are capable of learning relationships and patterns in a way that is analogous to organic memory. Some of the significant characteristics of this learning include * Learning to approach a correct answer with repeated examples or learning to group similar items together. * Generalization. From a set of typical examples the NN system can identify all patterns of the same type. EG if shown various samples of birds it can recognize birds it has never seen before. * Content recall. If shown part of a pattern the NN can recall the original pattern or type. * Fault and noise tolerance. NNs can recognize learned material even if the material is obscured by interference of various kinds.
Neural Networks find applications in situations where relationships or patterns cannot easily be described because they are nom-linear, ill-defined, time-variant multipleinput/multiple-output and so on. The neural network can learn from a "teacher" or can organize information into similar groups by itself. In either case the network can then be used with inputs or combinations that it has never seen before and can produce outputs according to its learned memory. The inherent structure of an ANN is massively parallel. All the input neurons receive information together, react together and pass the information on the next layer, eventually passing it on to the output layer. For example a graphic figure can be presented to an ANN where each pixel represents an input neuron. The picture is processed until the output layer produces one or more outputs identifying the picture. The parallelism of NNs partially explains why a human being whose neurons process approximately once per millisecond can outperform a computer in many tasks despite the fact that the computer can process (of the order of) once per nanosecond. The computer operation are processed serially. Humans are estimated to have approximately 1010 neurons with 10100 interconnections all being processed simultaneously. Artificial neural networks have found applications in image recognition, medical diagnosis and industrial control. They are best used where it is desirable to model an unknown process without extensive mathematical analysis of the process. Introduction to ANNs (Microsoft Word) ANNs and fuzzy logic are often used together. The neural network is used to identify important criteria and the fuzzy logic uses these criteria to control the process. ANN Review Questions
1. Consider the curve fitting analogy of ANNs. How many inputs to the ANN are represented by a simple graph of a curve being fit to a line? The function surface or error surface is an N-dimensional hyperplane. What defines the number of dimensions of the hyperplane? 2. Sketch a simple ANN. What are the essential three layers of the ANN. Simple ANNs are frequently described as having two layers beca use the inputs have no processing associated with them. Does the number of layers limit the number of inputs and outputs? 3. What are the implications of multi-input, multi-output systems like ANNs? Compare them to statistical correlation methods and closed loop controllers (E.G. PID). 4. Nodes in ANNs usually include a function such as a sigmoid (1/(1+e^-net) or tanh(net). What properties or abilities does this function give the ANN? Give one example of a system to be learned where this property is necessary.
5. ANNs learn to recognize or model unknown systems. Where is this learned knowledge stored? 6. One of the most popular forms of ANN is known as Back-Propagation. What is being propagated backwards? 7. For some time after ANNs were first conceived they did not develop because of a couple severe limitation in learning. Linear networks could not model which famous example problem? Non-linear networks had problems with which aspect? 8. What can be learned about the underlying physical properties of the system being modeled after the ANN is trained? 9. What is training set data and and test set data and how much of each is required (%)? 10. Assume you are using an ANN to predict plastics injection molding quality. You will use four different temperature and three different pressure signals and a flow rate signal to predict the breaking strain and amount of flash in the finished molded item. How many inputs and how many outputs would the ANN require. 11. How many hidden nodes would the ANN require for the previous problem? (Hint: Trick question) Fuzzy Logic: Concept: Many control ideas are understood well intuitively by human operators but are difficult to model for the purpose of creating an algorithmic control system. This is particularly true where control decisions are expressed in terms of generalized linguistic variables. ("Use full-power heating until it gets quite hot. If it gets very hot, turn down the heat a little until the reaction is fairly stable"). "Fuzzy" words like "quite hot," "very hot," and "turn down the heat a little" are difficult to convert into definite ("crisp") values. If the line between "hot" and "very hot" is taken to be, say, 120 degrees then that implies that 119.9999 is not "very hot" and 120.00001 is "very hot." Fuzzy logic provides a technique to quantify the "softness" in these variables and to make definite decisions on the basis of softly defined variables. Fuzzy logic control describes the control algorithm in terms of linguistic variables and sets of rules. Fuzzy logic, based on fuzzy sets provides a formal and structured way to convert these rules into a control scheme. Fuzzy logic is a superset of Boolean logic. In Boolean logic something can be '1' ("hot") or '0' ("not hot"). In fuzzy logic a fuzzy bit can assume any value, called degree of membership, between '0' and '1'. Application domain: "In general fuzzy logic is best applied to non-linear, time-variant, ill-defined systems." (Shetty & Kolk) Systems that can be represented by simple equations or sets of equations should be modeled using those equations and controlled using classical control techniques. Fuzzy logic will find applications where an expert is available who can describe in words what needs to be done to control a particular process. This expert's instructions can then be converted to a fuzzy logic rule set and
applied to the same problem without the need for extensive analytical and mathematical modeling of the system Example. Generate a set of rules in terms of linguistic variables. The four steps of fuzzy logic application are then Fuzzification (calculate membership in the fuzzy functions), Inference (MIN activation of each rule), Composition (Max activation of each output by each active rule), De-fuzzification. (centroids and singletons etc.) In practice some time will be spent "tuning" the system after it is built.
Implementation: The real-time processing is straight-forward and is usually configured to only require ordinary arithmetic operations. This allows it to be fast and to be executed on very simple processors. With many fuzzy logic packages available for industrial applications many more applications are including this technique. This should no longer be considered as an esoteric scientific concept but rather as available application-oriented technology. See the reference on fuzzy logic for more detail Applications of fuzzy logic: * Spell-checker to suggest the most likely word to replace the mis-spelled one. * Control of Japanese subway system * Camera exposure control Fuzzy Logic Review Questions
1. What are "fuzzy" and "crisp" values? Describe today's temperature in fuzzy and crisp terms. 2. What is a linguistic variable? 3. Is Boolean logic a subset of fuzzy logic or is fuzzy logic a subset of Boolean logic or do they partially intersect or is there no intersection between the two systems? 4. What is a membership function? What are common membership functions (describe in words or sketch). Is there a membership function that describes Boolean logic? 5. In Boolean logic NOT(1)=0 and NOT(0)=1. What is the meaning of the NOT operation in fuzzy logic? 6. What problems in Mechatronics could you use fuzzy logic to solve? Pick one example problem, define linguistic variables and a rule set for that application. 7. Earlier we noted that PID loops were single-input-single-output (SISO) and that neural networks were (in general) multiple-input-multiple-output MIMO. Furthermore PID loops are intended for 2nd order linear differential systems (systems describable in terms of 2nd order linear differential equations E.G. a capacitor-inductor or a spring-mass
system) and neural networks were intended for non -linear systems in general. How would you describe fuzzy logic in these terms? Reference:
FAQs for AI, ANN and fuzzy logic http://www2.cs.cmu.edu/Groups/AI/html/faqs/ai/top.html 1. ANN links http://www.emsl.pnl.gov:2080/proj/neuron/neural/ Fuzzy logic links: 1. Fuzzy logic newsgroup and FAQ 2. Fuzzy logic links page http://www.abo.fi/~rfuller/fuzs.html