May 5, 2021 16:4 Arti.cial Intelligence: From Beginning to Date 9in x 6in b4086-fm page vii
Foreword
In May 1983, I went to Purdue University in the United States to study arti.cial intelligence under the direction of Professor K.S. Fu, a member of the National Academy of Engineering Sciences, an international pioneer in arti.cial intelligence, and the father of international pattern recognition. Under the advice and guidance of Academician K.S. Fu, Guangyou Xu, and I edited and wrote a book, Arti.cial Intelligence and Its Applications.It was published by Tsinghua University Press in 1987 and became the .rst published arti.cial intelligence book with independent intellectual property rights in China. This book has played a major role in promoting the spread and development of arti.cial intelligence in China in the past 35 years. I deeply miss and thank Professor Fu, Professor Wenjun Wu and Professor Tong Chang, and deeply thank to Dr. Jian Song and Professor Yanda Li.
As a frontier and interdisciplinary subject, arti.cial intelligence has been advancing with the times along with the progress of world society and the development of science and technology. It has made great progress in the past 60 years. In recent years, a new round of arti.cial intelligence research and entrepreneurial climax has emerged, pushing the development of arti-.cial intelligence into a new era. New arti.cial intelligence algorithms rep-resented by deep learning promote the widespread application of arti.cial intelligence, and the industrialization of arti.cial intelligence has also risen to new heights.
At present, there is a lack of monographs at home and abroad that fully re.ect the latest developments in arti.cial intelligence. People hope to have such a book available. The book should not only have a novel system architecture, fully re.ect the scienti.c and technological connotation of arti-.cial intelligence, but also highlight the innovative development of arti.cial intelligence and guide its application. WSPC advised us to publish a new monograph on arti.cial intelligence at the right time. After reviewing and discussing our Publication Proposal, on the recommendation of AI experts
3
May 5, 2021 16:4 Arti.cial Intelligence: From Beginning to Date 9in x 6in b4086-fm page viii
Arti.cial Intelligence: From Beginning to Date
4
such as Dr. A. Ng and Dr. J.H. Zheng, as well as the recommendation of IEEE Fellow Dr.YC Jin and IEEE Fellow Dr. L.C. Jiao, the publishers agreed to compile and publish a new arti.cial intelligence monograph named Arti.-cial Intelligence: From Beginning to Date, and proposed a new architecture and related content.
This book covers a wide range of topics in arti.cial intelligence and has three characteristics. First, this book is a systematic and comprehen-sive book covering the core technologies of arti.cial intelligence, includ-ing the basic theories and techniques of traditional arti.cial intelligence, and the basic principles and methods of computational intelligence. Sec-ondly, this book pays attention to innovation, focusing on the introduction of machine learning, especially deep learning technology and other arti.cial intelligence learning methods that have been widely used in recent years. Third, the theory and practice of this book are highly integrated. There are theories, techniques, methods, and many examples of deep learning appli-cations that can help readers understand the theory of arti.cial intelligence and its application development.
This book is divided into three parts, following the introductory Chapter 1, which describes the de.nition, classi.cation, origin, and devel-opment of arti.cial intelligence, introduces the research objectives and main contents of arti.cial intelligence, and lists the research and applica-tion .elds of arti.cial intelligence.The .rst part is knowledge-based arti.cial intelligence, including Chapters 2 to 4; Chapters 2 and 3 study the knowl-edge representation method and search inference technology of arti.cial intelligence, and Chapter 4 discusses knowledge-based machine learn-ing. Part 2 is data-based arti.cial intelligence, including Chapters 5 to 7; Chapters 5 and 6 introduce neural computing and evolutionary computing, respectively, and Chapter 7 discusses data-based machine learning. Part 3 contains application examples of arti.cial intelligence, including Chapters 8 to 11, elaborating on the important application areas of arti.cial intelligence in each of the following: expert system, intelligent planning, intelligent per-ception (including pattern recognition and speech recognition), and natural language processing, etc., with special attention being paid to introduce the application of deep learning in various related .elds. The last chapter, Chapter 12, is the prospect of arti.cial intelligence, involving the impact of arti.cial intelligence on humans, the deep integration of arti.cial intel-ligence technology, and the industrialization of arti.cial intelligence. This book is a practical guide for arti.cial intelligence research and development
May 5, 2021 16:4 Arti.cial Intelligence: From Beginning to Date 9in x 6in b4086-fm page ix
Foreword
5
personnel, and a valuable reference book for undergraduate and graduate students to learn arti.cial intelligence.
In the process of writing this book, we have referred to hundreds of ref-erences, citing some of their materials, so that this book can draw on the strengths of each family, more comprehensively re.ecting the latest devel-opments in various .elds of arti.cial intelligence. In Part III, each chapter focuses on X.A. Bao, Siyu Guo, J. Johnson, Fei-Fei Li, L.J. Li, J.D. Lu,
A.Y. Ng, D. Wu, as well as Z.H. Zhou, etc. in deep learning application research examples. Their works or discussions with them have provided rich nutrition for this book, which has greatly bene.ted us. We express our heartfelt thanks to them. We also wish to sincerely thank Dr. A. Ng and Dr. JH Zheng and other experts for their important suggestions on the content organization of this book, as well as IEEE Fellow Dr.Y.C Jin and IEEE Fel-low Dr. L.C. Jiao, who strongly recommend the topic selection of this book. Special thanks to Dr. J.F. Cai for his strong support and help in the work of this book.
We sincerely thank the relevant leaders, experts, and editors from Central South University, Hunan ZIXING Academy of Arti.cial Intelligence, Tsinghua University Press, and WSPC. Without their support and encour-agement, wisdom and talent, hard work and vigorous cooperation, this book could not have been made available to the readers as quickly.
The writing of this book is presided over and drafted by Zixing Cai. The division of tasks for each chapter is as follows: Chapter 15, Chapter 8 9, and Chapter 1112 by Zixing Cai, Chapter 6 by Yong Wang, Chapter 7 by Lijue Liu, and Chapter 10 by Baifan Chen. For a long time, the three young authors, Dr. Wang, Dr. Liu, and Dr. Chen, were all core members of my research team, and they have emerged in the .eld of arti.cial intelli-gence research and education. Due to our limited knowledge and the rapid development of arti.cial intelligence in recent years, we are not familiar enough with the latest developments in some areas of arti.cial intelligence; therefore, shortcomings are inevitable. We sincerely hope that experts and readers will not hesitate to provide valuable feedback.
Zixing Cai
Deyi Garden, Eyang Mountain, Changsha, China
March 21, 2021
List of Tables 17
List of Figures 19
About the Authors 25
Chapter 1. Introduction 1
1.1 De.nition and Development of Arti.cial Intelligence .... 2
1.1.1 De.nition of arti.cial intelligence ........... 3
1.1.2 Origin and development of arti.cial intelligence ........................ 5
1.2 Classi.cation of Arti.cial Intelligence Systems ........ 14
1.3 Research Objectives and Contents of Arti.cial Intelligence............................. 19
1.3.1 Research objectives of arti.cial intelligence .... 19
1.3.2 Research and application .elds of arti.cial intelligence ........................ 20
1.4 Core Elements of Arti.cial Intelligence ............ 25
1.5 OutlineoftheBook ........................ 27
References ................................ 29
Part 1: Knowledge-based Arti.cial Intelligence 33
Chapter 2. Knowledge Representation
35
2.1 StateSpaceRepresentation .................. 36
2.1.1 Problem state space description ........... 36
2.1.2 Graph theory terminology and graphic method . . . 38
2.1.3 Problem reduction representation .......... 40
7
2.2 KnowledgeBase ......................... 44
2.2.1 De.nition and characteristics ofknowledgebase ................... 45
2.2.2 Design and application of knowledge base ..... 46
2.3 Ontology .............................. 48
2.3.1 Concept and de.nition of ontology .......... 48
2.3.2 Composition and classi.cation of ontology ..... 52
2.3.3 Ontologymodeling ................... 53
2.4 SemanticNetworkRepresentation ............... 56
2.4.1 Composition and characteristics of the semantic network .......................... 56
2.4.2 Representation of a binary semantic network . . . 57
2.4.3 Representation of a multi-element semantic network .......................... 59
2.4.4 Inference process of a semantic network ...... 60
2.5 KnowledgeGraph......................... 61
2.5.1 De.nition and architecture ofknowledgegraph ................... 62
2.5.2 Key technologies of knowledge graph ........ 64
2.6 FrameRepresentation ...................... 70
2.6.1 Framecomposition ................... 70
2.6.2 Framereasoning..................... 73
2.7 PredicateLogicRepresentation ................ 75
2.7.1 Predicatecalculus.................... 76
2.7.2 Predicateformula .................... 79
2.8 Summary .............................. 81
References ................................ 83
Chapter 3. Knowledge Search and Reasoning 85
3.1 GraphSearchStrategy...................... 85
3.2 BlindSearch ............................ 88
3.2.1 Breadth-.rstsearch ................... 88
3.2.2 Depth-.rstsearch .................... 90
3.2.3 Uniformcostsearch................... 90
3.3 HeuristicSearch.......................... 91
3.3.1 Heuristic search strategy and valuation function .......................... 92
3.3.2 Orderedsearch ..................... 93
3.3.3 Algorithm A. ....................... 96
3.4 ResolutionPrinciples ....................... 99
3.4.1 Extractionoftheclauseset .............. 99
3.4.2 Rules of resolution reasoning ............. 102
3.4.3 Solving process of resolution refutation ....... 102
3.5 RuleDeductionSystem .....................104
3.5.1 Rule forward deduction system ............ 105
3.5.2 Rule reverse deduction system ............ 111
3.5.3 Rule bidirectional deduction system ......... 114
3.6 ReasoningwithUncertainty ...................117
3.6.1 Representation and measurement ofuncertainty.......................118
3.6.2 Algorithmofuncertainty ................119
3.7 ProbabilisticReasoning .....................121
3.7.1 Basic properties and computing formulas ofprobability .......................122
3.7.2 Method of probabilistic reasoning ........... 124
3.8 SubjectiveBayesianMethod ..................126
3.8.1 Representation about knowledge uncertainty . . . 127
3.8.2 Representation about evidence uncertainty .... 128
3.8.3 Reasoning procedure of the subjective Bayesianmethod ....................131
3.9 Summary ..............................132
References ................................135
Chapter 4. Knowledge-Based Machine Learning 137
4.1 De.nition and Development of Machine Learning ...... 137
4.1.1 De.nition of machine learning ............. 137
4.1.2 Development history of machine learning ...... 141
4.2 Main Strategies and Basic Structure of Machine Learning ..............................146
4.2.1 Main strategies of machine learning ......... 146
4.2.2 Basic structure of the machine learning system...........................148
4.3 InductiveLearning ........................151
4.3.1 Modes and rules of inductive learning ........ 151
4.3.2 Example-basedlearning ................155
4.3.3 Learning from observation and discovery ...... 157
4.4 LearningbyExplanation .....................158
4.4.1 Process and algorithm of explanatory learning . . . 158
4.4.2 Example of explanatory learning ........... 160
4.5 LearningbyAnalogy .......................162
4.5.1 Analogy inference and form of analogy learning ..........................162
4.5.2 Process and research type of analogy learning ..........................164
4.6 ReinforcementLearning .....................166
4.6.1 Overview of reinforcement learning ......... 166
4.6.2 Q-learning ........................170
4.7 Summary ..............................171
References ................................173
Part 2: Data-based Arti.cial Intelligence 175
Chapter 5. Neural Computation 177
5.1 Overview of Computational Intelligence ............ 178
5.2 Research Advances in Arti.cial Neural Networks ...... 180
5.3 Basic Structure of Arti.cial Neural Network .......... 182
5.3.1 Neuron and its characteristics ............. 183
5.3.2 Basic characteristics and structure of ANN ..... 184
5.3.3 Main learning algorithms of ANN ........... 186
5.4 DeepNeuralNetworks ......................187
5.4.1 BriefintroductiontoDNNs ...............187
5.4.2 Common models of deep neural network ...... 189
5.4.3 Structure analysis of convolutional neural network ..........................198
5.5 Summary ..............................201
References ................................202
Chapter 6. Evolutionary Computation 207
6.1 EvolutionaryAlgorithms(EAs) .................208
6.1.1 ThebasicideaofEAs .................208
6.1.2 The research areas and paradigms of EAs ..... 209
6.2 Solving Constrained Optimization Problems byEvolutionaryAlgorithms ...................213
6.2.1 Constrained optimization problems (COPs) and constraint-handling techniques ......... 213
6.2.2 Further analysis of the methods based on multi-objective optimization techniques ..... 215
6.2.3 A multi-objective optimization-based EA forCOPs .........................216
6.3 Solving Multi-objective Optimization Problems byEvolutionaryAlgorithms ...................223
6.3.1 Multi-objective optimization problems (MOPs) andtherelatedde.nitions ...............223
6.3.2 A regularity model-based multi-objective estimation of distribution algorithm (RM-MEDA) .......................224
6.3.3 The drawback of modeling in RM-MEDA ...... 227
6.3.4 An improved RM-MEDA (IRM-MEDA) ........ 231
6.3.5 Experimentalstudy ...................233
6.4 An Application of EA for Descriptor Selection in Quantitative StructureActivity/Property Relationship(QSAR/QSPR)...................239
6.4.1 Background........................239
6.4.2 Weighted sampling PSO-PLS (WS-PSO-PLS) . . . 240
6.4.3 Experimentalstudy ...................244
6.5 Summary ..............................249
References ................................249
Chapter 7. Data-based Machine Learning 255
7.1 LinearRegression.........................255
7.2 DecisionTree ...........................258
7.2.1 Decision tree model and learning ........... 258
7.2.2 Featureselection ....................259
7.2.3 Generation algorithm of decision trees ....... 263
7.2.4 Pruningofdecisiontrees................263
7.3 SupportVectorMachine .....................265
7.3.1 Intervals and support vectors ............. 265
7.3.2 Dualityproblem .....................267
7.3.3 Soft interval and regularization ............ 269
7.3.4 Kernelfunction......................273
7.4 IntegratedLearning ........................276
7.4.1 Randomforest ......................276
7.4.2 Adaboostalgorithm ...................278
7.5 Clustering..............................283
7.5.1 Distancecalculation...................283
7.5.2 Thek-meansclustering.................284
7.5.3 Sampledescription ...................284
7.6 DeepLearning...........................286
7.6.1 De.nition and characteristics of deep learning . . . 287
7.6.2 Deep learning model training and optimization . . . 288
7.6.3 Applications of deep learning ............. 294
7.7 Summary ..............................297
References ................................298
Part 3: Application Examples of Arti.cial Intelligence 301
Chapter 8. Expert System 303
8.1 OverviewofExpertSystems ..................303
8.1.1 De.nition, characteristics, and types of expert systems ..........................303
8.1.2 Structure and construction steps of expert systems ..........................306
8.2 Rule-basedExpertSystem ...................310
8.2.1 Working model and architecture of a rule-basedexpertsystem ...............310
8.2.2 Features of a rule-based expert system ....... 312
8.3 Model-basedExpertSystem ..................314
8.3.1 Proposal of a model-based expert system ..... 315
8.3.2 Expert system based on neural networks ...... 315
8.4 Web-basedExpertSystem ...................318
8.4.1 Structure of the Web-based expert system ..... 318
8.4.2 Example of a Web-based expert system ...... 322
8.5 DesignoftheExpertSystem ..................323
8.6 Expert Systems Based on Machine Learning ........ 325
8.6.1 Introduction to expert systems based onmachinelearning ..................326
8.6.2 Example of expert systems based ondeeplearning.....................328
8.7 Summary ..............................333
References ................................334
Chapter 9. Intelligent Planning 339
9.1 OverviewofIntelligentPlanning ................339
9.1.1 Concept and function of planning ........... 340
9.1.2 Classi.cationofplanning................342
9.2 TaskPlanning ...........................344
9.2.1 Robot planning in the block world .......... 344
9.2.2 Task planning based on the resolution principle ..........................348
9.3 Planning System with Learning Ability ............. 355
9.3.1 Structure and operation modes of the PULP-Isystem......................355
9.3.2 World model and planning results of the PULP-Isystem......................357
9.4 PlanningBasedonExpertSystems ..............359
9.4.1 Structure and planning mechanism of the system...........................359
9.4.2 ROPESrobotplanningsystem ............361
9.5 PathPlanning ...........................366
9.5.1 Main methods of robot path planning ........ 367
9.5.2 Development trends of path planning ........ 369
9.6 Robot Path Planning Based on Ant Colony Algorithm . . . 370
9.6.1 Introduction to the ant colony optimization algorithm .........................371
9.6.2 Path planning based on ant colony algorithm .... 373
9.7 Intelligent Planning Based on Machine Learning ...... 379
9.7.1 Advances in intelligent planning based onmachinelearning ..................379
9.7.2 Autonomous path planning based on deep reinforcement learning for unmanned ships ..... 382
9.8 Conclusion.............................387
9.9 Summary ..............................388
References ................................389
Chapter 10. Intelligent Perception
395
10.1 Introduction to Pattern Recognition .............. 395
10.1.1 Whatispatternrecognition?..............395
10.1.2 The difference between pattern recognition andmachinelearning..................396
10.1.3 Research methods of pattern recognition ...... 397
10.2 Image Analysis and Understanding .............. 399
10.2.1 Imageengineering ...................399
10.2.2 Image processing and image analysis ........ 401
10.2.3 Imageunderstanding ..................405
10.3 The Case of Image Understanding Based onDeepLearning:DenseCap .................413
10.4 Basic Principles and Development of Speech Recognition ............................414
10.4.1 How does speech recognition work? ......... 415
10.4.2 Development of speech recognition ......... 417
10.5 Key Technologies of Speech Recognition ........... 419
10.5.1 Acousticfeatureextraction...............419
10.5.2 Acousticmodel......................422
10.5.3 Languagemodel.....................424
10.5.4 Search algorithms in speech recognition ...... 425
10.5.5 Performanceevaluation ................426
10.5.6 Outlook of speech recognition technology ..... 427
10.6 Case of Speech Recognition Based on Deep Learning: DeepSpeech ...........................429
10.7 Summary ..............................429
References ................................430
Chapter 11. Natural Language Understanding 433
11.1 Overview of Natural Language Understanding ........ 433
11.1.1 Language and language understanding ....... 434
11.1.2 Concept and de.nitions of natural language processing ........................436
11.1.3 Research areas and signi.cance of natural languageprocessing ..................437
11.1.4 Basic methods and advances in research on natural language understanding ......... 441
11.1.5 Levels of the natural language understanding process ..........................448
11.2 LexicalAnalysis ..........................450
11.3 SyntacticAnalysis.........................452
11.3.1 Phrasestructuregrammar ...............452
11.3.2 Chomskysformalgrammar ..............454
11.3.3 Transitionnetwork ....................456
11.3.4 Lexical functional grammar .............. 458
11.4 SemanticAnalysis ........................460
11.5 Automatic Understanding of Sentences ............ 463
11.5.1 Understanding of simple sentences ......... 463
11.5.2 Understanding of complex sentences ........ 466
11.6 CorpusLinguistics ........................468
11.7 Main Models of Natural Language Understanding Systems ..............................471
11.8 Natural Language Processing Based onDeepLearning.........................474
11.8.1 Overview of deep learning-based natural language processing technologies .......... 475
11.8.2 Natural language processing example basedondeeplearning ................482
11.9 Summary ..............................485
References ................................487
Chapter 12. Prospects of Arti.cial Intelligence 493
12.1 The Impact of Arti.cial Intelligence on Humans ....... 494
12.1.1 Great bene.ts of arti.cial intelligence ........ 494
12.1.2 Security issues of arti.cial intelligence ........ 497
12.2 Deep Fusion of Arti.cial Intelligence Technology ...... 502
12.2.1 Fusion of arti.cial intelligence technology inmachinelearning ...................502
12.2.2 Fusion of AI technology in deep reinforcementlearning .................503
12.2.3 Fusion of deep learning and traditional arti.cial intelligence technology ............ 505
12.3 Industrialization of Arti.cial Intelligence ............ 506
12.3.1 Current status of AI industrialization ......... 506
12.3.2 Development trend of AI industrialization ...... 508 References ................................512
Epilogue 515
Index 517
書(shū)單推薦
