Where Does Machine Learning Fit In?

 Machine Learning is a multidisciplinary field and it can be very confusing when you are getting started to differentiate machine learning from the closely related fields of Artificial Intelligence and Data Mining.

In this post you will learn about those fields that are related to machine learning. Specifically, you will learn about the boundaries of the field by learning how machine learning builds on fields of mathematics and artificial intelligence and is used within fields such as data mining and data science.

Foundations

Machine Learning is built on the field of Mathematics and Computer Science. Specifically, machine learning methods are best described using linear and matrix algebra and their behaviours are best understood using the tools of probability and statistics. The fields of Statistics, Probability and Artificial Intelligence that represent the foundational subjects for machine learning.

Probability

The field of probability theory is the study of characterising the likelihood of random events. Probability theory is a branch of mathematics and provides the basis for the field of statistics.

Machine learning methods are often described in the language of probability and there are methods that directly employ probability theories such as Bayes’ Theorem.

Statistics

The field of statistics is the study of methods to collect, analyze, describe and present data. Statistics is a branch of mathematics. The field is concerned with questions like what does the data mean.

Machine learning can be well understood in a statistical framework where learning from training data is taken as a modelling of the structures and relationships in the data. As such, statistical modelling methods are adopted in machine learning but machine learning includes more than statistical modelling methods.

Artificial Intelligence

The field of artificial intelligence is the study and construction of computational systems that do things that humans can do or that do things that we think are intelligent. For example humans can move around an environment, understand what they see and understand language they read and hear, and we have corresponding subfields of robotics, computer vision and natural language processing. A grand master chess champion is considered intelligent, and so chess playing intelligent systems are created. Artificial Intelligence is a branch of computer science. The field is concerned with questions of what is intelligence and how to create intelligences.

Learning is a feature of an intelligent system. As such, Machine Learning is considered a branch of artificial intelligence concerned with the study and construction of systems that are capable of learning.

Progenitors

Algorithms that can learn from data to describe the data and predict outcomes for unseen data are useful for addressing complex problems. As such, machine learning methods are used in applied computer science fields such as Data Mining and Data Science. Additionally, there are related fields of Artificial Intelligence that study intelligent methods that also learn from data and their environment. Examples include Computational Intelligence and Mateheuristics.

Let’s review the related fields of Computational Intelligence, Data Mining and Data Science and learn how machine learning methods applied.

Computational Intelligence

The field of Computational Intelligence is concerned with the study and construction of systems that are easy to specify but result in complex emergent behaviours. Many computational intelligence systems are inspired by natural systems such as evolution, the immune system and the nervous system for subfields such as evolutionary computing, artificial immune systems and artificial neural networks. Computational Intelligence is a branch of artificial intelligence. The field is concerned with questions of explaining how complex emergent behaviours are derived from simple rules and what problems they are best suited to address.

Many computational intelligence systems learn from interactions with their environment and as such have been adopted as machine learning methods.

Data Mining

The field of data mining is the study and construction of systems that discover interesting relationships from large data sets. As such data mining spans both the storage and maintenance of data and the process of making discoveries in the data. Data mining is a process and is also known as knowledge discovery in databases (KDD). Data Mining is a subfield of computer science. The field is concerned with questions of what relationships are interesting and how to best discover them.

Machine learning provides a set of tools used in the data mining process for learning relationships in data that provide the basis of discovery.

Data Science

The field of Data Science is concerned with the practicality of solving complex problems using data. Data science is a subfield of computer science. Data science is the application of the data mining process and the use of machine learning methods in a specific domain. A data scientist is a practitioner of data science.

Like data mining, machine learning provides a set of tools used in data science for learning relationships in data in order to characterise data or make predictions.

Machine learning is related to other fields of mathematics (like decision theory and information theory) and computer science (like operations research and convex optimization).

Resources

I’ve linked to some papers and books if you would like to dig a little deeper.

• Leo Breiman, Statistical Modeling: The Two Cultures, 2001
• Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, 2009
• Andries Engelbrecht, Computational Intelligence: An Introduction, 2007

Are there other fields that you think machine learning is closely related? Do you have a clearer definition for one of the fields described? Leave a comment.