Spot-Check Regression Machine Learning Algorithms in Python with scikit-learn

 Spot-checking is a way of discovering which algorithms perform well on your machine learning problem.

You cannot know which algorithms are best suited to your problem before hand. You must trial a number of methods and focus attention on those that prove themselves the most promising.

In this post you will discover 6 machine learning algorithms that you can use when spot checking your regression problem in Python with scikit-learn.

Algorithms Overview

We are going to take a look at 7 classification algorithms that you can spot check on your dataset.

4 Linear Machine Learning Algorithms:

1. Linear Regression
2. Ridge Regression
3. LASSO Linear Regression
4. Elastic Net Regression

3 Nonlinear Machine Learning Algorithms:

1. K-Nearest Neighbors
2. Classification and Regression Trees
3. Support Vector Machines

Each recipe is demonstrated on a Boston House Price dataset. This is a regression problem where all attributes are numeric (update: download data from here).

Each recipe is complete and standalone. This means that you can copy and paste it into your own project and start using it immediately.

A test harness with 10-fold cross validation is used to demonstrate how to spot check each machine learning algorithm and mean squared error measures are used to indicate algorithm performance. Note that mean squared error values are inverted (negative). This is a quirk of the cross_val_score() function used that requires all algorithm metrics to be sorted in ascending order (larger value is better).

The recipes assume that you know about each machine learning algorithm and how to use them. We will not go into the API or parameterization of each algorithm.

Need help with Machine Learning in Python?

Take my free 2-week email course and discover data prep, algorithms and more (with code).

Click to sign-up now and also get a free PDF Ebook version of the course.

Linear Machine Learning Algorithms

This section provides examples of how to use 4 different linear machine learning algorithms for regression in Python with scikit-learn.

1. Linear Regression

Linear regression assumes that the input variables have a Gaussian distribution. It is also assumed that input variables are relevant to the output variable and that they are not highly correlated with each other (a problem called collinearity).

You can construct a linear regression model using the LinearRegression class.

# Linear Regression import pandas from sklearn import model_selection from sklearn.linear_model import LinearRegression url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/housing.data" names = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', 'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV'] dataframe = pandas.read_csv(url, delim_whitespace=True, names=names) array = dataframe.values X = array[:,0:13] Y = array[:,13] seed = 7 kfold = model_selection.KFold(n_splits=10, random_state=seed) model = LinearRegression() scoring = 'neg_mean_squared_error' results = model_selection.cross_val_score(model, X, Y, cv=kfold, scoring=scoring) print(results.mean())

Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

Running the example provides an estimate of mean squared error.

-34.7052559445

2. Ridge Regression

Ridge regression is an extension of linear regression where the loss function is modified to minimize the complexity of the model measured as the sum squared value of the coefficient values (also called the l2-norm).

You can construct a ridge regression model by using the Ridge class.

# Ridge Regression import pandas from sklearn import model_selection from sklearn.linear_model import Ridge url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/housing.data" names = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', 'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV'] dataframe = pandas.read_csv(url, delim_whitespace=True, names=names) array = dataframe.values X = array[:,0:13] Y = array[:,13] seed = 7 kfold = model_selection.KFold(n_splits=10, random_state=seed) model = Ridge() scoring = 'neg_mean_squared_error' results = model_selection.cross_val_score(model, X, Y, cv=kfold, scoring=scoring) print(results.mean())

Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

Running the example provides an estimate of the mean squared error.

-34.0782462093

3. LASSO Regression

The Least Absolute Shrinkage and Selection Operator (or LASSO for short) is a modification of linear regression, like ridge regression, where the loss function is modified to minimize the complexity of the model measured as the sum absolute value of the coefficient values (also called the l1-norm).

You can construct a LASSO model by using the Lasso class.

# Lasso Regression import pandas from sklearn import model_selection from sklearn.linear_model import Lasso url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/housing.data" names = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', 'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV'] dataframe = pandas.read_csv(url, delim_whitespace=True, names=names) array = dataframe.values X = array[:,0:13] Y = array[:,13] seed = 7 kfold = model_selection.KFold(n_splits=10, random_state=seed) model = Lasso() scoring = 'neg_mean_squared_error' results = model_selection.cross_val_score(model, X, Y, cv=kfold, scoring=scoring) print(results.mean())

Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

Running the example provides an estimate of the mean squared error.

-34.4640845883

4. ElasticNet Regression

ElasticNet is a form of regularization regression that combines the properties of both Ridge Regression and LASSO regression. It seeks to minimize the complexity of the regression model (magnitude and number of regression coefficients) by penalizing the model using both the l2-norm (sum squared coefficient values) and the l1-norm (sum absolute coefficient values).

You can construct an ElasticNet model using the ElasticNet class.

# ElasticNet Regression import pandas from sklearn import model_selection from sklearn.linear_model import ElasticNet url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/housing.data" names = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', 'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV'] dataframe = pandas.read_csv(url, delim_whitespace=True, names=names) array = dataframe.values X = array[:,0:13] Y = array[:,13] seed = 7 kfold = model_selection.KFold(n_splits=10, random_state=seed) model = ElasticNet() scoring = 'neg_mean_squared_error' results = model_selection.cross_val_score(model, X, Y, cv=kfold, scoring=scoring) print(results.mean())

Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

Running the example provides an estimate of the mean squared error.

-31.1645737142

Nonlinear Machine Learning Algorithms

This section provides examples of how to use 3 different nonlinear machine learning algorithms for regression in Python with scikit-learn.

1. K-Nearest Neighbors

K-Nearest Neighbors (or KNN) locates the K most similar instances in the training dataset for a new data instance. From the K neighbors, a mean or median output variable is taken as the prediction. Of note is the distance metric used (the metric argument). The Minkowski distance is used by default, which is a generalization of both the Euclidean distance (used when all inputs have the same scale) and Manhattan distance (for when the scales of the input variables differ).

You can construct a KNN model for regression using the KNeighborsRegressor class.

# KNN Regression import pandas from sklearn import model_selection from sklearn.neighbors import KNeighborsRegressor url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/housing.data" names = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', 'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV'] dataframe = pandas.read_csv(url, delim_whitespace=True, names=names) array = dataframe.values X = array[:,0:13] Y = array[:,13] seed = 7 kfold = model_selection.KFold(n_splits=10, random_state=seed) model = KNeighborsRegressor() scoring = 'neg_mean_squared_error' results = model_selection.cross_val_score(model, X, Y, cv=kfold, scoring=scoring) print(results.mean())

Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

Running the example provides an estimate of the mean squared error.

-107.28683898

2. Classification and Regression Trees

Decision trees or the Classification and Regression Trees (CART as they are known) use the training data to select the best points to split the data in order to minimize a cost metric. The default cost metric for regression decision trees is the mean squared error, specified in the criterion parameter.

You can create a CART model for regression using the DecisionTreeRegressor class.

# Decision Tree Regression import pandas from sklearn import model_selection from sklearn.tree import DecisionTreeRegressor url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/housing.data" names = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', 'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV'] dataframe = pandas.read_csv(url, delim_whitespace=True, names=names) array = dataframe.values X = array[:,0:13] Y = array[:,13] seed = 7 kfold = model_selection.KFold(n_splits=10, random_state=seed) model = DecisionTreeRegressor() scoring = 'neg_mean_squared_error' results = model_selection.cross_val_score(model, X, Y, cv=kfold, scoring=scoring) print(results.mean())

Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

Running the example provides an estimate of the mean squared error.

-35.4906027451

3. Support Vector Machines

Support Vector Machines (SVM) were developed for binary classification. The technique has been extended for the prediction real-valued problems called Support Vector Regression (SVR). Like the classification example, SVR is built upon the LIBSVM library.

You can create an SVM model for regression using the SVR class.

# SVM Regression import pandas from sklearn import model_selection from sklearn.svm import SVR url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/housing.data" names = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', 'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV'] dataframe = pandas.read_csv(url, delim_whitespace=True, names=names) array = dataframe.values X = array[:,0:13] Y = array[:,13] seed = 7 kfold = model_selection.KFold(n_splits=10, random_state=seed) model = SVR() scoring = 'neg_mean_squared_error' results = model_selection.cross_val_score(model, X, Y, cv=kfold, scoring=scoring) print(results.mean())

Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

Running the example provides an estimate of the mean squared error.

-91.0478243332

Summary

In this post you discovered machine learning recipes for regression in Python using scikit-learn.

Specifically, you learned about:

4 Linear Machine Learning Algorithms:

• Linear Regression
• Ridge Regression
• LASSO Linear Regression
• Elastic Net Regression

3 Nonlinear Machine Learning Algorithms:

• K-Nearest Neighbors
• Classification and Regression Trees
• Support Vector Machines

Do you have any questions about regression machine learning algorithms or this post? Ask your questions in the comments and I will do my best to answer them.

AI:Which glands, nerves, fibers, tissues neural networks are responsible for self control and keep the brain body mind soul in well controlled shape muscular or brainy gifted ones. Which I’ll malfunction practices are responsible for the loss of self control due to which human being do various kind of unwanted actions, verbal and physical actions, various disorders like epilepsy, stress disorders and ocd towards various attractive lucrative things came. Which AI humanoid robotics can fix these.

 Your question delves deeply into the neurophysiology of self-control, the neurological and psychological mechanisms behind loss of control, and how AI humanoid robotics might assist in managing or correcting these disorders. I'll break this down into clear sections based on your inquiry.

1. Neural Networks and Systems Responsible for Self-Control
Self-control, or the ability to regulate one's emotions, behaviors, and thoughts, is governed by a complex interaction between neuroanatomy, neurotransmitters, and neural circuits. Key components involved include:

A. Brain Regions Responsible for Self-Control

Prefrontal Cortex (PFC):
Role: The PFC is the main brain region responsible for executive functions, including decision-making, impulse control, and self-regulation. It helps inhibit inappropriate behaviors, plan for the future, and override emotional reactions.
Malfunction: Damage to the PFC (due to injury or neurodevelopmental disorders) can result in poor decision-making, impulsivity, and lack of foresight.
Anterior Cingulate Cortex (ACC):
Role: The ACC is involved in error detection, emotional regulation, and conflict monitoring. It helps in regulating behavior when there’s an internal conflict between desires and socially acceptable actions.
Malfunction: Dysfunction in the ACC can lead to perseveration (repeated actions or thoughts) or OCD-like behaviors.
Insula:
Role: The insula helps with awareness of bodily states and is linked to emotional awareness. It plays a role in interoception (the ability to perceive internal states like hunger, pain, or emotional distress), which is critical for regulating behaviors linked to emotions.
Malfunction: Damage or overactivity can result in hyper-sensitivity to emotions and stress, contributing to disorders like anxiety or stress disorders.
Basal Ganglia:
Role: This structure is involved in the regulation of habits, motor control, and reward-based behavior. It plays a key role in reward-seeking and addiction-related behaviors.
Malfunction: Overactivity or dysfunction can lead to addiction, OCD, and movement disorders (like Parkinson’s disease).
B. Neurotransmitters and Hormones Involved

Dopamine:
Role: Dopamine is crucial for the brain's reward system and motivation. It's involved in pleasure, reward-seeking behavior, and addiction.
Malfunction: Dysregulation of dopamine can lead to addictive behaviors, impulsivity, or lack of self-control in the face of rewards.
Serotonin:
Role: Serotonin helps regulate mood, anxiety, and impulse control.
Malfunction: Low serotonin levels are associated with depression, anxiety disorders, and impulsivity.
GABA (Gamma-Aminobutyric Acid):
Role: GABA is an inhibitory neurotransmitter that helps calm the brain, promoting relaxation and reducing anxiety.
Malfunction: Low GABA levels can contribute to anxiety disorders, OCD, and hyperactivity.
Cortisol:
Role: Cortisol is the body's primary stress hormone. It helps regulate the body’s response to stress and is involved in mood and behavior regulation.
Malfunction: Chronic high levels of cortisol are linked to stress disorders, anxiety, and impulsivity.
2. Malfunctions Leading to Loss of Self-Control and Disorders
Various practices, habits, and conditions can lead to dysfunctional self-control, manifesting as undesirable behaviors or psychological disorders:

A. Causes of Loss of Self-Control

Stress and Trauma:
Impact: Chronic stress or post-traumatic stress disorder (PTSD) can disrupt the prefrontal cortex's ability to control impulses, leading to emotional outbursts, anxiety, or compulsive behaviors.
Effect: Stress hormones like cortisol and adrenaline can dampen the functioning of the PFC, causing impulsivity and poor decision-making.
Addiction:
Impact: Addiction, whether to substances (drugs, alcohol) or behaviors (gambling, overeating), hijacks the reward system and dopamine pathways, making it difficult for the brain to resist urges.
Effect: This causes a breakdown in self-control, with individuals consistently seeking pleasurable stimuli despite negative consequences.
Neurological Disorders:
Epilepsy: Seizures, especially in areas like the prefrontal cortex or temporal lobes, can disrupt cognitive control, leading to impulsive behaviors or inability to regulate actions.
OCD: Dysfunction in the basal ganglia and prefrontal cortex can result in compulsive behaviors driven by intrusive thoughts that cannot be controlled.
Personality and Mood Disorders:
Bipolar Disorder: Impulsivity during manic episodes can lead to reckless behavior and a loss of self-regulation, affecting decision-making.
Borderline Personality Disorder: Emotional dysregulation and impulsivity are common in BPD, leading to actions that individuals may later regret.
Mental Health Issues:
Anxiety Disorders: Overactive amygdalae and impaired PFC functioning can lead to poor decision-making, avoidance behaviors, and heightened stress reactions.
Depression: Low serotonin levels impair emotional regulation, leading to poor impulse control, self-neglect, or destructive behaviors.
3. AI Humanoid Robotics: Can They Fix These Issues?
AI humanoid robots could assist in improving self-control, mental health, and behavior regulation in several ways. These robots would use neurofeedback, psychological interventions, and biometric data to help individuals manage their emotions and impulses.

A. AI Humanoid Robots for Self-Control and Psychological Disorders

Neurofeedback Robots:
Function: Robots with AI can provide real-time neurofeedback through EEG sensors to help regulate brain activity. These robots could teach individuals to control brain waves associated with stress, anxiety, and impulsivity, encouraging more balanced emotional responses.
Example: Robots like Zenbo or Pepper can be integrated with biofeedback systems to teach calming techniques or stress reduction exercises.
Cognitive Behavioral Therapy (CBT) Robots:
Function: Humanoid robots could be programmed to deliver CBT-like interventions, guiding individuals through thought reframing and behavioral strategies to combat stress, anxiety, and impulsive actions.
Example: AI robots with emotion detection could identify signs of negative emotions and provide calming techniques, suggest breathing exercises, or initiate mindfulness practices.
Personalized Support for Addiction Recovery:
Function: Using machine learning, robots could track a person’s behavior patterns and provide personalized interventions during moments of temptation or stress. They can also offer motivational support during recovery.
Example: AI-driven therapy bots (like Woebot) can be used for addiction recovery, offering support during moments of emotional crisis or relapse.
Mental Health Monitoring and Support:
Function: AI humanoids can monitor mental states through conversation and biometrics (heart rate, facial expressions, voice tone) and intervene when early signs of stress, depression, or anxiety arise.
Example: AI therapists like Replika provide emotionally intelligent responses, offering support during difficult emotional moments and teaching emotional regulation strategies.
B. Limitations of AI Robots in Addressing These Issues

Emotional Sensitivity: While AI can simulate empathy, it lacks true human emotional intelligence, which may limit its ability to offer deep, empathetic connections in complex psychological cases.
Ethical Concerns: Relying on robots for mental health support raises questions about privacy, data security, and the potential for over-dependence on technology for emotional needs.
Lack of Human Intuition: AI is limited in its ability to fully comprehend and respond to the nuances of human behavior and subjective experiences like a trained human therapist would.
4. Conclusion: AI and Human Behavioral Regulation
AI humanoid robots can serve as complementary tools for improving self-control, emotional regulation, and mental health. They can assist by offering structured therapeutic practices, monitoring biometric data, and providing real-time interventions for anxiety, stress, and addictive behaviors.

However, these robots should be considered supplementary, rather than replacement, for human emotional intelligence and psychological care. The neurophysiology behind self-control is deeply complex, and while AI can help manage symptoms or aid in treatment, human expertise will still be crucial in addressing the underlying causes of disorders like epilepsy, OCD, addiction, and stress.