Calculating the length or magnitude of vectors is often required either directly as a regularization method in machine learning, or as part of broader vector or matrix operations.
In this tutorial, you will discover the different ways to calculate vector lengths or magnitudes, called the vector norm.
After completing this tutorial, you will know:
- The L1 norm that is calculated as the sum of the absolute values of the vector.
- The L2 norm that is calculated as the square root of the sum of the squared vector values.
- The max norm that is calculated as the maximum vector values.
Vector Norm
Calculating the size or length of a vector is often required either directly or as part of a broader vector or vector-matrix operation.
The length of the vector is referred to as the vector norm or the vector’s magnitude.
The length of a vector is a nonnegative number that describes the extent of the vector in space, and is sometimes referred to as the vector’s magnitude or the norm.
— Page 112, No Bullshit Guide To Linear Algebra, 2017
The length of the vector is always a positive number, except for a vector of all zero values. It is calculated using some measure that summarizes the distance of the vector from the origin of the vector space. For example, the origin of a vector space for a vector with 3 elements is (0, 0, 0).
Notations are used to represent the vector norm in broader calculations and the type of vector norm calculation almost always has its own unique notation.
We will take a look at a few common vector norm calculations used in machine learning.
Calculating the size or length of a vector is often required either directly or as part of a broader vector or vector-matrix operation.
The length of the vector is referred to as the vector norm or the vector’s magnitude.
The length of a vector is a nonnegative number that describes the extent of the vector in space, and is sometimes referred to as the vector’s magnitude or the norm.
— Page 112, No Bullshit Guide To Linear Algebra, 2017
The length of the vector is always a positive number, except for a vector of all zero values. It is calculated using some measure that summarizes the distance of the vector from the origin of the vector space. For example, the origin of a vector space for a vector with 3 elements is (0, 0, 0).
Notations are used to represent the vector norm in broader calculations and the type of vector norm calculation almost always has its own unique notation.
We will take a look at a few common vector norm calculations used in machine learning.
Vector L1 Norm
The length of a vector can be calculated using the L1 norm, where the 1 is a superscript of the L, e.g. L^1.
The notation for the L1 norm of a vector is ||v||1, where 1 is a subscript. As such, this length is sometimes called the taxicab norm or the Manhattan norm.
The L1 norm is calculated as the sum of the absolute vector values, where the absolute value of a scalar uses the notation |a1|. In effect, the norm is a calculation of the Manhattan distance from the origin of the vector space.
The L1 norm of a vector can be calculated in NumPy using the norm() function with a parameter to specify the norm order, in this case 1.
First, a 1×3 vector is defined, then the L1 norm of the vector is calculated.
Running the example first prints the defined vector and then the vector’s L1 norm.
The L1 norm is often used when fitting machine learning algorithms as a regularization method, e.g. a method to keep the coefficients of the model small, and in turn, the model less complex.
The length of a vector can be calculated using the L1 norm, where the 1 is a superscript of the L, e.g. L^1.
The notation for the L1 norm of a vector is ||v||1, where 1 is a subscript. As such, this length is sometimes called the taxicab norm or the Manhattan norm.
The L1 norm is calculated as the sum of the absolute vector values, where the absolute value of a scalar uses the notation |a1|. In effect, the norm is a calculation of the Manhattan distance from the origin of the vector space.
The L1 norm of a vector can be calculated in NumPy using the norm() function with a parameter to specify the norm order, in this case 1.
First, a 1×3 vector is defined, then the L1 norm of the vector is calculated.
Running the example first prints the defined vector and then the vector’s L1 norm.
The L1 norm is often used when fitting machine learning algorithms as a regularization method, e.g. a method to keep the coefficients of the model small, and in turn, the model less complex.
Vector L2 Norm
The length of a vector can be calculated using the L2 norm, where the 2 is a superscript of the L, e.g. L^2.
The notation for the L2 norm of a vector is ||v||2 where 2 is a subscript.
The L2 norm calculates the distance of the vector coordinate from the origin of the vector space. As such, it is also known as the Euclidean norm as it is calculated as the Euclidean distance from the origin. The result is a positive distance value.
The L2 norm is calculated as the square root of the sum of the squared vector values.
The L2 norm of a vector can be calculated in NumPy using the norm() function with default parameters.
First, a 1×3 vector is defined, then the L2 norm of the vector is calculated.
Running the example first prints the defined vector and then the vector’s L2 norm.
Like the L1 norm, the L2 norm is often used when fitting machine learning algorithms as a regularization method, e.g. a method to keep the coefficients of the model small and, in turn, the model less complex.
By far, the L2 norm is more commonly used than other vector norms in machine learning.
The length of a vector can be calculated using the L2 norm, where the 2 is a superscript of the L, e.g. L^2.
The notation for the L2 norm of a vector is ||v||2 where 2 is a subscript.
The L2 norm calculates the distance of the vector coordinate from the origin of the vector space. As such, it is also known as the Euclidean norm as it is calculated as the Euclidean distance from the origin. The result is a positive distance value.
The L2 norm is calculated as the square root of the sum of the squared vector values.
The L2 norm of a vector can be calculated in NumPy using the norm() function with default parameters.
First, a 1×3 vector is defined, then the L2 norm of the vector is calculated.
Running the example first prints the defined vector and then the vector’s L2 norm.
Like the L1 norm, the L2 norm is often used when fitting machine learning algorithms as a regularization method, e.g. a method to keep the coefficients of the model small and, in turn, the model less complex.
By far, the L2 norm is more commonly used than other vector norms in machine learning.
Vector Max Norm
The length of a vector can be calculated using the maximum norm, also called max norm.
Max norm of a vector is referred to as L^inf where inf is a superscript and can be represented with the infinity symbol. The notation for max norm is ||x||inf, where inf is a subscript.
The max norm is calculated as returning the maximum value of the vector, hence the name.
The max norm of a vector can be calculated in NumPy using the norm() function with the order parameter set to inf.
First, a 1×3 vector is defined, then the max norm of the vector is calculated.
Running the example first prints the defined vector and then the vector’s max norm.
Max norm is also used as a regularization in machine learning, such as on neural network weights, called max norm regularization.
The length of a vector can be calculated using the maximum norm, also called max norm.
Max norm of a vector is referred to as L^inf where inf is a superscript and can be represented with the infinity symbol. The notation for max norm is ||x||inf, where inf is a subscript.
The max norm is calculated as returning the maximum value of the vector, hence the name.
The max norm of a vector can be calculated in NumPy using the norm() function with the order parameter set to inf.
First, a 1×3 vector is defined, then the max norm of the vector is calculated.
Running the example first prints the defined vector and then the vector’s max norm.
Max norm is also used as a regularization in machine learning, such as on neural network weights, called max norm regularization.
Extensions
This section lists some ideas for extending the tutorial that you may wish to explore.
- Create 5 examples using each operation using your own data.
- Implement each matrix operation manually for matrices defined as lists of lists.
- Search machine learning papers and find 1 example of each operation being used.
If you explore any of these extensions, I’d love to know.
This section lists some ideas for extending the tutorial that you may wish to explore.
- Create 5 examples using each operation using your own data.
- Implement each matrix operation manually for matrices defined as lists of lists.
- Search machine learning papers and find 1 example of each operation being used.
If you explore any of these extensions, I’d love to know.
Further Reading
This section provides more resources on the topic if you are looking to go deeper.
This section provides more resources on the topic if you are looking to go deeper.
Books
- Introduction to Linear Algebra, 2016.
- Chapter 2, Linear Algebra, Deep Learning, 2016.
- Introduction to Linear Algebra, 2016.
- Chapter 2, Linear Algebra, Deep Learning, 2016.
API
Articles
Summary
In this tutorial, you discovered the different ways to calculate vector lengths or magnitudes, called the vector norm.
Specifically, you learned:
- The L1 norm that is calculated as the sum of the absolute values of the vector.
- The L2 norm that is calculated as the square root of the sum of the squared vector values.
- The max norm that is calculated as the maximum vector values.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
In this tutorial, you discovered the different ways to calculate vector lengths or magnitudes, called the vector norm.
Specifically, you learned:
- The L1 norm that is calculated as the sum of the absolute values of the vector.
- The L2 norm that is calculated as the square root of the sum of the squared vector values.
- The max norm that is calculated as the maximum vector values.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
![](https://blogger.googleusercontent.com/img/proxy/AVvXsEgiYV_qJof_iG3Vfme-VO0SYYPzs1aybB5LTgE0xw-lYZ-B80MJMw9p2cefwChHUmES3FgIa_Zr3uk3tonjksLNddOSOTJjWre5kw84WPYA-nR_9fs3ldFzFiBAO2-0-cl-lRVmFBj5YJIlCoKkZqRMjyU7g00NqvKpPg=s0-d-e1-ft"")
No comments:
Post a Comment