Linear Algebra Basics for AI/ML
Difficulty: Beginner | Time: 45-60 minutes | Key Concepts: Vectors, Matrices, Operations
What is Linear Algebra?
Linear algebra is the mathematics of vectors and matrices. In machine learning, data is represented as vectors and matrices, and neural networks are fundamentally matrix operations.
1. Vectors Explained
What is a Vector?
A vector is an ordered list of numbers. It has both magnitude (size) and direction.
Examples:
# 2D vector
v = [3, 4]
# 3D vector
w = [1, 2, 3]
# In machine learning, a vector represents a data point
person_features = [age=25, height=180, weight=75]
Vector Operations
Addition
v = [1, 2, 3]
w = [4, 5, 6]
v + w = [1+4, 2+5, 3+6] = [5, 7, 9]
Scalar Multiplication
v = [2, 3]
2 ร v = [2ร2, 2ร3] = [4, 6]
Dot Product (Most Important!)
v = [1, 2, 3]
w = [4, 5, 6]
v ยท w = (1ร4) + (2ร5) + (3ร6) = 4 + 10 + 18 = 32
Why is dot product important? It measures similarity between vectors and is used in neural network computations.
2. Matrices Explained
What is a Matrix?
A matrix is a 2D array of numbers arranged in rows and columns.
Example:
A = [1 2 3]
[4 5 6]
[7 8 9]
This is a 3ร3 matrix (3 rows, 3 columns)
Matrix Representation of Data
# 4 people, 3 features each (age, height, weight)
people_data = [25 180 75] โ person 1
[30 175 68] โ person 2
[22 165 55] โ person 3
[28 180 70] โ person 4
3. Matrix Operations
Matrix Addition
A = [1 2] B = [5 6] A + B = [1+5 2+6] = [6 8]
[3 4] [7 8] [3+7 4+8] [10 12]
Matrix Multiplication (Most Important!)
A (2ร3) ร B (3ร2) = C (2ร2)
A = [1 2 3] B = [7 8] C = [1ร7+2ร9+3ร11 1ร8+2ร10+3ร12]
[4 5 6] [9 10] [4ร7+5ร9+6ร11 4ร8+5ร10+6ร12]
[11 12]
Result: C = [58 64]
[139 154]
Transpose
A = [1 2] A^T = [1 3 5]
[3 4] [2 4 6]
[5 6]
Rows become columns and columns become rows
4. Special Matrices
Identity Matrix (I)
I = [1 0 0]
[0 1 0]
[0 0 1]
Property: A ร I = A (like multiplying by 1)
Zero Matrix
0 = [0 0]
[0 0]
Diagonal Matrix
D = [5 0 0]
[0 3 0]
[0 0 2]
Only diagonal has non-zero values
5. Important Concepts
Matrix Determinant
For a 2ร2 matrix A = [[a, b], [c, d]], the determinant is: det(A) = ad – bc
It tells us if a matrix can be inverted.
Matrix Inverse
For a square matrix A, its inverse A^(-1) satisfies: A ร A^(-1) = I
It’s like division in matrix mathematics.
Eigenvalues and Eigenvectors
Special vectors v and scalars ฮป such that: A ร v = ฮป ร v
Important for dimensionality reduction and PCA.
6. In Machine Learning Context
Neural Network Forward Pass
Input vector: x = [1, 2, 3]
Weights matrix: W = [[0.1, 0.2],
[0.3, 0.4],
[0.5, 0.6]]
Output: y = x ยท W (matrix multiplication!)
Data Representation
Dataset with 1000 images (28ร28 pixels) = Matrix of 1000ร784
Each row: one image flattened
Each column: one pixel across all images
7. Python Examples
Using NumPy
import numpy as np
# Create vectors
v = np.array([1, 2, 3])
w = np.array([4, 5, 6])
# Vector operations
print(v + w) # [5 7 9]
print(np.dot(v, w)) # 32 (dot product)
# Create matrices
A = np.array([[1, 2], [3, 4], [5, 6]])
B = np.array([[7, 8], [9, 10]])
# Matrix multiplication
C = np.dot(A, B)
print(C)
# Transpose
print(A.T)
# Matrix properties
print(np.linalg.det(np.array([[1, 2], [3, 4]]))) # Determinant
print(np.linalg.inv(np.array([[1, 2], [3, 4]]))) # Inverse
8. Practice Problems
Problem 1
Given v = [2, 3] and w = [4, 1], calculate v ยท w
Answer: (2ร4) + (3ร1) = 8 + 3 = 11
Problem 2
What is the shape of the result when multiplying a (3ร2) matrix by a (2ร4) matrix?
Answer: (3ร4)
Problem 3
What is the transpose of [[1, 2, 3], [4, 5, 6]]?
Answer: [[1, 4], [2, 5], [3, 6]]
Key Takeaways
- Vectors are ordered lists of numbers
- Matrices are 2D arrays used to represent data
- Dot product measures similarity between vectors
- Matrix multiplication is fundamental to neural networks
- Transpose flips rows and columns
- Matrix operations are the foundation of deep learning
Next: Learn Calculus
Once you’re comfortable with linear algebra, move on to Calculus for Machine Learning to understand optimization.
