I know its been a while since my first lab entry, but I'm trying to reset....... ohh what the heck, I've just being inconsistent. Truth is I get distracted easily - like a kid who sees flashy toys at the mall. But enough of that, I decided to start learning Machine learning and Artificial intelligence from scratch - I'm talking calculus and linear algebra before we move on to the fancy stuff.

---

Machine Learning Basics

1. Data Representation: Why Vectors and Matrices?

In machine learning, we represent numerical data as vectors. You can think of an individual example in a dataset (like one row in a table) as a D-dimensional vector where each entry corresponds to a specific "feature" or "attribute".

• The Design Matrix (X): To handle many examples at once, we collect these vectors into a matrix. In this format, each row typically represents a different person or data point, and each column describes a different feature, such as height or age.

• Notation: In standard lecture notation, x(i) denotes the input features of the i-th training example, and y(i) denotes the target variable you are trying to predict. Don't worry if any these doesn't make sense I'll break it down after.

2. Vector Spaces and Basis

A vector space is a structured environment where these vectors live. It is defined by "closure," meaning if you add two vectors in that space or scale them by a number, you never leave that space.

• Basis: A basis is a set of linearly independent vectors that "spans" the space. Think of it as a coordinate system.

• Change of Basis: A core task in ML engineering is finding a better coordinate system to represent data. For example, in Dimensionality Reduction, we use Principal Component Analysis (PCA) to find a new basis where the first few vectors capture the most "variance" (information) in the data.

3. Matrix Operations as Linear Mappings

Matrices are more than just tables; they represent linear mappings. When you multiply a vector by a matrix, you are transforming that data—rotating it, stretching it, or projecting it into a different space.

• The Dot Product: This operation measures the similarity between two vectors. In many algorithms, vectors that are "similar" in direction should result in similar outputs.

• System of Equations: Many ML problems are phrased as finding a vector x that satisfies Ax=b. When no perfect solution exists, we use the Moore-Penrose pseudo-inverse to find the "best-fit" solution, which is the foundation of Linear Regression.

4. Engineering Efficiency: Vectorization

A critical skill for an ML engineer is vectorization.

• The Problem: In code, the most "natural" way to handle many data points is a for loop, but this is incredibly slow when processing millions of points.

• The Solution: By using Matrix Algebra and optimized libraries (like NumPy), we perform calculations on entire blocks of data simultaneously. This leverages the parallelism of modern hardware like GPUs, which is essential for training deep neural networks.

---

Lab Digest

• So basically what Data representation means is, in machine learning data must be stored numerically. The same way computers store information in 1s and 0s. So text, images or even audio will eventually be represented by numbers. A vector (one data point) is a list or say a group of numbers that gets stored as a whole. A good example would say you have a dataset and each row gets stored as a vector, and if that row has 3 values - that vector will have 3 dimensions, because there are 3 features/values.

If there are many features in the row that means we have many (data points) - it's called a Design Matrix. All those values get converted into a matrix.

ML notation - in machine learning we write;

x(1) = first student

x(2) = second student

x(3) = third student

So the value we want to predict is:

y = exam score

then the dataset becomes:

(x, y)

• When we talk about Vector Spaces and Basis, its basically just a place where vectors use math operations - to which there are two rules; 1. You can add vectors. 2. You can multiply them. A good example would be - [1,2] + [3,4] = [4,6]. Basis (the coordinate system) - A basis is the set of vectors that define how we measure the space. Example in 2D; Standard basis - e1 = [1,0] e2 = [0,1].These define the normal x and y axes.

  Every vector can be written as a combination of them.

  Example: [3,4] = 3e1 + 4e2

ML changes basis because sometimes the current coordinate system is bad for understanding the data. So we change it. The most famous method is: Principal Component Analysis (PCA).

Instead of measuring along: height age study hours PCA finds new axes like: overall body size lifestyle factor etc. These new axes capture the most variation in the data. So PCA is basically: Find a better coordinate system for the data. This is called dimensionality reduction.

• Now we have Matrix operations as transformations. A matrix is a transformation machine, if that makes sense. Geometric meaning. Matrix multiplication can: rotate vectors, stretch them, compress them, project them into another dimension. A good example would be a matrix that could rotate every point 45 degrees. Another could stretch everything twice as wide. Neural networks are basically many transformations stacked together.

Dot Product (very important)

The dot product measures similarity.

Formula:

a⋅b = a1b1 + a2b2+...+anbna \cdot b = a_1b_1 + a_2b_2 + ... + a_nb_na⋅b=a1​b1​+a2​b2​+...+an​bn​ Meaning: Multiply corresponding entries and sum them.

Why dot product matters, in machine learning: : prediction = w · x. Where: w = weights x = features. Basically all this tells us is how strongly the input matches the model. Example uses: linear regression, logistic regression, neural networks, transformers, recommendation systems.

• Finally we have the Systems of equations in ML. Many ML models solve: Ax = B. Which simply means - A = data, x = unknown parameters, b = target values. In linear regression - Xw = y. But usually the system has no exact solution. So we find the best approximation. Moving to Vectorization. This solves the problem of slow code by using matrix algebra and optimized libraries like Numpy.

Traditionally, if you have 10 million data points, writing the code would look something like this;

for i in range(10000000):

prediction[i] = dot(w, x[i]). This is slow and highly inefficient. Then the vectorized version would look like thsi instead: predictions = X @ w. One line, and it calculates everything simultaneously. All modern libraries like Numpy, Pytorch, Tensorflow, use optimized C/C++ and GPU code. GPUs are built for massive parallel matrix operations. Which is exactly what neural networks need.

---

Lab Insight

Machine learning is basically:

Data → Vectors
Many Vectors → Matrices
Matrices → Transformations
Training → Finding the best transformation

Neural networks are just:

vector → matrix multiply → activation
vector → matrix multiply → activation
vector → matrix multiply → output

Over and Over again.

---

Lab Summary

So basically, this is the math and logic behind machine learning. Understand these concepts and every other part of ML becomes clear. Like Neural networks, Transformers, Diffusion models, Recommendation systems etc.

The next post in the series will be on Calculus Essentials, but I'll have to go through that first, before I document what I learnt so you can learn too.

Sorry if my explanations are a bit unclear, I'm taking a technical writing course so I'll try and do better with time. But if there's anything you don't understand leave a comment and I'll see what I can do. Till next time peeps.