A roadmap for learning deep learning math by implementing linear regression in plain C++, with Python side by side. See what each of the 11 parts covers and where to start.
Most deep learning tutorials start with Python and NumPy. A few lines train a model and draw a chart, but what actually happens inside those few lines stays hidden. This deep learning math in C++ series writes that computation out line by line in C++, so the math behind training becomes something you can watch.
The short answer
This series implements linear regression in C++17 with no external libraries, and explains gradient descent, learning rate, backpropagation, mini-batches, and feature scaling using numbers and charts from real runs. Instead of memorizing formulas, you check “this is what happened when I ran it” and build an intuition for how training works.
What this page answers
- Why implement deep learning math in C++ at all
- How the 11 parts connect
- What you need before reading
- Where a first-time reader should start
Why implement deep learning math in C++
Python is convenient, and convenience hides things. Arrays of different lengths are matched for you, frameworks compute derivatives for you, and a chart is one line away. You get results quickly, but when someone asks why the model learns, the explanation often stalls.
In C++ none of that is free. You decide how arrays of different lengths combine, you write the derivative you worked out by hand, and you pick the engine that generates random numbers. It is more work, and that is exactly why the computation that makes learning happen is fully visible.
C++ is not a better language for deep learning. In practice you should still use Python and a framework. But while you are learning the principles, a language that makes you write every step is a good teacher.
What makes this series different
1. Every number comes from a real run
The learned values, update counts, and loss values in each post come from actual runs. The training logs of the original C++ examples were recomputed in Python and matched to a relative error below 10⁻¹⁰, and the C++ and Python code shown in each post were both run to confirm they print the same output.
2. Charts come from the same runs
Charts are drawn from training logs that the C++ examples saved as CSV, or from Python runs of the same computation as the code in the post. The curves and the numbers in the text never disagree.
3. C++ and Python side by side
Each computation is shown in both C++ and Python. If you know Python, you can compare it with code you already read and see what C++ makes explicit. If you write C++, you get practice reading the Python that most deep learning material uses.
4. Only the math you need
Each part uses one or two key formulas, and every formula is followed by the line of code it becomes.
Key terms used throughout the series
| Term | One-line meaning |
|---|---|
| Parameter | A value inside the model that training changes |
| Weight | A parameter multiplied by an input. For a line, the slope |
| Bias | A parameter added regardless of the input. For a line, the intercept |
| Loss / cost | The error of one sample / the average error over several samples |
| Gradient | The direction and size of the change in loss when a parameter changes |
| Gradient descent | Training by moving parameters a little against the gradient |
| Learning rate | How large each move is |
| Epoch | One full pass over the training data |
| Batch | The group of samples used for one update |
| Backpropagation | Computing gradients by walking a computational graph backwards |
Part 1 explains each term with a real experiment.
The series map
The series has two stages. Stage 1 uses the simplest model with a single slope, ŷ = θx. Stage 2 adds an intercept (bias): ŷ = θ1·x + θ0. Going from one parameter to two is enough to bring in contour plots, zigzag paths, and feature scaling.
| Part | Title | Key question |
|---|---|---|
| 0 | Series roadmap (this page) | What do we learn, and in what order? |
| 1 | Deep Learning Linear Regression: Why It Comes First | Why start with linear regression? |
| 2 | Deep Learning Gradient Descent: Deriving the Update Rule (coming soon) | Where does θ ← θ + 2αx(y − ŷ) come from? |
| 3 | Deep Learning Learning Rate: Converge, Oscillate, or Diverge (coming soon) | Why do some learning rates converge and others blow up? |
| 4 | Deep Learning Backpropagation with 5 Graph Nodes (coming soon) | How do you split the derivative of a long formula? |
| 5 | Deep Learning Data Distribution: Why Training Breaks (coming soon) | Why do the mean and spread of inputs matter? |
| 6 | Deep Learning Mini-batch vs Batch vs SGD Explained (coming soon) | How many samples should one update use? |
| 7 | Deep Learning Bias: What Changes with One More Parameter (coming soon) | What changes with two parameters? |
| 8 | Deep Learning Cost Contours: Reading the Loss Landscape (coming soon) | What does the error landscape look like? |
| 9 | Deep Learning Gradient Descent Path: Why It Zigzags (coming soon) | Why does the path bend and zigzag? |
| 10 | Deep Learning Feature Scaling: What Happens Without It (coming soon) | Why does rescaling the input rescue training? |
| 11 | Deep Learning in C++: Lessons from Porting Python (coming soon) | What does it take to match NumPy without NumPy? |
Parts 1 to 6 cover the stage 1 model and parts 7 to 10 the stage 2 model. Part 11 looks at the implementation itself from a developer’s point of view. A second series on multivariate regression, polynomial regression and overfitting, logistic regression, and cross entropy is in preparation.
How each part is structured
- Short answer: the conclusion first.
- Intuition: charts and analogies before formulas.
- Key formula: one or two formulas only.
- C++ and Python code: the formula as code, in both languages.
- Results: the numbers and charts from real runs.
- Common misreadings: where results are easy to misinterpret.
Each part also opens with a table of new terms and ends with a small exercise you can try.
What you need before reading
- Math: linear and quadratic functions and the idea of a slope. Derivatives are explained as needed.
- C++ or Python: enough to read loops and functions in either one. Apart from the small node classes in part 4 and one template function in part 11, the C++ uses only functions and loops.
- Setup: any compiler with C++17. I used MSVC with VS Code on Windows. No external libraries are needed, and results are saved as CSV so you can chart them in a spreadsheet. Only part 1’s Python needs NumPy; the rest runs on the standard library.
Every result is printed in the posts, so you can follow along without running anything.
Where to start
- New to how training works: read parts 1, 2, and 3 in order to see what “the model finds the value by itself” really means.
- Backpropagation always confused you: check the update rule in part 2, then jump to part 4.
- Training keeps failing and you want to know why: start with part 3 (learning rate) and part 10 (feature scaling).
- C++ developers: skim the structure in part 11 first, then start from part 1.
Sources and principles
This series is my study log from FastCampus’s course “Deep Learning with a Mathematical Approach” (수학적으로 접근하는 딥러닝), where I re-implemented the exercises in C++. Course slides and code are not reproduced here. The code, charts, and numbers in this series are my own work and my own runs.
If you spot a mistake or know a clearer way to explain something, please leave a comment. I will keep improving the series as it goes.
Start here
The first part shows why deep learning study starts with linear regression, and watches a one-parameter model find its value step by step.
→ Part 1: Deep Learning Linear Regression: Why It Comes First
