08 · Learning path
Support Vector Machine (SVM)
Finds the boundary that separates the classes with the widest possible margin.
The Support Vector Machine looks for a decision boundary that not only separates the classes, but does so leaving the widest possible margin between them.
Experiment with the margin
See how the boundary, the margin and the support vectors respond to the data.
SVM lab
Maximum margin and support vectors (SMO)
Settings
Small C = wide, tolerant margin; large C = fewer errors allowed.
Key idea
Maximum margin
Margin width ≈ 19.3 units; only the 15 support vectors determine it.
Accuracy
95.7%
Support
15
- Class 1
- Class 0
- Support vector
- Boundary f = 0
- Margin f = ±1
What to look for
- Support vectors (αᵢ > 0) are the only points that move the boundary.
- On “Ring”, the linear kernel fails; RBF wraps around the region.
- Raise C or γ and the boundary fits the noise: overfitting.
The mathematical idea
In its linear version, SVM defines a separating hyperplane:
A point is classified according to the sign of
When the classes are not perfectly separable, some error is allowed through the hinge loss and a regularization parameter
A large
What to look for
- The support vectors are the most critical points: if they move, the boundary changes; the rest do not affect it.
- A kernel such as RBF allows curved boundaries when a straight line is not enough.
- A wide margin makes the model more robust to small perturbations.
Applications
SVM is used in text classification, bioinformatics, computer vision and problems where good geometric separation matters more than a calibrated probability.