Blogs

// notes, deep dives, and experiments.

Apr 29, 2026

deep learninglearning rate

Learning Rate Techniques in Deep Learning

Training a neural network requires many decisions: which optimizer to use, what learning rate to set, whether to regularize, and how to schedule the LR over …

Read →

Jan 5, 2026

deep learningkfold

Gradient accumulation and KoLeo loss - Part 3/3

In the previous chapter, we observed that the KoLeo loss successfully spreads embeddings in the representation space. However, we noted that the KoLeo loss is …

Read →

Dec 28, 2025

deep learningkfold

Effect of KoLeo loss on triplet loss - Part 2/3

Introduction The KoLeo loss is a regularizer that pushes representations to spread uniformly in their space by maximizing their minimum distance from each …

Read →

Dec 26, 2025

deep learningsiamese neworks

Training a siamese network with a triplet loss on CIFAR-10 - Part 1/3

Introduction In computer vision, the most common goal is to classify an image (“cat”, “dog”, “car”, etc.). But in many cases, we are more interested in …

Read →