Trivial or Wrong

A Whole New Dimension of Optimization

Tue, 10 Jan 2017 00:00:00 +0000

A Whole New Dimension of Optimization

Take the Rough With the Smooth

Mon, 28 Nov 2016 00:00:00 +0000

The starting point of the previous post was optimization of differentiable functions, but trying to utilize Fermat's theorem led us eventually to bracketing algorithms, which make no use of the derivative and are applicable for non-differentiable functions.

The Roots of No Evil

Tue, 15 Nov 2016 00:00:00 +0000

“…nothing at all takes place in the universe in which some rule of maximum or minimum does not appear…” -Leonhard Euler

Random Integers

Mon, 26 Sep 2016 00:00:00 +0000

The need to sample from a discrete distribution given by a probability vector $\vec{p}=(p_1,p_2,...,p_n)$ comes up very often. For example, when taking a bootstrapped sample from a weighted dataset, or when selecting parents from a population based on their realtive fitness in genetic algorithms, or when implementing an agent that applies a mixed strategy, or... well, I made my point: often.

Machine Learning. Literally.

Wed, 08 Jun 2016 00:00:00 +0000

What’s the most successful application of machine learning?

Laws, Sausages and ConvNets

Wed, 01 Jun 2016 00:00:00 +0000

Laws, like sausages, cease to inspire respect in proportion as we know how they are made.

The Generative-Discriminative Fallacy

Sun, 22 May 2016 00:00:00 +0000

1. The Generative-Discriminative Dichotomy

It is customary to categorize predictive algorithms as either "generative" or "discriminative". Denoting by $Y$ the target and by $X$ the observable, discriminative algorithms are those that model the conditional distribution $P(Y|X;\theta)$ and generative algorithms are those that model the joint distribution $P(X,Y|\theta)$) (note that this entire discussion presumes a probabilistic perspective on predictive tasks).

Learning Dynamical Systems

Sat, 14 May 2016 00:00:00 +0000

1. Motivational Speech

The Name of The Rose

Tue, 22 Dec 2015 00:00:00 +0000

Many ideas in machine learning came up independently in different contexts, and it's not uncommon to have multiple terms for the same concept. Those instances are usually dismissed as nuisance; after all, a rose by any other name, et cetera. Indeed, often this is just a matter of nomenclature, and with time conventions form, and some terms disappear while others acquire universal meaning. But sometimes the distinction is not as superficial as it may seem.

Pointless Topology via Abstract Nonsense

Tue, 15 Dec 2015 00:00:00 +0000

In 1940. G.H. Hardy published his essay "A Mathematician's Apology" in which he used number theory as a prominent example for pure and inapplicable mathematics. He meant it seriously, not as a joke. To the defense of his apology (pun a bit intended), at the time only a handful of programmers thought about hash-tables, cryptosystems were yet to be widely implemented in E-commerce platforms, and not many people used FFT before 1965 (no one but Gauss, I think). Indeed, computers forced mathematicians to work harder and harder in order to keep annoying engineers away from their offices.