Most people who have had some experience in the field of data science have heard of the Curse of Dimensionality. It describes a host of phenomena that arise when studying manipulating and analyzing data in high-dimensional spaces that don’t occur in low-dimensional spaces.

Consider, for instance, how the concept of distance is distorted in a high dimensional plane. The formula for distance between two points a and b defined by coordinates (_x, y, _…) each with n dimensions is as follows:

You can find a proof for the formula here.

When the formula is plotted, where x is the number of dimensions and y the distance between the origin and a point (1, 1, 1, …), one can see that distance gradually will inevitably peak, or, more likely, tend to one single value.

Graphed in Desmos.

This is evidence of the nature of distance in high-dimensional spaces — as dimensionality increases, the importance, or value, of any single one-dimensional line diminishes, as can be seen with the diminishing returns on the y-axis. This is to be expected, considering how quickly volume and the possibilities of points grows as dimensionality increases.

This is also the reason why a hypersphere’s volume tends towards zero as the dimensionality increases — since a sphere is defined as consisting of all points that are one radius’ distance in Euclidean distance away from a center point, the number of dimensions increases but the distance gradually tapers off. Hence, however, unintuitive it may seem, as dimensionality tends towards infinity, a hypersphere’s volume will tend towards zero, while the hypercube it is inscribed in will continue growing (or stay constant, if the side length is 1).

Let us consider a hypercube in two dimensions (a square) with side lengths of five units. There are, then 5² = 25 units. A similar hypercube in three dimensions (a cube) has 125 units. From there, it skyrockets. The power of exponents is really very incredible — just within ten dimensions, the hypercube already has a hypervolume of 9,765,625 units. Adding an addition dimension to a space expands the current space by a huge magnitude, so it should be no surprise that a miniscule one-dimensional distance has diminishing value.

#data-science #ai #data analysis