1594020300

# You Don’t Understand Neural Networks

The Universal Approximation Theorem is, very literally, the theoretical foundation of why neural networks work. Put simply, it states that a neural network with one hidden layer containing a sufficient but finite number of neurons can approximate any continuous function to a reasonable accuracy, under certain conditions for activation functions (namely, that they must be sigmoid-like).

Formulated in 1989 by George Cybenko only for sigmoid activations and proven by Kurt Hornik in 1991 to apply to all activation functions (the architecture of the neural network, not the choice of function, was the driver behind performance), its discovery was a significant driver in spurring the excited development of neural networks into the plethora of applications in which they are used today.

Most importantly, however, the theorem is an eye-opening explanation of why neural networks appear to behave so intelligently. Understanding it is a key step in developing a strong and deep understanding of neural networks.

## A Deeper Exploration

Any continuous function on a compact (bounded, closed) set can be approximated by a piecewise function. Take, for instance, the sine wave between -3 and 3, which can be very convincingly approximated with three functions — two quadratic and one linear.

Graphed in Desmos.

Cybenko was more specific about this piecewise function, however, in that it could be constant, essentially consisting of several steps fitted to the function. With enough constant regions (‘steps’), one can reasonably estimate the function within the bounds it is given in.

Graphed in Desmos.

Based on this approximation, one could construct a network by delegating each neuron to one ‘step’. Using the weights and biases as ‘gates’ to determine which an input falls and hence which neuron should be activated, a neural network with a sufficient number of neurons could estimate a function simply be divvying it up into several constant regions.

For inputs that fall in a neuron’s delegated section, by blowing the weight to huge values, the final value approaches 1 (when evaluated using the sigmoid function). If it does not fall into the section, moving the weight towards negative infinity will yield a final result near 0. Using the sigmoid function as a “processor” of sorts to determine the degree of presence of a neuron, any function can be approximated almost perfectly, given an 6abundance of neurons. In multi-dimensional space, Cybenko generalized this idea, each neuron ‘controlling’ a hypercube of space within a multidimensional function.

The key point in the Universal Approximation Theorem is that instead of creating complex mathematical relationships between the input and output, it uses simple linear manipulations to divvy up the complicated function into many small, less complicated pieces, each of which are taken by one neuron.

Image created by Author.

Since Cybenko’s initial proof, many additional developments have been composed, such as testing out the Universal Approximation Theorem for different activation functions, like ReLU, which is unbounded (on one side), or with various architectures (recurrent, convolutional, etc.).

Regardless, all of these explorations center around one idea — neural networks find strength in numbers. Each neuron watches over one pattern or area of the feature space, whose size is determined by the number of neurons in the network. The less neurons there are, the more space each one will need to watch over, and hence approximation capability will go down. With more neurons, however, regardless of activation function, _any _function can be pieced together with many small fragments.

#machine-learning #ai #data-science #data-analysis #artificial-intelligence #data analysis

## Buddha Community

1635748246

Sự hiện diện của giải pháp Smart Locker, như một nâng tầm dịch vụ, sẽ giúp khách hàng chứa đựng tư trang trước khi thoải mái tận hưởng thời gian mua sắm.

#tủ_locker #tủ_sắt_locker   #locker #tu_sat_locker #tu_locker #tủ_locker_sắt #tủ_nhân_viên #tu_locker_sat #tủ_locker_giá rẻ #tu_locker_gia_re #tủ_cá_nhân_locker #tủ_sắt_nhiều_ngăn #tủ_đựng_đồ_nhân_viên

Website:

locker

tủ sắt nhiều ngăn

tủ đựng đồ nhân viên

tủ locker

tủ sắt locker

1637298473

## cloek

Tủ locker ABS w600 khối gồm có 24 ngăn đều nhau, cánh sắt mở sử dụng khóa thông minh , tay nắm và có thêm tấm chia ngăn, trên mỗi cánh cửa được dập 1 bảng tên nhân viên và 1 lỗ thông gió

#tủ_locker #tủ_sắt_locker #locker #tu_sat_locker #tu_locker #tủ_locker_sắt #tủ_nhân_viên #tu_locker_sat #tủ_locker_giá rẻ #tu_locker_gia_re #tủ_cá_nhân_locker #tủ_sắt_nhiều_ngăn #tủ_đựng_đồ_nhân_viên

Website:

ilocker

tủ sắt nhiều ngăn

tủ đựng đồ nhân viên

tủ locker

tủ sắt locker

1637202929

## locker

Tủ thông minh giúp tăng đáng kể thời gian lao động cá nhân và đơn giản khâu vận hành. Công nghệ điện tử và điện thoại thông minh được sử dụng nhằm hạn chế lãng phí nguồn lực.

#tủ_locker #tủ_sắt_locker #locker #tu_sat_locker #tu_locker #tủ_locker_sắt #tủ_nhân_viên #tu_locker_sat #tủ_locker_giá rẻ #tu_locker_gia_re #tủ_cá_nhân_locker #tủ_sắt_nhiều_ngăn #tủ_đựng_đồ_nhân_viên

Website:

ilocker

tu locker sat

tủ locker giá rẻ

tu locker gia re

tủ cá nhân locker

1637049082

## tủ locker

Hệ thống tủ locker trong trường học giúp cho các học sinh sinh viên có một môi trường học tập hiện đại, thoải mái và an toàn hơn.

#tủ_locker #tủ_sắt_locker #locker #tu_sat_locker #tu_locker #tủ_locker_sắt #tủ_nhân_viên #tu_locker_sat #tủ_locker_giá rẻ #tu_locker_gia_re #tủ_cá_nhân_locker #tủ_sắt_nhiều_ngăn #tủ_đựng_đồ_nhân_viên

Website:

locker

tủ đựng đồ nhân viên

tủ locker

tủ sắt locker

locker

1636613387

## Smart Locker

APROP đã tin tưởng chọn Smart Locker là đơn vị đồng hành trong việc triển khai lắp đặt hệ thống Smart Locker Wireless với công nghệ hiện đại để phục vụ việc gửi/ nhận hồ sơ cho văn phòng.

#tủ_locker #tủ_sắt_locker #locker #tu_sat_locker #tu_locker #tủ_locker_sắt #tủ_nhân_viên #tu_locker_sat #tủ_locker_giá rẻ #tu_locker_gia_re #tủ_cá_nhân_locker #tủ_sắt_nhiều_ngăn #tủ_đựng_đồ_nhân_viên

Website:

tủ locker

tủ locker

tủ sắt locker

locker

tu sat locker