1597277640

**TL;DR:**_ Have you even wondered what is so special about convolution? In this post, I derive the convolution from first principles and show that it naturally emerges from translational symmetry._

La connoissance de certains principes supplée facilement à la connoissance de certains faits. (Claude Adrien Helvétius)

During my undergraduate studies, which I did in Electrical Engineering at the Technion in Israel, I was always appalled that such an important concept as convolution [1] just landed out of nowhere. This seemingly arbitrary definition disturbed the otherwise beautiful picture of the signal processing world like a grain of sand in one’s eye. How nice would it be to have the convolution emerge from first principles rather than have it postulated! As I will show in this post, such first principles are the notion of translational invariance or symmetry.

Let me start with the formula taught in basic signal processing courses defining the discrete convolution [2] of two *n*-dimensional vectors **x** and **w**:

Here, for convenience, I assume that all the indices run from zero to _n_−1 and are modulo *n*; it is convenient to think of vectors as defined on a circle. Writing the above formula as a matrix-vector multiplication leads to a very special matrix that is called *circulant*:

A circulant matrix has multi-diagonal structure, with elements on each diagonal having the same value. It can be formed by stacking together shifted (modulo *n*) versions of a vector **w [3]; for this reason, I use the notation C(w) referring to a circulant matrix formed by the vector w. Since any convolution x∗w**can beequivalently represented as a multiplication by the circulant matrix

#ai & machine learning #convolution #convolutional newral net #deep learning #deep learning

1597277640

**TL;DR:**_ Have you even wondered what is so special about convolution? In this post, I derive the convolution from first principles and show that it naturally emerges from translational symmetry._

La connoissance de certains principes supplée facilement à la connoissance de certains faits. (Claude Adrien Helvétius)

During my undergraduate studies, which I did in Electrical Engineering at the Technion in Israel, I was always appalled that such an important concept as convolution [1] just landed out of nowhere. This seemingly arbitrary definition disturbed the otherwise beautiful picture of the signal processing world like a grain of sand in one’s eye. How nice would it be to have the convolution emerge from first principles rather than have it postulated! As I will show in this post, such first principles are the notion of translational invariance or symmetry.

Let me start with the formula taught in basic signal processing courses defining the discrete convolution [2] of two *n*-dimensional vectors **x** and **w**:

Here, for convenience, I assume that all the indices run from zero to _n_−1 and are modulo *n*; it is convenient to think of vectors as defined on a circle. Writing the above formula as a matrix-vector multiplication leads to a very special matrix that is called *circulant*:

A circulant matrix has multi-diagonal structure, with elements on each diagonal having the same value. It can be formed by stacking together shifted (modulo *n*) versions of a vector **w [3]; for this reason, I use the notation C(w) referring to a circulant matrix formed by the vector w. Since any convolution x∗w**can beequivalently represented as a multiplication by the circulant matrix

#ai & machine learning #convolution #convolutional newral net #deep learning #deep learning

1596633180

**TL;DR:**_ Have you even wondered what is so special about convolution? In this post, I derive the convolution from first principles and show that it naturally emerges from translational symmetry._

La connoissance de certains principes supplée facilement à la connoissance de certains faits. (Claude Adrien Helvétius)

D

uring my undergraduate studies, which I did in Electrical Engineering at the Technion in Israel, I was always appalled that such an important concept as convolution [1] just landed out of nowhere. This seemingly arbitrary definition disturbed the otherwise beautiful picture of the signal processing world like a grain of sand in one’s eye. How nice would it be to have the convolution emerge from first principles rather than have it postulated! As I will show in this post, such first principles are the notion of translational invariance or symmetry.

Let me start with the formula taught in basic signal processing courses defining the discrete convolution [2] of two *n*-dimensional vectors **x** and **w**:

Here, for convenience, I assume that all the indices run from zero to _n_−1 and are modulo *n*; it is convenient to think of vectors as defined on a circle. Writing the above formula as a matrix-vector multiplication leads to a very special matrix that is called *circulant*:

A circulant matrix has multi-diagonal structure, with elements on each diagonal having the same value. It can be formed by stacking together shifted (modulo *n*) versions of a vector **w [3]; for this reason, I use the notation C(w) referring to a circulant matrix formed by the vector w. Since any convolution x∗w**can beequivalently represented as a multiplication by the circulant matrix

One of the first things we are taught in linear algebra is that matrix multiplication is non-commutative, i.e.,in general, **AB**≠**BA**. However, circulant matrices are very special exception:

Circulant matrices commute,

or in other words, **C**(**w**)**C**(**u**)=**C**(**u**)**C**(**w**). This is true for any circulant matrix, or any choice of **u** and **w**. Equivalently, we can say that the convolution is a commutative operation, **x**∗**w**=**w**∗**x**.

A particular choice of **w**=[0,1,0…,0] yields a special circulant matrix that shifts vectors to the right by one position. This matrix is called the (right) _shift operator _[4] and denoted by **S**. The transpose of the right shift operator is the left shift operator. Obviously, shifting left and then right (or vice versa) does not do anything, which means **S** is an orthogonal matrix:

Circulant matrices can be characterised by their commutativity property. It appears to be sufficient to show only commutativity with shift (Lemma 3.1 in [5]):

A matrix is circulant if and only if it commutes with shift.

The first direction of this “if and only if” statement leads to a very important property called *translation* or _shift equivariance _[6]: the convolution’s commutativity with shift implies that it does not matter whether we first shift a vector and then convolve it, or first convolve and then shift — the result will be the same.

The second direction allows us to *define* convolution as the shift-equivariant linear operation: in order to commute with shift, a matrix must have the circulant structure. This is exactly what we aspired to from the beginning, to have the convolution emerge from the first principles of translational symmetry [7]. Instead of being given a formula of the convolution and proving its shift equivariance property, as it is typically done in signal processing books, we can start from the requirement of shift equivariance and arrive at the formula of the convolution as the only possible linear operation satisfying it.

Illustration of shift equivariance as the interchangeability of shift and blur operations.

A nother important fact taught in signal processing courses is the connection between the convolution and the Fourier transform [8]. Here as well, the Fourier transform lands out of the blue, and then one is shown that it diagonalises the convolution operation, allowing to perform convolution of two vectors in the frequency domain as element-wise product of their Fourier transforms. Nobody ever explains where these sines and cosines come from and what is so special about them.

#deep-learning #convolutional-neural-net #data-science #machine-learning #convolution #deep learning

1617959650

Antier Solutions is consider as one of the best **Cryptocurrency Derivatives Exchange Development company** that is specialized in offering world’s best cryptocurrency Derivatives Exchange Development Services that enables the following 4 key types- Futures trading, Forwards trading, Options trading, and Perpetual trading. Our skilled team of blockchain developers build Derivative exchange on both centralized and decentralized platform. It’s your choice to build the best one for you. Exchange owners can leverage the potential of a cryptocurrency derivatives exchange to reach more investors. A Crypto derivative trading platform is more flexible than Spot Margin trading and opens access to otherwise unavailable markets. To know more about us, feel free to visit us.

#cryptocurrency derivatives exchange #crypto derivatives exchange development #cryptocurrency derivatives exchange development #derivatives exchange development services #derivative exchange development company

1599921480

I understand your frustration and I have some good news and bad news.

- Stick around another 5-10 years and there will be plenty more firsts to add to your collection!
- Definitions of these Firsts can vary from expert to expert.
- You cannot just pick a single first and run with it. No first is an island. You will probably end up using a lot of these…

While there are a lot of different “first” methodologies out there, some are very similar and have just matured just as our technology stack has.

Here is the first stack I recommend looking at when you are starting a new project:

Know the high-level, big-picture view of what you are building. Define the problem you are solving and the requirements to solve it. Are you going to need a Mobile app? Website? Something else?

Have the foresight to realize that whatever you think you will need, it will change in the future. I am not saying design for every possible outcome but use wisdom and listen to your experts.

API First means you think of APIs as being in the center of your little universe. APIs run the world and they are the core to every (well, almost every) technical product you put on a user’s phone, computer, watch, tv, etc. If you break this first, you will find yourself in a world of hurt.

Part of this `First`

is the knowledge that you better focus on your API first, before you start looking at your web page, mobile app, etc. If you try to build your mobile app first and then go back and try to create an API that matches the particular needs of that one app, the above world of hurt applies.

Not only this but having a working API will make design/implementation of your mobile app or website MUCH easier!

Another important point to remember. There will most likely be another client that needs what this API is handing out so take that into consideration as well.

I’ve grouped these next two together. Now I know I am going to take a lot of flak for this but hear me out.

I agree that you should always design your API first and not just dig into building it, However, code is a legitimate design tool, in the right hands. Not everyone wants to use some WYSIWYG tool that may or may not take add eons to your learning curve and timetable. Good Architects (and I mean GOOD!) can design out an API in a fraction of the time it takes to use some API design tools. I am NOT saying everyone should do this but don’t rule out Code-First because it has the word “Code” in it.

*You have to know where to stop though.*

Designing your API with code means you are doing design-only. You still have to work with the technical and non-technical members of your team to ensure that your API solves your business problem and is the best solution. If you can’t translate your code-design into some visual format that everyone can see and understand, DON’T use code.

#devops #integration #code first #design first #api first #api

1602394680

**S.O.L.I.D** design principle comes from Object oriented programming guidelines. It is designed to develop software that can be easily maintained and extended; prevents code smells; easy to refractor; promotes agility and finally incorporates rapid + frequent changes quickly without bugs.

Generally, technical debt is the result of prioritising speedy delivery over perfect code. To keep it under control - use SOLID principles, during development.

Robert Martin, is credited with writing the SOLID principles and stated 4 major software issues if S.O.L.I.D is not followed diligently. They are :

**Rigidity**:

Implementing even a small change is difficult since it’s likely to translate into a cascade of changes.

**Fragility**:

Any change tends to break the software in many places, even in areas not conceptually related to the change.

**Immobility**:

We’re unable to reuse modules from other projects or within the same project because those modules have lots of dependencies.

**Viscosity**:

Difficult to implement new features the right way.

**SOLID is a guideline and not a rule. It is important to understand the crux of it and incorporate it with a crisp judgement. There can be a case when only few principles out of all is required.**

#typescript #javascript #solid-in-javascript #what-are-solid-principles #what-is-a-solid-principle #design-patterns #design-principles #object-oriented-design