The question of single unit semantics in deep networks

The question of single unit semantics in deep networks

The question of single unit semantics in deep networks. Do we have reason to be a little less bewildered by so-called grandmother cells in the context of deep neural networks?

A familiar hypothesis in deep learning is that a single higher-layer unit of a network may correspond to a complex semantic entity, such as a person or a specific type of dog [2, 9]. This has colloquially been referred to as grandmother cells or Homer Simpson cells.

These names stem from neuroscience. It has been observed for the human brain that specific cortical cells in late stages of the visual pathway can activate when a person is shown any variant of a photograph of, for instance, Jennifer Aniston, as was done in the study by Quiroga et al. [6]. In the study, however, it was also found that the same response could be elicited by the printed name of Jennifer Aniston, indicating that the firing resulted from memory mechanisms rather than pure visual stimuli and geometric transformations thereof.

The concept of grandmother cells has made an impression on the broad community of practitioners in deep learning, being both captivating and easy to grasp. Visualization methods of these cells have been presented, both network-based [2,7] and input-based [9].

In deep learning papers, it is not uncommon to show these kinds of visualizations and assume that they are proof that the network has learned something sensible, e.g., [3].

Whether this really is the case is what I intend to investigate in this blog post. For this purpose, I will mainly refer to three papers that represent three different perspectives on the matter. They are Visualizing and understanding convolutional neural networks by Zeiler & Fergus [9], Salient deconvolutional networks by Mahendran & Vedaldi [4], and On the importance of single directions for generalization by Morcos & Barrett [5].

deep-learning representation-learning machine-learning explainability

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Hire Machine Learning Developers in India

We supply you with world class machine learning experts / ML Developers with years of domain experience who can add more value to your business.

What is Supervised Machine Learning

What is neuron analysis of a machine? Learn machine learning by designing Robotics algorithm. Click here for best machine learning course models with AI

Pros and Cons of Machine Learning Language

AI, Machine learning, as its title defines, is involved as a process to make the machine operate a task automatically to know more join CETPA

Deep Learning Explained in Layman's Terms

Deep Learning Explained in Layman's Terms. In this post, you will get to learn deep learning through a simple explanation (layman terms) and examples.

Top 10 Deep Learning Sessions To Look Forward To At DVDC 2020

Looking to attend an AI event or two this year? Below ... Here are the top 22 machine learning conferences in 2020: ... Start Date: June 10th, 2020 ... Join more than 400 other data-heads in 2020 and propel your career forward. ... They feature 30+ data science sessions crafted to bring specialists in different ...