The term "deep" in deep learning directly refers to the architecture of the neural network having multiple layers between the input and output layers. These intermediate layers are known as hidden layers. A shallow neural network might have only one hidden layer, whereas a deep neural network is characterized by having many hidden layers stacked upon each other. Each hidden layer learns to represent the data at a different level of abstraction, allowing the model to learn complex patterns and hierarchical features. The number of hidden layers determines the network's depth.

A. Convolution: This is a specific type of layer, common in deep learning for tasks like image recognition, but it is not the general term for the layers that create depth.

B. Dropout: This is a regularization technique applied to layers to prevent overfitting by randomly ignoring some neurons during training; it does not define the network's depth.

C. Pooling: This is a specific type of layer used for down-sampling feature maps, typically in conjunction with convolutional layers, not the fundamental component of depth.

---

1. LeCun

Y.

Bengio

Y.

& Hinton

G. (2015). Deep learning. Nature

521(7553)

436–444.

Page 436

Column 1

Paragraph 2: "Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction." These "multiple processing layers" are the hidden layers that constitute the model's depth.

DOI: https://doi.org/10.1038/nature14539

2. Goodfellow

I.

Bengio

Y.

& Courville

A. (2016). Deep Learning. MIT Press.

Chapter 6

Deep Feedforward Networks

Section 6.4

Page 195: The text defines deep models by their depth

stating

"The core idea of deep learning is that we can learn a deep model by introducing more layers... In this book

we use the term 'deep learning' to refer to the study of deep models." This directly links the concept of "deep" to the number of layers (specifically

hidden layers).

3. Stanford University. (n.d.). CS231n: Convolutional Neural Networks for Visual Recognition

Course Notes.

Module 1: Neural Networks Part 1: Setting up the Architecture: The notes explain the structure of neural networks

defining the input layer

hidden layer(s)

and output layer. It clarifies that a "deep" network is one with multiple hidden layers

distinguishing it from a "2-layer Net" (one hidden layer). This establishes that hidden layers are the components that create depth.