What are Deep Features?
A deep feature is the consistent response of a node or layer within a hierarchical model to an input that gives a response that’s relevant to the model’s final output. One feature is considered “deeper” than another depending on how early in the decision tree or other framework the response is activated.
How are Deep Features Different than other Features?
In a neural network designed for image classification, it is trained on a set of natural images and learns filters (features), such as image edge and contour detectors from earlier layers. The “deeper” layers can respond and create their own feature filters for more complicated patterns in the input, such as textures, shapes or variations of features processed earlier.
So while a conventionally trained network has later filter nodes that can identify a specific feature such as a face, they wouldn’t be able to tell the difference between a face and any similar round object. However, the response from a layer deeper in the algorithm’s hierarchy serves as a feature filter that the model can use to not just distinguish faces from non-facial items, but create new classifiers during classification.
