Using deep neural networks for modeling representational spaces: the prevalence and impact of rarely-firing nodes

Deep neural networks (DNNs) are increasingly being used as computational models of human vision and higher-level cognition. When DNNs are trained to recognize objects in images, they develop a similarity space, measured by the distance between image pairs in DNNs' nodes. This similarity space can be made to be more human-like by pruning redundant nodes, which suggests that DNNs need only a subset of nodes to model human similarity judgments. Because the pruning method requires supervision by similarity judgments from humans which is costly to collect, in our work, we investigate if it is possible to prune DNNs to improve the prediction of human similarity judgments without human data. It has been shown that after being trained, DNNs contain many nodes that are not activated (zero) for a majority of images. We hypothesize that because these nodes carry less information they can be pruned. To quantify the effect of pruning, we used Pearson correlation as a measure of fit between two representational similarity matrices (RMS): 1) RSM from pruned or un-pruned network, 2) RSM from human similarity judgments for images. Our results showed that: 1) nodes with mostly zero-firing values are prevalent but contribute minimally to the DNN’s own similarity space, and 2) removing a majority of these nodes does not affect, and sometimes even improves the prediction of human similarity judgments. We suggest that these nodes should be considered as a separate class when constructing encoding or decoding models of human cognition.