Is it enough to optimize CNN architectures on ImageNet?

Tuggener, Lukas and Schmidhuber, Jürgen and Stadelmann, Thilo (2022) Is it enough to optimize CNN architectures on ImageNet? Frontiers in Computer Science, 4. ISSN 2624-9898

[thumbnail of pubmed-zip/versions/1/package-entries/fcomp-04-1041703/fcomp-04-1041703.pdf] Text
pubmed-zip/versions/1/package-entries/fcomp-04-1041703/fcomp-04-1041703.pdf - Published Version

Download (8MB)

Abstract

Classification performance based on ImageNet is the de-facto standard metric for CNN development. In this work we challenge the notion that CNN architecture design solely based on ImageNet leads to generally effective convolutional neural network (CNN) architectures that perform well on a diverse set of datasets and application domains. To this end, we investigate and ultimately improve ImageNet as a basis for deriving such architectures. We conduct an extensive empirical study for which we train 500 CNN architectures, sampled from the broad AnyNetX design space, on ImageNet as well as 8 additional well-known image classification benchmark datasets from a diverse array of application domains. We observe that the performances of the architectures are highly dataset dependent. Some datasets even exhibit a negative error correlation with ImageNet across all architectures. We show how to significantly increase these correlations by utilizing ImageNet subsets restricted to fewer classes. These contributions can have a profound impact on the way we design future CNN architectures and help alleviate the tilt we see currently in our community with respect to over-reliance on one dataset.

Item Type: Article
Subjects: European Scholar > Computer Science
Depositing User: Managing Editor
Date Deposited: 06 Dec 2022 12:53
Last Modified: 25 May 2024 07:40
URI: http://article.publish4promo.com/id/eprint/384

Actions (login required)

View Item
View Item