Not all pixels are created equal, neither are all lenses, or sensors, or manufacturers. This causes a large variance in image quality from cameras with nominally the same fundamental specifications, such as pixel size, focal length and f-number. Individual objective camera metrics can provide insight into the sharpness or noise performance of cameras, for example, and instinctively we desire more of everything.

Far too often in papers exploring DNN performance, the description of the images used is limited to the pixel count, total number and split between training and validation sets. This talk explores some desirable characteristics of image quality metrics, approaches, and pitfalls of combining them and some strategies for ranking camera performance for use with autonomous systems.