With increasing degree of automation, vehicles require more and more external sensors to observe their surrounding environment. Car manufacturers are facing the challenge of defining a suitable sensor setup that covers all requirements. Besides the sensors’ performance and surround-view coverage, other factors like setup costs, vehicle integration and design aspects need to be taken into account. Additionally, a redundant sensor arrangement and the sensors’ sensitivity to environmental influences are of crucial importance for safety. It is not feasible to explore every possible sensor constellation in test drives at the concept phase of vehicle development. Thus, we established a new simulation-based evaluation methodology, which allows the configuration of arbitrary sensor setups and enables virtual test drives within specific scenarios to evaluate the environmental perception in early development phases with metrics and key performance indicators. This evaluation suite is an important tool for researchers and developers to analyze setup correlations and to define optimal setup solutions.