In the context of ADAS applications for Level3 and beyond, reaching robustness against different environmental, roadway and traffic conditions demand the usage of several sensor data. Subsequently, a low or a high level fusion algorithm is required for the perception output. Whereas high level fusion has the advantage of suitability to the sensor data, it lacks a broader understanding of the scenario compared to low level fusion approaches, and vice-versa. In the current work we propose an architecture for modularized AI perception which is usable to any dense pointcloud independent of the sensor data. We show the proposed perception architecture working on both LiDAR, Radar and also on fused pointcloud data. As a second step, we present the HW optimized version where, thanks to relatively low TOPS consumption, reaches a max runtime of 20FPS on the SOC -crucial for real time reaction for the tested ODDs-. Thanks to its modularity, this proposal can be applied on any pointcloud sensor or fused data from it, opening scalability to different low, mid and high fusion approaches and different LiDAR and Radar versions.