A testing framework that uses signal temporal logic (STL) theory is presented. The framework evaluates test cases against the STL specifications reflecting test requirements to automatically identify test cases that fail to satisfy the requirements. One of the key features is the support for machine learning components while developing the closed loop, perception-action system, such as neural networks. An example driving scenario to demonstrate the framework, followed by a discussion on complementary issues of importance when designing and developing a reliable autonomous system, is shown.