An object detection system comprises a physical object recognition unit which recognizes physical objects outside the vehicle, and a physical object memory which stores information concerning the physical objects. A controller recognizes presence of a physical object when the physical object is sensed by one or more sensors a predetermined number of times, which is larger for a central portion of the detecting area than for a peripheral portion of the detecting area. The system includes a memory for storing information about the physical objects. The controller holds the physical object stored in the memory indicating that the physical object is existent until the controller fails to recognize presence of the corresponding physical object a predetermined number of recognition cycles.