A method and system dynamically calculates confidence levels associated with accuracy of Z depth information obtained by a phase-shift time-of-flight (TOF) system that acquires consecutive images during an image frame. Knowledge of photodetector response to maximum and minimum detectable signals in active brightness and total brightness conditions is known a priori and stored. During system operation brightness threshold filtering and comparing with the a priori data permits identifying those photodetectors whose current output signals are of questionable confidence. A confidence map is dynamically generated and used to advise a user of the system that low confidence data is currently being generated. Parameter(s) other than brightness may also or instead be used.