A method and system for determining an alpha matte for a video is presented. A set of videos is acquired by an array of cameras. A centrally located camera in the array is designated as a reference camera and acquires a reference video. A foreground depth plane is selected from the set of videos. A trimap is determined from variances of pixel intensities in each image. Variances of the intensities of pixels labeled as background and pixels labeled as foreground are extrapolated to the pixels labeled as unknown in the trimap. Means of the intensities of the pixels labeled as background are extrapolated to the pixels labeled as unknown to determine an alpha matte for the reference video.