Effects of diffraction of a spectral beam from an edge of the micromirrors are reduced in order to optimize the passband in a wavelength selective switch. The effects of diffraction on the pass band may be reduced by using rotation of the micromirror about both the attenuation axis and the switching axis to achieve the desired level of attenuation. Peak coupling can be attained by dithering the micromirror about an axis tangent to a contour of constant attenuation using simultaneous rotation about the switching and attenuation axes.