At a face of a silicon semiconductor substrate tilted about one degree from a [100] orientation, a readout integrated circuit (ROIC) is implemented, specially designed and fabricated for direct epitaxial growth. Layers of II-VI semiconductor material, preferably including layers of HgCdTe of different bandgaps, are successively and monolithically grown on the face by molecular beam epitaxy (MBE) within a window masking the face and then patterned and wet-etched to create mesas of two-color detector elements in an array. Preferably a beginning buffer layer of CdTe is grown to minimize crystalline mismatch between the Si and the HgCdTe. Sloped sidewalls of the mesas ensure good step coverage of the conductive interconnects from the detector elements to the ROIC.