The present invention relates to a method and system for compressing a
continuous data flow in real-time based on lossy compression. In
real-time data compression, a series of multi-dimensional data subsets
acquired in a given period of time are treated as a regional data cube
for the purpose of dividing a continuous series of data subsets into a
plurality of data cubes. In a first embodiment implementation of parallel
processing using a plurality of compression engines is facilitated by
separating a data cube into a plurality of clusters comprising similar
spectral vectors. By separating the data cube into clusters of similar
spectral vectors no artificial spatial boundaries are introduced
substantially improving image quality. Furthermore, the spectral vectors
within a cluster are more easily compressed due to their similarity. In a
second embodiment a predetermined number of 2D focal plane frames in a
boundary area of a previous regional data cube close to a current
regional data cube are included in a training set used for codevector
training for the current region. Therefore, no artificial boundary occurs
between the two adjacent regions when codevectors trained in this way are
used for codebook generation and encoding of the spectral vectors of the
current regional data cube substantially reducing image artifacts between
adjacent regions. A remedy for the single bit error problem is provided
in a third embodiment. Full redundancy of compressed data for a regional
data cube is obtained by combining the previous regional data cube and
the current regional data cube for codebook training. In order to obtain
redundancy for the index map, the codebook is used to encode the current
regional data cube as well as the previous regional data cube producing a
baseline index map for the current regional data cube and a redundant
index map for the previous regional data cube. Therefore, full redundancy
for a regional data cube is provided allowing restoration of a regional
data cube if its codebook and/or index map are corrupted or lost due to
single bit errors.
