The present invention relates generally to a method and system for processing data. In a particular embodiment, the method includes receiving data to be processed from a network communication channel, storing the received data to be processed in memory based files at a computer memory that is local to and directly coupled to a processor via a high-speed memory bus, processing the received and stored data at the processor to produce processed data, compressing the processed data using a data compression software routine resident at the computer memory to produced processed and compressed data, and storing the processed and compressed data at a computer disk storage unit.