An arrangement is provided for performing Montgomery multiplications. A Montgomery multiplication comprises a plurality of iterations of basic operations (e.g., carry-save additions), and is performed by a Montgomery multiplication engine (MME). Basic operations in each iteration may be performed by multiple Montgomery multiplication processing elements (MMPEs). An MME may be arranged to pipeline the process of performing iterations of multiple basic operations and other operations required to complete a Montgomery multiplication both horizontally and vertically. An MME may also be arranged to interleave processes of performing two Montgomery multiplications.