Here are links to some relevant journals:

The Journal of Supercomputing (TJS) - Kluwer Academic
IEEE Transactions on parallel and distributed systems (TPDS) - IEEE Computer Society
Parallel Computing (PC) - Elsevier
Parallel and Distributed Computing Practices (PDCP) - Nova Science Publishers
Parallel Algorithms and Applications (PAA) - Taylor and Francis
Concurrey and Computation: Practice and Experience (CCPE) - Wiley Interscience
More relevant journals to be added