Actually you are free do do most anything with FPGAs and high speed interconnection is also no problem.
Some FPGAs have Gigabit Transceivers that are usefull for FPGA interconnection.
The HPI port seems to be some "hardware hack" that is convenient and usefull for DSP systems.
But it's not a proper way for building a generic multiprocessor system.
You should first think about your requirements.
There are many things to be considered for bulding a multiprocessor system.
Do you want a system with shared memory or with local memory?
Both have their pros and cons.
If the CPUs are actually in different devices, high speed communication can be established in many ways on many levels. But in the end it matters how the software can make use of these communication paths.
If you invent something special, you need to write drivers and OS-Kernel modifications.
If you use standards (e.g. ethernet TCP/IP) you can concentrate on the higher levels of programming for your multiprocessor system.
Designing a multiprocessor system is no easy task.
You should do some reading about it to avoid the many pitfalls that have already been identified, but there's no "state of the art" design method for such systems. This is still a research topic.