PyLops: MPI backend for distributed inverse problems

Pylops has been developed to solve large-scale inverse problems. Keeping in mind the memory size of a single machine, there is a need to perform the operations in a distributed fashion. The main goal of this project is to use the mpi4py package to exchange messages between different processes across distributed memory. The main outcomes of this project are to develop a DistributedArray class using mpi4py to broadcast or scatter numpy/cupy arrays and perform basic operations like sum, product, dot-product, element-wise product, etc in a distributed fashion. Additionally, MPI functionality will be added to BlockDiag, VStack, and HStack, and the code for forward, centered, and backward finite difference stencils will be updated using Distributed Arrays. Lastly, at least one of the CG, CGLS, and LSQR solvers will be modified to use Distributed Arrays. This will benefit the Pylops Codebase as integrating mpi4py into the codebase will enhance the operators' performance and prevent exceeding memory on a single machine.