Scheduler using MPI for running the TESS photometry pipeline on a large scale multi-core computer.
The setup uses the task-pull paradigm for high-throughput computing
mpi4py. Task pull is an efficient way to perform a large number of
independent tasks when there are more tasks than processors, especially
when the run times vary for each task.
The basic example was inspired by https://github.com/jbornschein/mpi4py-examples/blob/master/09-task-pull.py
To run the program using four processes (one master and three workers) you can execute the following command:
>>> mpiexec -n 4 python mpi_scheduler.py
Code author: Rasmus Handberg <firstname.lastname@example.org>