run_tesscorr_mpi.py command line utility
Scheduler using MPI for running the TESS lightcurve corrections pipeline on a large scale multi-core computer.
The setup uses the task-pull paradigm for high-throughput computing
using mpi4py. Task pull is an efficient way to perform a large number of
independent tasks when there are more tasks than processors, especially
when the run times vary for each task.
The basic example was inspired by https://github.com/jbornschein/mpi4py-examples/blob/master/09-task-pull.py
Example
To run the program using four processes (one master and three workers) you can execute the following command:
>>> mpiexec -n 4 python run_tesscorr_mpi.py
Code author: Rasmus Handberg <rasmush@phys.au.dk>