run_starclass_mpi.py command line utility
Scheduler using MPI for running the TASOC classification pipeline on a large scale multi-core computer.
The setup uses the task-pull paradigm for high-throughput computing
mpi4py. Task pull is an efficient way to perform a large number of
independent tasks when there are more tasks than processors, especially
when the run times vary for each task.
The basic example was inspired by https://github.com/jbornschein/mpi4py-examples/blob/master/09-task-pull.py
To run the program using four processes (one master and three workers) you can execute the following command:
>>> mpiexec -n 4 python run_starclass_mpi.py
Code author: Rasmus Handberg <email@example.com>