pysep.api.parallel package

Submodules

pysep.api.parallel.dispatch module

Author: Thomas M. Boudreaux

Created: October 2021

Last Modified: June 2022

Parallel stellar model class to automatically dispactch DSEP jobs to all avalible threads.

Classes

  • pStellarModel

    parallel stellar model class

pysep.api.parallel.dispatch.iEvolve(model, stash, data)
class pysep.api.parallel.dispatch.pStellarModel(stellarModels: Optional[list] = None, name: str = 'pModels')

Bases: object

This class is a container which can hold many pysep stellarModel classes. When a user calls the pEvolve method those models are distributed to a pool of workers to be evolved in parallel. The worker pool size is equal to the maximum avalible number of threads.

name

Name to use for pStellarModel.

Type

str, default=pModels

stellarModels

list of stellar models

Type

list[pysep.dsep.stellarModel], optional

Examples

Say you have a set of stellar models

  • m070

  • m080

  • m090

  • m100

stored in some list models

>>> models = [m070, m080, m090, m100]

you could evolve these in serial

>>> [x.evolve() for x in models]

however, this will only use one thread on your computer. It would be faster and make better use of your computer to evolve them simultaniously

This can be done using a pStellarModel

>>> from pysep.api.parallel import pStellarModel
>>> pModel = pStellarModel(models)
>>> pModel.pEvolve()

Now, for a computer with n avalible threads n models from the models list will be evolved syncronously. Finally, once they are done evolving, you can save all out of their output automatically

>>> pModel.pStash()
append(model: pysep.dsep.dsepModel.stellarModel) → int

Append new models to pStellarModel (self.stellarModels)

Parameters

model (pysep.dsep.stellarModel) – Model to append

Returns

length of self.stellarModels, how many models have been enrolled

Return type

int

Raises

TypeError – If model is not of type stellarModel then a type error will be raised

new(outputDir: str, control: pysep.io.nml.control.control.cnml, physics: pysep.io.nml.physics.physics.pnml, opacf: pysep.dm.filetypes.opacf_file, premsf: pysep.dm.filetypes.premsf_file, **kwargs) → int

Generate a new stellar model and automatically append it to the pStellarModel object. This function takes the exact same argument set as the generator for pysep.dsep.stellarModel takes (*args and **kwargs are passed to it)

Parameters

  • outputDir (str) – Path where dsep will save all results to. Will always resolve to an absolute path.

  • control (pysep.io.nml.control.control.cnml) – Control namelist file object. Will be written to a temporary file and then system linked to the correct unit file by the model object.

  • physics (pysep.io.nml.physics.physics.pnml) – Physics namelist file object. Will be written to a temporary file and then system linked to the correct unit file by the model object.

  • opacf (pysep.dm.filetypes.opacf_file) – opacf_file object to be used in evolution. Will be symlinked to proper unit file.

  • premsf (pysep.dm.filetypes.premsf_file) – premfs_file object to be used in evolution. Will be symlinked to proper unit file.

  • fermi (pysep.dm.filetypes.fermi_file, default=FERMI) – fermi conductive opacity file object to be used in evolution. Will be symlinked to proper unit file.

  • bckur (pysep.dm.filetypes.bckur_file, default=atmk1990p00) – bckur file object to be used in evolution. Will be symlinked to proper unit file.

  • bcphx95 (pysep.dm.filetypes.bcphx95_file, default=z_p0d0_afe_p0d0) – bcphx95 file object to be used in evolution. Will be symlinked to proper unit file.

  • bcphx96 (pysep.dm.filetypes.bcphx96_file, default=z_m0d5_afe_p0d0) – bcphx96 file object to be used in evolution. Will be symlinked to proper unit file.

  • bcphx97 (pysep.dm.filetypes.bcphx97_file, default=z_m0d7_afe_p0d0) – bcphx97 file object to be used in evolution. Will be symlinked to proper unit file.

  • bcphx98 (pysep.dm.filetypes.bcphx98_file, default=z_m1d0_afe_p0d0) – bcphx98 file object to be used in evolution. Will be symlinked to proper unit file.

  • bcphx99 (pysep.dm.filetypes.bcphx99_file, default=z_m1d5_afe_p0d0) – bcphx99 file object to be used in evolution. Will be symlinked to proper unit file.

  • branch (str, optional) – Git branch to consider the locate the executable from. If a branch is set the model will look for the executable called dsepX.<branch>

  • default (bool, default=True) – If default is true the model will look for an executable called dsepX instead of looking for one with the branch extension.

  • loadOnly (bool, default=False) – If set the model will not be able to be evolved. This is intended for when a model is to be loaded from disk on a computer which may not have dsep installed on it.

  • mass (float, optional) – model mass. If not provided mass will automatically be read in from the premain sequence model. Note this will not account for mass rescaling only the mass of the premain sequence model used. This is here to allow for effienct probing of the mass without having to open the prems file

Returns

length – The current length of the pStellarModel object

Return type

int

pEvolve(autoStash: bool = False, data: bool = True)

Evolve models in parallel with the number of workers being equal to the number of avalible threads. Models will be evaluated from their own temporary directories.

Parameters

  • autoStash (bool, default=False) – If True models will be saved to their final location automatically upon completion of evolution, if false this will not happen and pStash will have to be called. This is useful when many models are being evolved (> 100,000) and the python tmpfile module might have issues maintaing all the temp directories.

  • data (bool, default=True) – Whether to store datafiles within .dsep model dump file. This only has an effect if autoStash is set to true

pStash(data: bool = True)

Store the data of all of the models for a given pModel. Note that once evolved these are copies of the original models.

Parameters

data (bool, default=True) – Store the data with the stashs

swap_into_DB(outDir: str, pbar: bool = False) → pysep.misc.runDB.dbmanage.model_DB

Swap models in self.stellarModels into a model_DB DB

Parameters

  • outDir (str) – Directory to output models to / enroll in DB to

  • pbar (bool, default=True) – If true a progress bar will be displayed when enrolling models into DB

Returns

DB – Database which all models in self.stellarModels have been enrolled within

Return type

pysep.misc.runDB.model_DB

Raises

  • RuntimeError – If there is not at least one model in stellarModels (enrolled with pStellarModel)

  • RuntimeError – If all models are already enrolled in a database

pysep.api.parallel.load module

pysep.api.parallel.load.pLoad(modelPaths: list) → Tuple[list, list]

Load a set of stellar models in parallel using either all the avalible CPU cores, or the number of stellar model threads (whichever is smaller). Note that this still may be heavily botllnecked by IO.

Parameters

modelPaths (list[str]) – List of paths to different DSEP models to load (can be either old or new format and either compressed or uncompressed).

Returns

  • models (list[stellarModel]) – List of stellar model objects loaded, one for each path in modelPaths and they are returned in the same order as they were ordered in modelPaths

  • tdirs (list[TemporaryDirectory]) – List of temp dirs where model data files are stored.

Examples

Say you have some directory called “models” in the current working directory and it contains sub directories names with the format “Mass_M” where M is the mass. In each of these subdirectories is, amoung other things a file called “model.dsep”. You can load these model.dsep files in the following way

>>> from pathlib import Path
>>> from pysep.api.parallel impot pLoad
>>> modelPaths = list(Path("models").rglob("model.dsep"))
>>> models, tdirs = pLoad(modelPaths)

Module contents