pysep.misc.runDB package¶

Submodules¶

pysep.misc.runDB.base module¶

pysep.misc.runDB.dbmanage module¶

dbmanage module¶

Author: Thomas M. Boudreaux

Created: February 2022

Last Modified: June 2022

Control module for DSEP ensemble model

Classes¶

model_DB
DSEP ensemble database. Meant to make it easier to organize large number of DSEP models

class pysep.misc.runDB.dbmanage.model_DB(dbName: str, saveDir: str, mode: str = 'r', autoInit: bool = True)¶

Bases: object

DSEP ensemble database. Meant to make it easier to organize large numbers of DSEP models.

name¶

path where the database file will be stored. If this is set to “:memory:” then the database file will not be saved to disk but will instead live in memory. This later option is really only helpful when debugging

Type: str

saveDir¶

Path where models will be moved into if the dababase job distributor is used

Type: str

Examples

Creating a database from a list of models. Assume that there is some list of stellarModels stored in the list “models”. Assume there is some directory in the local directory called “modelResults”.

>>> from pysep.misc.runDB.dbmanage import model_DB
>>> DB = model_DB("DSEPModels.db", "modelResults", mode="wr")
>>> DB.enroll_models_to_dir(models)

By using enroll_models_to_dir we put each model in a randomly generated directory within the folder modelResults. This makes it harder for a human to read; however, it means we don’t have to worry about name conflicts.

The general pattern should be then use the DB to figure out what model is where.

Now that we have these models enrolled in the database we can query them

>>> for model in DB.iter_models():
>>>     print(model.premsMass, model.output)

If you want to see say which run cards are tied with each model

>>> for model in DB.iter_models():
>>>     for card in model.cards:
>>>         print(card)

Eventually, I will impliment more quering abilities. The DB’s schema is already built to allow this quite easily.

enroll_model(model: pysep.dsep.dsepModel.stellarModel, commit: bool = True, stash: bool = False)¶

Entroll DSEP stellarModel into DB. Will only work if mode is set to ‘ar’ or ‘wr’.

Parameters

model (pysep.dsep.stellarModel) – Model to enroll into database. Model will be stored as a one-to-many relation where the non kind card keys of the control namelist file make up most of the columns in the parent table (along with the 2 description keys, the path, the pre main sequence mass, and the basename of the path, and the time of evolution if the model has been evolved). The keys from each kind card from the control namelist file are stored in the child database.
commit (bool, default=True) – If true the model will be commited automatically, if not that will be left to the calling function or user. Commit will only happen is stash is set to False
stash (bool, default=False) – If true the model will be added to the session in this function, if false the model will be added to the model stash so that it may be bulk added latter. If stash is set to true this function will never commit (regardless of whether commit is set to true as nothing will have been added to the session to commit to the database).

Raises

RuntimeError – If database mode is set to ‘r’ (readonly)

Examples

Assume You have some model called “m100GS98” and an initialized DSEP DB called “DB”, then you can enroll that model

>>> DB.enroll_model(m100GS98)

enroll_model_to_dir(model: pysep.dsep.dsepModel.stellarModel, commit: bool = True, stash: bool = False) → pysep.dsep.dsepModel.stellarModel¶

Enroll a model and set its output directory to be some directory within saveDir. The basename of this will be a 20 char long random ascii string The reason to do this is we can store any arbitrary models without having to worry about name conflicts that might come up if we tried to name these directories based on some combination of parameters (unless those names included literally all of the information defining the run which would be unwieldy).

If you do this you will basically need to use the DB to access these models again cause the directory structure is totally obscured.

Note, this enroll method should only be used before the models are evolved

Parameters

model (pysep.dsep.stellarModel) – Some model to be enrolled in the DB and to have its output path changed to saveDir/<randomasciiString>/
commit (bool, default=True) – If true the model will be commited automatically, if not that will be left to the calling function or user. Commit will only happen is stash is set to False
stash (bool, default=False) – If true the model will be added to the session in this function, if false the model will be added to the model stash so that it may be bulk added latter. If stash is set to true this function will never commit (regardless of whether commit is set to true as nothing will have been added to the session to commit to the database).

Returns

nModel – Stellar model with output path updated

Return type

stellarModel

Raises

RuntimeError – If the model being enrolled has already been evolved

enroll_models(models: list)¶

Automatically enroll a list of models

Parameters: models (list[pysep.dsep.stellarModel]) – List of models to be enrolled

enroll_models_to_dir(models: list, pbar: bool = False) → list¶

Enroll a set of models to a set of guarenteed unique directories.

Parameters

models (list[pysep.dsep.stellarModel]) – List of models to enroll. Each one will be saved at a guarenteed unique directory within saveDir. This makes it hard for you by youself to parse through but it means that any arbitrary number of models can be run (limited by disk space only) and indexed by UUIDs in the DB.
pbar (bool, default=False) – Sometime enrolling models can take some time, show a progress bar if True

Returns

nModels – list of stellar models with their output directories updated

Return type

list[stellarModel]

iter_models(orderByMass: bool = False)¶

Iterate over all enrolled models in DB

Parameters: orderByMass (bool, default=False) – Flag to order iterated models by mass
Yields: modelInstance (pysep.misc.runDB.schema.Model) – Model object which can be further quieried for its parameters or run cards

Examples

Say you have some DB called DB with enrolled models. The models have been evolved (after enrolling) and stashed (to the file model.dsep) and you want to look at their iso output files.

>>> from pysep.dsep import load_model
>>> for model in DB.iter_models():
>>>     dsepModelFile = os.path.join(model.path, "model.dsep")
>>>     loadedModel, tdir = load_model(dsepModelFile)
>>>     print(loadedModel.iso())

This is the ideal general access pattern when using the model_DB

pysep.misc.runDB.schema module¶

Author: Thomas M. Boudreaux

Created: February 2022

Last Modified: February 2022

Schema for model_DB / DSEP ensemble models dababase.

class pysep.misc.runDB.schema.KindCard(**kwargs)¶

Bases: sqlalchemy.orm.decl_api.Base

cmixla¶

endage¶

id¶

kindrn¶

lfirst¶

lsenv0a¶

model¶

modelID¶

nmodls¶

numrun¶

rsclm¶

rsclx¶

rsclz¶

senv0a¶

xenv0a¶

zenv0a¶

class pysep.misc.runDB.schema.Model(**kwargs)¶

Bases: sqlalchemy.orm.decl_api.Base

basepath¶

cards¶

descrip1¶

descrip2¶

fo95cobin¶

iafe¶

id¶

ipver¶

lalex95¶

lbnout¶

lcorr¶

lhb¶

liso¶

lopal95¶

lpulse¶

lrwsh¶

ltrack¶

lzams¶

mix¶

nbn¶

numrun¶

opecalex1¶

opecalex2¶

opecalex3¶

opecalex4¶

opecalex5¶

opecalex6¶

opecalex7¶

opecalex8¶

path¶

premsMass¶

rundatetime¶

zalex¶

pysep.misc.runDB.utils module¶

Author: Thomas M. Boudreaux

Created: February 2022

Last Modified: February 2022

Utilities for use with DSEP ensemble database

pysep.misc.runDB.utils.connect_session(engine)¶

Connect an engine to a session using session maker

Parameters: engine (sqlalchemy.engine.base.Engine) – usable sqlalchemy engine
Returns: usable sqlalchemy session
Return type: session

pysep.misc.runDB.utils.construct_engine(name: str, verbose: bool = False)¶

build an engine with sqlalchemy

Parameters

name (str) – Name of the engine, if this is “:memory:” the DB will be stored in memory, otherwise it will be written to disk at the path = to name.
verbose (bool, default=False) – If true echo is set to true, this means sqlalchemy will print out all of its SQL language commands.

Returns

engine – usable sqlalchemy engine

Return type

sqlalchemy.engine.base.Engine

pysep.misc.runDB.utils.construnct_schema(name: str, verbose: bool = False)¶

Build any schema loaded into the declarative_base Base

Parameters

name (str) – Name of the engine, if this is “:memory:” the DB will be stored in memory, otherwise it will be written to disk at the path = to name.
verbose (bool, default=False) – If true echo is set to true, this means sqlalchemy will print out all of its SQL language commands.

Returns

engine – usable sqlalchemy engine

Return type

sqlalchemy.engine.base.Engine

pysep.misc.runDB.utils.format_card_insert_dict(cdict: dict) → dict¶

Format a dictionary to be enrolled in the DB for a kind card so that it only has valid keys in it (removes things like descrip) which go elsewhere in the DB.

Parameters: cdict (dict) – Dictionary of a kind card from pysep.io.nml.control.crun.as_dict()
Returns: outDict – Same as cdict but only including keys in cdict that the DB kind card schema can accept / has as a column
Return type: dict

pysep.misc.runDB.utils.format_model_insert_dict(mdict: dict, d1: str, d2: str) → dict¶

Format a dictionary to be enrolled in the DB for a full model so that it only has valid keys in it. Adding descip in and splitting opecalex into 8 columns.

Parameters

mdict (dict) – Dictionary of a kind card from pysep.io.nml.control.cnml.as_dict()[‘additional_attrs’]
d1 (str) – String for descrip[0]
d2 (str) – String for descrip[1]

Returns

outDict – Same as mdict but only including keys in mdict that the DB model schema can accept / has as a column

Return type

dict

Module contents¶

Utilities to help keep many DSEP runs organized