pyTOPSScrape.api package¶

Submodules¶

pyTOPSScrape.api.api module¶

Author: Thomas M. Boudreaux

Created: September 2021

Last Modified: September 2022

Psuedo API for querying TOPS webform

pyTOPSScrape.api.api.TOPS_query(mixString: str, mixName: str, nAttempts: int) → bytes¶

Query TOPS form and retry n times

Parameters

mixString (string) – string in the form of: “massFrac0 Element0 massFrac1 Element1 …” which will be submitted in the webform for mixture
mixName (string) – name to be used in the webform
nAttemptes (int) – How many times to reattempt after a failure.

Returns

tableHTML – Table queried from TOPS cite.

Return type

bytes

pyTOPSScrape.api.api.TOPS_query_async_distributor(compList: list, outputDirectory: str, njobs: int = 10)¶

Distributes TOPS query jobs to different threads and gathers the results together. Writes out output.

Parameters

comList (list) – 3D list containing the mass frac for each element for each rescaled composition requested. For n compositions this should be of the shape (n, 30, 2). n compositions, 30 elements, then the first element of the last axis is the elemental symbol (i.e. H, He, Li, Be, etc…) and the second element is the mass fraction.
outputDirectory (str) – Path to directory to save TOPS query results to.
njobs (int, default=10) – Number of concurrent jobs to allow at a time.

pyTOPSScrape.api.api.call(aMap: str, aTable: str, outputDir: str, jobs: int)¶

Main TOPS psuedo API call function. Will save results to outputDir with file format OP:IDX_X_Y_Z.dat where IDX is the ID of the composition (parallel to DSEP composition ID), X is the classical Hydrogen mass fraction, Y is the classical Helium mass fraction, and Z is the classical metal mass fraction.

Parameters

aMap (str) – Path to the list of classical compositions to be used. List should be given as an ascii file where ecach row is X,Y,Z
aTable (str) – Path to chemical abundance table to be used as base composition.
outputDir (str) – Path to directory save TOPS query results into
jobs (int) – Number of threads to query TOPS webform on

Examples

If you have some map of rescalings you would like to used at “./rescalings.dat” and you have a base composition in the correct form at “./comp.dat” then you can generate and cache the raw output for those rescalings of that composition using

>>> call("./rescalings.dat", "./comp.dat", "./cache", 5)

This will save the cache results to the folder ./cache (note that this folder must exist before calling call. Moreover, this will query using 5 workers. You may increase this number to make call run faster; however, this will only work to a point. I find that around 20 workes is about the most that gives me any speed increase. This will somewhat depend on your computer though.

pyTOPSScrape.api.api.parse_table(html: bytes) → str¶

Parse the bytes table returned from mechanize into a string

Parameters: html (bytes) – bytes table retuend from mechanize bowser at second TOPS submission form
Returns: table – parsed html soruce in the form of a string
Return type: string

pyTOPSScrape.api.api.query_and_parse(compList: list, outputDirectory: int, i: int, nAttempts: int = 10)¶

Async coroutine to query TOPS webform, parse the output, and write that to disk.

Parameters

comList (list) – 3D list containing the mass frac for each element for each rescaled composition requested. For n compositions this should be of the shape (n, 30, 2). n compositions, 30 elements, then the first element of the last axis is the elemental symbol (i.e. H, He, Li, Be, etc…) and the second element is the mass fraction.
outputDirectory (str) – Path to write out results of TOPS webquery
i (int) – Index of composition so file name can properly keep track of where it is, even in parallel processing.
nAttempts (int, default=10) – Number of time to retry TOPS query before failing out

pyTOPSScrape.api.api.submit_TOPS_form(mixString: str, mixName: str, massFrac: bool = True) → bytes¶

Open the Los Alamos opacity website, submit a given composition and then return the resultant table.

Parameters

mixString (string) – string in the form of: “massFrac0 Element0 massFrac1 Element1 …” which will be submitted in the webform for mixture
mixName (string) – name to be used in the webform
massFrac (bool, default=True) – Submit as massFrac instead of numberFrac

Returns

tableHTML – Table quired from TOPS cite.

Return type

bytes

pyTOPSScrape.api.convert module¶

Author: Thomas M. Boudreaux

Created: September 2021

Last Modified: September 2022

Main conversion code for TOPS api, responsible for takine many TOPS results and merging them into a single OPAL formate high temperature opacity file.

pyTOPSScrape.api.convert.TOPS_2_OPAL(outputDirectory: str, aTable: str, aMap: str, output: str, nonRect: bool = False)¶

Main conversoin utility to go between some set of TOPS tables and an OPAl table. Will take a set of 126 TOPS tables where each one is the opacity for one composition over a number of temperature and densities and rearange them into one large file with 126 tables within it. Each table will be over a range of temperatures and R values. To get to R val interpolation is used.

Parameters

outputDirectory (str) – Path to directory where TOPS query results are stored
aTable (str) – Path to a reference abundance table to use when filling the header with compositional information
aMap (str) – Path to the abundance map. This should be an ascii file where each row is X,Y,Z. Each row will correspond to one rescaled composition which will be queried.
output (str) – Path to save final OPAL formated table too
nonRect (bool, default=False) – Flag to control whether output tables will be rectangular or have their corners cut off in a way consistant which how DSEP expects OPAL tables.

pyTOPSScrape.api.convert.comp_list_2_dict(compList: List[Tuple[str, float, float]]) → dict¶

Take a list containing compsoition information for a star in the form of [(‘Element Symbol’, massFraction, numberFraction),…] and convert that into a dictionary of the form: {‘Element Symbol’: (massFraction, numberFraction),..}.

Parameters: compList (Iterator) – list of the form: [(ElementSymbol, massFrac, numberFrac, ZmassFrac, massFrac Uncertanity, numberFrac Uncertanity, ZMassFrac Uncertanity),…]
Returns: Dictionary of the form {‘Element’:(massFrac, numberFrac, ZmassFrac, massFrac Uncertanity, numberFrac Uncertanity, ZMassFrac Uncertanity),…}
Return type: dict

pyTOPSScrape.api.convert.convert_rho_2_LogR(rho: numpy.ndarray, LogT: numpy.ndarray, RMO: numpy.ndarray) → Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray]¶

Maps a given kappa(rho,logT) parameter space onto a kappa(LogR, LogT) field through interpolation. The final field is the field that DSEP needs. :param rho: mass density array of size n :type rho: np.ndarray :param LogT: LogT array of size m :type LogT: np.ndarray :param RMO: Opacity Array of size m x n :type RMO: np.ndarray

Returns

targetLogR (np.ndarray(shape=19)) – Log R values which dsep requires
targetLogT (np.ndarray(shape=70)) – Log T values which dsep requires
Opacity (np.ndarray(shape=(70, 19))) – Opacity array now interpolated into LogR, LogT space from rho LogT space and sampled at the exact LogR and LogT values required.

pyTOPSScrape.api.convert.format_OPAL_header(compDict: dict) → str¶

Writes the header of the opacity table that DSEP expects. This is written to be the same length (and basically the same contents) of the header from the OPACITY project. Not sure if that is required; however, if so I am matching it.

Parameters: compDict (dict) – dictionary in the form: {‘Element’: (massFrac, numFrac), …} used to fill up the header with composition information. This is meant to be the “solar” compositon of whatever mix you are using so… [Fe/H] = 0.0, [alpha/H] = 0.0, a(He) = 10.93
Returns: The Header to be prepended to the opacity table file
Return type: string

pyTOPSScrape.api.convert.format_OPAL_table(tableDict: List[dict], compDict: dict) → str¶

Given a dictionary of tables and a composition Dictionary for solar composition in a given mixture (AGSS08, GS98, etc…) merge all the information together into a string which can be written to disk and would be the format of an opacoty project table (what DSEP expects)

Parameters

tableDict (dict) – dictionary of table elements, containing a “Summary” entry (metadata) and a “Table” entry. All occcurences of the the string “TNUM” will be replaced with the index+1 of where that table occurs in the file.
compDict (dict) – dictionary in the form: {‘Element’: (massFrac, numFrac), …} used to fill up the header with composition information. This is meant to be the “solar” compositon of whatever mix you are using so… [Fe/H] = 0.0, [alpha/H] = 0.0, a(He) = 10.93

Returns

OPALFormatted – Opacity Project formated table as a string which can be written to disk

Return type

string

pyTOPSScrape.api.convert.format_TOPS_to_OPAL(TOPSTable: str, comp: tuple, tnum: int, upperNonRect: Optional[numpy.ndarray] = None) → Tuple[str, float, float, float, str, numpy.ndarray, numpy.ndarray, numpy.ndarray]¶

Take the path to a table queried from the TOPS web form and fully convert it into a table which can be directly read by DSEP. (Note this function does not write anything to disk; however, the return products can be written to disk)

Parameters

TOPSTable (string) – path to table queried from TOPS web form
comp (dict) – composition dictionary
tnum (int) – table number
upperNonRect (np.ndarray, optional) – array describing how to fill the top of the table non rectangurally

Returns

metaLine (str) – metadata extracted from table
X (float) – Hydrogen mass fraction
Y (float) – Helium mass fraction
Z (float) – Metal mass fraction
fullTable (str) – The full table as a string which may be directly written to a file. Note that this includes all the relevant white space to allow this table to be parsed by DSEP.
LogR (np.ndarray(shape=19)) – Log R values which dsep expects
LogT (np.ndarray(shape=70)) – Log Temperature values which dsep expects.
LogRMO (np.ndarray(shape=(70, 19))) – Log rossland mean opacities for the LogT and LogR arrays

pyTOPSScrape.api.convert.format_opal_comp_table(LogR: numpy.ndarray, LogT: numpy.ndarray, LogRMO: numpy.ndarray, TNUM: int, comp: Optional[dict] = None, upperNonRect: Optional[numpy.ndarray] = None, nonRect: bool = False) → Tuple[str, str]¶

Take in all the information from a given TOPS tables and format it to the proper format for DSEP to undersand. Leave in some placeholders so that in future table can be labeld as the proper number.

Parameters

LogR (ndarray) – The Log R value array (horizontal axis of table)
LogT (ndarray) – The Log Temperature value array (vertical axis)
LogRMO (ndarray) – all of the RMO values associated with R and T
TNUM (int) – Table number
comp (dict, optional) – composition dictionary. If not provided placeholders are left in place so that it may be filled later on
upperNonRect (ndarray, default=None) – Array describing how to fill the top of the table non rectangurally This array should be of the shape (nRowsPerTable * nTables, 3). So if you have 5 tables each with 70 rows then this array should have a shape of (350,3). The first column of this array correspond to the table that the row is a member of, the second column correspond to the row in that table that the row is. so the first row of the first table would be at upperNonRect[0,:] = [0,0,…] while the first row of the second table would be at upperNonRect[n,:] [1,0,…]. The final column desribes how many of the elements, counting from the left of the opacity table should be blanked out to 99.999 (the sentinal value DSEP uses for non entries). So a row of [2,55,8] would mean that for the 56th row in the 3rd table blank out the firts 8 opacity values (opacities for the first 8 values of logR).
nonRect (bool, default = False) – Flag to control whether output tables will be rectangular or have their corners cut off in a way consistant which how DSEP expects OPAL tables.

Returns

metaLine (str) – header line for each table, may or may not have placeholders in it
fullTable (str) – full table to be places in opacity file

pyTOPSScrape.api.convert.rebuild_formated_tables(formatedTables: List[dict], interpRMO: numpy.ndarray, pContents: numpy.ndarray, upperNonRect: Optional[numpy.ndarray], nonRect: bool = False) → List[dict]¶

Iterate over a list of opacity tables and a list of desired chemical compositions then replace the contents of the table list with the newly updated RMOs frmo the interpolation.

Parameters

formatedTables (list of dicts) – List of dictionaries holding three axis each. X, Z, and LogRMO. These are the “observed” values to be interpolared
interpRMO (ndarray) – RMOs after interpolation to LogR-LogT space from rho-LogT space.
pContents (np.array(shape=(n, 3))) – Numpy array of all the compositions of length n. For a dsep n=126. Along the second axis the first column is X, the second is Y, and the third is Z.
upperNonRect (ndarray) – Array describing how to fill the top of the table non rectangurally This array should be of the shape (nRowsPerTable * nTables, 3). So if you have 5 tables each with 70 rows then this array should have a shape of (350,3). The first column of this array correspond to the table that the row is a member of, the second column correspond to the row in that table that the row is. so the first row of the first table would be at upperNonRect[0,:] = [0,0,…] while the first row of the second table would be at upperNonRect[n,:] [1,0,…]. The final column desribes how many of the elements, counting from the left of the opacity table should be blanked out to 99.999 (the sentinal value DSEP uses for non entries). So a row of [2,55,8] would mean that for the 56th row in the 3rd table blank out the firts 8 opacity values (opacities for the first 8 values of logR).
nonRect (bool, default = False) – Flag to control whether output tables will be rectangular or have their corners cut off in a way consistant which how DSEP expects OPAL tables.

Returns

formatedTables – List of dictionaries holding three axis each. X, Z, and LogRMO. These have been updated to reflect the compositions in pContents.

Return type

list of dicts

pyTOPSScrape.api.utils module¶

Author: Thomas M. Boudreaux

Created: September 2021

Last Modified: September 2021

Utilities to help with the TOPS query api

pyTOPSScrape.api.utils.format_TOPS_string(compList: list) → str¶

Format the composition list from pasrse_abundance_file into a string in the form that the TOPS web form expects for a mass fraction input.

Parameters: compList (list) – composition list in the form of: [(‘Element’, massFrac, numFrac),…]
Returns: TOPS_abundance_string – string in the form of: “massFrac0 Element0 massFrac1 Element1 …”
Return type: string

pyTOPSScrape.api.utils.validate_extant_tables(path: str, prefix: str) → bool¶

Check if there is a quiried table from TOPS for every number frac file generated by the program passed to call_num_frac.

Parameters

path (string) – Path to where the results of the number frac and TOPS query files are stored
prefix (string) – start prefix given to all abundance / number frac files

Returns

validated – Whether or not all number frac files have a corresponding TOPS opacity table

Return type

bool

Module contents¶

The api module provides query and convert functionality for pyTOPSScrape.

The function you are most likley interested in here is call().