adding transitions using Pysyncrosim

[davidanderson]

I’m working up a process to build models using Pysyncrosim. Part of the the process is to be separate the transitions between a base set and some suite of optional sets.
I came up with the following code to add transitions. Is there a better/faster way?

# create base transitions
TransitionType = pd.read_excel(os.path.join(path_to_stsim_files,’TransitionType_Base.xlsx’),sheet_name=’Transition Type’)
TransitionType.rename(columns={‘Map Legend’:’Legend’},inplace=True)
r3_project.save_datasheet(‘stsim_TransitionType’,TransitionType)

# add climate change transitions
TransitionType_climate = pd.read_excel(os.path.join(path_to_stsim_files,’TransitionType _ClimateChange.xlsx’),sheet_name=’Transition Type’)
TransitionType_climate.rename(columns={‘Map Legend’:’Legend’},inplace=True)
r3_project.save_datasheet(name=’stsim_TransitionType’,data=pd.concat([r3_project.datasheets(‘stsim_TransitionType’),TransitionType_climate]))

I could concatenate the dataframes together before creating a data sheet. I’m not doing that as in the final product I won’t have the two dataframes being read in at the same time.

[katie-birchard]

Hi David,

Currently, the only way to add to existing Datasheets in pysyncrosim is to reload the Datasheet as a pandas DataFrame, concatenate with your new data, and then overwrite the old saved Datasheet (as you are doing in your example above). However, the next version of pysyncrosim (v1.0.19) will have a new append argument in the save_datasheet() method that will take advantage of the SyncroSim core software to efficiently append new data to existing Datasheets. The next version will become available early next week.

Cheers,
Katie

[davidanderson]

Thanks Katie for that update. The append would be cleaner than how I’m currently doing it.

I was doing some more work and noticed a odd behavior. When I rerun the command

TransitionType = pd.read_excel(os.path.join(path_to_stsim_files,’TransitionType_Base.xlsx’),sheet_name=’Transition Type’)
TransitionType.rename(columns={‘Map Legend’:’Legend’},inplace=True)
r3_project.save_datasheet(‘stsim_TransitionType’,TransitionType)

Then check the transitions datasheet using r3_project.datasheets(‘stsim_TransitionType’)
the returned dataframe is the size of the expanded on (477) rows, not the 114 rows that are in the spreadsheet ’TransitionType_Base.xlsx’.
Resetting the transitions datasheet back to a smaller set of values is not working the way I think it would work.

I did do another test of adding a single unique transition using the same technique. That did add another row.

Poking a bit further, there does not seem to be a way to empty out the transitions datasheet. I tried this:
r3_project.save_datasheet(name=’stsim_TransitionType’,data=r3_project.datasheets(‘stsim_TransitionType’,empty=True))
which returns the following error:
RuntimeError: No data was found in the specified CSV file.

Am I missing something here?

[katie-birchard]

Hi David,

It looks like there is a bug in save_datasheet(). You should be able to overwrite the currently saved Datasheet and there should be no issue in saving an empty DataFrame as a Datasheet. We’ll fix these issues for the next release (v1.0.19) as well. Thanks for letting us know!

Cheers,
Katie

[davidanderson]

Glad to help.

[katie-birchard]

Hi David,

Version 1.0.19 of pysyncrosim is now available from conda-forge and PyPI. We’ve added two new arguments to the save_datasheet() method – append and force. When saving a Project-scoped Datasheet, the default is for append to be set to True, since overwriting this Datasheet can have consequences on Scenario-scoped Datasheets that reference Project-scoped Datasheet values. In order to overwrite a Project-scoped Datasheet, you must set append to False and force to True. This is the same functionality as the saveDatasheet() function in rsyncrosim. You can also now delete information in a Datasheet by saving an empty DataFrame (with force=True for Project-scoped Datasheets).

Hope this makes your modeling a bit easier!

Katie

[Author]

Posts