saveobj#

saveobj(filename='default.obj', obj=None, folder=None, method='pickle', compression='gzip', compresslevel=5, verbose=0, sanitizepath=True, die=False, allow_empty=False, **kwargs)#

Save any object to disk

This function is similar to pickle.dump() in that it serializes the object to a file. Key differences include:

  • It takes care of opening/closing the file for writing

  • It compresses the data by default

  • It supports different serialization methods (e.g. pickle or dill)

Once an object is saved, it can be loaded with sc.load(). Note that sc.save()/sc.saveobj() are identical.

Note 1: Since this function relies on pickle, it can potentially execute arbitrary code, so you should only use it with sources you trust. For more information, see: https://docs.python.org/3/library/pickle.html

Note 2: When a pickle file is loaded, Python imports any modules that are referenced in it. This is a problem if module has been renamed (in which case the pickle usually can’t be opened). For more robustness (e.g. to pickle custom classes), use method='dill' and/or the sc.savearchive()/sc.loadarchive() functions.

If you do not need to save arbitrary Python and just need to save data, consider saving the data in a standard format, e.g. JSON (sc.savejson()). For large amounts of tabular data, also consider formats like HDF5 or PyArrow.

Parameters:

  • filename (str/path) – the filename or path to save to; if None, return an io.BytesIO filestream instead of saving to disk

  • obj (anything) – the object to save

  • folder (str) – optional additional folder, passed to sc.makepath()

  • method (str) – whether to use ‘pickle’ (default) or ‘dill’

  • compression (str) – type of compression to use: ‘gzip’ (default), ‘zstd’ (zstandard), or ‘none’ (no compression)

  • compresslevel (int) – the level of gzip/zstd compression (1 to 9 for gzip, -7 to 22 for zstandard, default 5)

  • verbose (int) – level of detail to print

  • sanitizepath (bool) – whether to sanitize the path prior to saving

  • die (bool) – whether to fail if the object can’t be pickled (else, try dill); if die is ‘never’

  • allow_empty (bool) – whether or not to allow “None” to be saved (usually treated as an error)

  • kwargs (dict) – passed to pickle.dumps() (or dill.dumps())

Examples:

# Standard usage
my_obj = ['this', 'is', 'my', 'custom', {'object':44}]
sc.save('myfile.obj', my_obj)
loaded = sc.load('myfile.obj')
assert loaded == my_obj

# Use dill instead, to save custom classes as well
class MyClass:
    def __init__(self, x):
        self.data = np.random.rand(100) + x
    def sum(self):
        return self.data.sum()
my_class = MyClass(10)

sc.save('my_class.obj', my_class, method='dill', compression='zstd')
loaded = sc.load('my_class.obj') # With dill, can be loaded anywhere, not just in the same script
assert loaded.sum() == my_class.sum()

See also sc.zsave() (identical except defaults to zstandard compression).

New in version 1.1.1: removed Python 2 support.
New in version 1.2.2: automatic swapping of arguments if order is incorrect; correct passing of arguments
New in version 2.0.4: “die” argument for saving as dill
New in version 2.1.0: “zstandard” compression method
New in version 3.0.0: “allow_empty” argument; removed “args”
On this page