lgdo package

LEGEND Data Objects (LGDO) are defined in the LEGEND data format specification. This package serves as the Python implementation of that specification. The general strategy for the implementation is to dress standard Python and NumPy objects with an attr dictionary holding LGDO metadata, plus some convenience functions. The basic data object classes are:

• LGDO: abstract base class for all LGDOs

• Scalar: typed Python scalar. Access data via the value attribute

• Array: basic numpy.ndarray. Access data via the nda attribute.

• FixedSizeArray: basic numpy.ndarray. Access data via the nda attribute.

• ArrayOfDetectorIDs: an array of uint32 values encoding the name of a detector in the LEGEND experiment. Access data via the nda attribute.

• ArrayOfEqualSizedArrays: multi-dimensional numpy.ndarray. Access data via the nda attribute.

• VectorOfVectors: an n-dimensional variable length array of variable length arrays. Implemented as a pair of datasets: flattened_data holding the raw data (Array or VectorOfVectors, if the vector dimension is greater than 2), and cumulative_length (always an Array) whose i-th element is the sum of the lengths of the vectors with index <= i

• VectorOfEncodedVectors: an array of variable length encoded arrays. Implemented as a VectorOfVectors encoded_data holding the encoded vectors and an Array decoded_size specifying the size of each decoded vector. Mainly used to represent a list of compressed waveforms.

• ArrayOfEncodedEqualSizedArrays: an array of equal sized encoded arrays. Similar to VectorOfEncodedVectors except for decoded_size, which is now a scalar.

• Struct: a dictionary containing LGDO objects. Derives from dict

• Table: a Struct whose elements (“columns”) are all array types with the same length (number of rows)

• Histogram: holds an array of histogrammed data, and the associated binning of arbitrary dimensionality.

Currently the primary on-disk format for LGDO object is LEGEND HDF5 (LH5) files. IO is done via the class lh5_store.LH5Store. LH5 files can also be browsed easily in python like any HDF5 file using h5py.

Subpackages

• lgdo.compression package
• lgdo.lh5 package
• lgdo.types package
  • Submodules
  • lgdo.types.array module
    • Array
      • Array.append()
      • Array.datatype_name()
      • Array.dtype
      • Array.form_datatype()
      • Array.get_capacity()
      • Array.insert()
      • Array.nda
      • Array.replace()
      • Array.reserve_capacity()
      • Array.resize()
      • Array.shape
      • Array.trim_capacity()
      • Array.view_as()
  • lgdo.types.arrayofdetectorids module
    • ArrayOfDetectorIDs
      • ArrayOfDetectorIDs.form_datatype()
  • lgdo.types.arrayofequalsizedarrays module
    • ArrayOfEqualSizedArrays
      • ArrayOfEqualSizedArrays.datatype_name()
      • ArrayOfEqualSizedArrays.form_datatype()
      • ArrayOfEqualSizedArrays.to_vov()
      • ArrayOfEqualSizedArrays.view_as()
  • lgdo.types.arrow module
    • _arrow_col_to_lgdo()
    • _deserialize_attr()
    • _lgdo_col_to_arrow()
    • _nested_fixed_list_to_nda()
    • _serialize_attr()
    • arrow_to_lgdo()
    • lgdo_to_arrow()
  • lgdo.types.encoded module
    • ArrayOfEncodedEqualSizedArrays
      • ArrayOfEncodedEqualSizedArrays.append()
      • ArrayOfEncodedEqualSizedArrays.datatype_name()
      • ArrayOfEncodedEqualSizedArrays.form_datatype()
      • ArrayOfEncodedEqualSizedArrays.get_capacity()
      • ArrayOfEncodedEqualSizedArrays.insert()
      • ArrayOfEncodedEqualSizedArrays.replace()
      • ArrayOfEncodedEqualSizedArrays.reserve_capacity()
      • ArrayOfEncodedEqualSizedArrays.resize()
      • ArrayOfEncodedEqualSizedArrays.trim_capacity()
      • ArrayOfEncodedEqualSizedArrays.view_as()
    • VectorOfEncodedVectors
      • VectorOfEncodedVectors.datatype_name()
      • VectorOfEncodedVectors.form_datatype()
      • VectorOfEncodedVectors.get_capacity()
      • VectorOfEncodedVectors.insert()
      • VectorOfEncodedVectors.replace()
      • VectorOfEncodedVectors.reserve_capacity()
      • VectorOfEncodedVectors.resize()
      • VectorOfEncodedVectors.trim_capacity()
      • VectorOfEncodedVectors.view_as()
  • lgdo.types.fixedsizearray module
    • FixedSizeArray
      • FixedSizeArray.datatype_name()
      • FixedSizeArray.view_as()
  • lgdo.types.histogram module
    • Histogram
      • Histogram.Axis
      • Histogram.add_field()
      • Histogram.binning
      • Histogram.fill()
      • Histogram.isdensity
      • Histogram.remove_field()
      • Histogram.view_as()
      • Histogram.weights
  • lgdo.types.lgdo module
    • LGDO
      • LGDO.datatype_name()
      • LGDO.form_datatype()
      • LGDO.getattrs()
      • LGDO.view_as()
    • LGDOCollection
      • LGDOCollection.append()
      • LGDOCollection.clear()
      • LGDOCollection.get_capacity()
      • LGDOCollection.insert()
      • LGDOCollection.replace()
      • LGDOCollection.reserve_capacity()
      • LGDOCollection.resize()
      • LGDOCollection.trim_capacity()
  • lgdo.types.scalar module
    • Scalar
      • Scalar.datatype_name()
      • Scalar.form_datatype()
      • Scalar.view_as()
  • lgdo.types.struct module
    • Struct
      • Struct.add_field()
      • Struct.datatype_name()
      • Struct.form_datatype()
      • Struct.items()
      • Struct.keys()
      • Struct.remove_field()
      • Struct.update()
      • Struct.update_datatype()
      • Struct.values()
      • Struct.view_as()
    • _get_struct_fields()
    • _is_struct_datatype()
    • _sort_datatype_fields()
    • _struct_datatype_equal()
  • lgdo.types.table module
    • Table
      • Table.add_column()
      • Table.add_field()
      • Table.datatype_name()
      • Table.eval()
      • Table.flatten()
      • Table.get_capacity()
      • Table.get_dataframe()
      • Table.insert()
      • Table.join()
      • Table.remove_column()
      • Table.replace()
      • Table.reserve_capacity()
      • Table.resize()
      • Table.trim_capacity()
      • Table.view_as()
    • _ak_to_lgdo_or_col_dict()
  • lgdo.types.vectorofvectors module
    • VectorOfVectors
      • VectorOfVectors._ak_is_jagged()
      • VectorOfVectors._ak_is_valid()
      • VectorOfVectors._set_vector_unsafe()
      • VectorOfVectors.append()
      • VectorOfVectors.cumulative_length
      • VectorOfVectors.datatype_name()
      • VectorOfVectors.dtype
      • VectorOfVectors.form_datatype()
      • VectorOfVectors.get_capacity()
      • VectorOfVectors.insert()
      • VectorOfVectors.ndim
      • VectorOfVectors.replace()
      • VectorOfVectors.reserve_capacity()
      • VectorOfVectors.resize()
      • VectorOfVectors.to_aoesa()
      • VectorOfVectors.trim_capacity()
      • VectorOfVectors.view_as()
    • _OffsetArrayView
      • _OffsetArrayView._refresh_view()
      • _OffsetArrayView.get_capacity()
      • _OffsetArrayView.insert()
      • _OffsetArrayView.reserve_capacity()
      • _OffsetArrayView.resize()
      • _OffsetArrayView.trim_capacity()
    • _to_aoesa()
  • lgdo.types.vovutils module
    • _nb_build_cl()
    • _nb_explode()
    • _nb_explode_cl()
    • build_cl()
    • explode()
    • explode_arrays()
    • explode_cl()
  • lgdo.types.waveformtable module
    • WaveformTable
      • WaveformTable.dt
      • WaveformTable.dt_units
      • WaveformTable.resize_wf_len()
      • WaveformTable.t0
      • WaveformTable.t0_units
      • WaveformTable.values_units
      • WaveformTable.view_as()
      • WaveformTable.wf_len

Submodules

lgdo.logging module

This module implements some helpers for setting up logging.

lgdo.logging.setup(level=20, logger=None)

Setup a colorful logging output.

If logger is None, sets up only the lgdo logger.

Parameters:

• level (int) – logging level (see logging module).

• logger (Logger | None) – if not None, setup this logger.

Examples

>>> from lgdo import logging
>>> logging.setup(level=logging.DEBUG)

lgdo.units module

lgdo.utils module

Implements utilities for LEGEND Data Objects.

class lgdo.utils.NumbaDefaults

Bases: MutableMapping

Bare-bones class to store some Numba default options. Defaults values are set from environment variables

Examples

Set all default option values for a processor at once by expanding the provided dictionary:

>>> from numba import guvectorize
>>> from lgdo.utils import numba_defaults_kwargs as nb_kwargs
>>> @guvectorize([], "", **nb_kwargs, nopython=True) # def proc(...): ...

Customize one argument but still set defaults for the others:

>>> from lgdo.utils import numba_defaults as nb_defaults
>>> @guvectorize([], "", **nb_defaults(cache=False) # def proc(...): ...

Override global options at runtime:

>>> from lgdo.utils import numba_defaults
>>> # must set options before explicitly importing lgdo modules!
>>> numba_defaults.cache = False
>>> numba_defaults.boundscheck = True
>>> from lgdo import compression # imports of numbified functions happen here
>>> compression.encode(...)

lgdo.utils.get_element_type(obj)

Get the LGDO element type of a scalar or array.

For use in LGDO datatype attributes.

Parameters:

obj (object) – if a str, will automatically return string if the object has a numpy.dtype, that will be used for determining the element type otherwise will attempt to case the type of the object to a numpy.dtype.

Returns:

element_type – A string stating the determined element type of the object.

Return type:

str

lgdo.utils.getenv_bool(name, default=False)

Get environment value as a boolean, returning True for 1, t and true (caps-insensitive), and False for any other value and default if undefined.

Return type:

bool