Exceptions & Debugging

While glom works well when all goes as intended, it even shines when data doesn’t match expectations. glom’s error messages and exception hierarchy have been designed to maximize readability and debuggability. Read on for a listing of glom’s exceptions and how to debug them.

Exceptions

glom introduces a several new exception types designed to maximize readability and debuggability. Note that all these errors derive from GlomError, and are only raised from glom() calls, not from spec construction or glom type registration. Those declarative and setup operations raise ValueError, TypeError, and other standard Python exceptions as appropriate.

Here is a short list of links to all public exception types in glom.

Reading a glom Exception

glom errors are regular Python exceptions, but may look a little different from other Python errors. Because glom is a data manipulation library, glom errors include a data traceback, interleaving spec and target data.

For example, let’s raise an error by glomming up some data that doesn’t exist:

 1 >>> target = {'planets': [{'name': 'earth', 'moons': 1}]}
 2 >>> glom(target, ('planets', ['rings']))
 3 Traceback (most recent call last):
 4   File "<stdin>", line 1, in <module>
 5   File "/home/mahmoud/projects/glom/glom/core.py", line 1787, in glom
 6     raise err
 7 glom.core.PathAccessError: error raised while processing, details below.
 8  Target-spec trace (most recent last):
 9  - Target: {'planets': [{'name': 'earth', 'moons': 1}]}
10  - Spec: ('planets', ['rings'])
11  - Spec: 'planets'
12  - Target: [{'name': 'earth', 'moons': 1}]
13  - Spec: ['rings']
14  - Target: {'name': 'earth', 'moons': 1}
15  - Spec: 'rings'
16 glom.core.PathAccessError: could not access 'rings', part 0 of Path('rings'), got error: KeyError('rings')

Let’s step through this output:

  • Line 1: We created a planet registry, similar to the one in the glom Tutorial.

  • Line 2-3: We try to get a listing of rings of all the planets. Instead, we get a Python traceback.

  • Line 7: We see we have a PathAccessError.

  • Line 8-9: The “target-spec trace”, our data stack, begins. It always starts with the target data as it was passed in.

  • Line 10: Next is the top-level spec, as passed in: ('planets', ['rings'])

  • Line 11: glom takes the first part of the spec from line 9, 'planets', to get the next target.

  • Line 12: Because the spec on line 11 updated the current target, glom outputs it. When a spec is evaluated but the target value is unchanged, the target is skipped in the trace.

  • Line 14-15: We get to the last two lines, which include the culprit target and spec

  • Line 16: Finally, our familiar PathAccessError message, with more details about the error, including the original KeyError('rings').

This view of glom evaluation answers many of the questions a developer or user would ask upon encountering the error:

  • What was the data?

  • Which part of the spec failed?

  • What was the original error?

The data trace does this by peeling away at the target and spec until it hones in on the failure. Both targets and specs in traces are truncated to terminal width to maximize readability.

Note

If for some reason you need the full Python stack instead of the glom data traceback, pass glom_debug=True to the top-level glom call.

Reading Branched Exceptions

Some glom spec types, like Coalesce and Switch, can try multiple specs in succession. These “branching” specs can also get multiple exceptions.

Initially, debugging data for these branching specs was limited. But in v20.7.0, branching error trees were introduced, exposing information about every spec and target attempted before raising the final exception.

All the exception reading advice in the “Reading a glom Exception” section applies, but there’s a bit of extra formatting to visualize the error tree in the target-spec trace.

Let’s step line by line through a Coalesce error tree:

 1 >>> target = {'n': 'nope', 'xxx': {'z': {'v': 0}}}
 2 >>> glom(target, Coalesce(('xxx', 'z', 'n'), 'yyy'))
 3 Traceback (most recent call last):
 4   File "tmp.py", line 9, in _make_stack
 5     glom(target, spec)
 6   File "/home/mahmoud/projects/glom/glom/core.py", line 2029, in glom
 7     raise err
 8 glom.core.CoalesceError: error raised while processing, details below.
 9  Target-spec trace (most recent last):
10  - Target: {'n': 'nope', 'xxx': {'z': {'v': 0}}}
11  + Spec: Coalesce(('xxx', 'z', 'n'), 'yyy')
12  |\ Spec: ('xxx', 'z', 'n')
13  || Spec: 'xxx'
14  || Target: {'z': {'v': 0}}
15  || Spec: 'z'
16  || Target: {'v': 0}
17  || Spec: 'n'
18  |X glom.core.PathAccessError: could not access 'n', part 0 of Path('n'), got error: KeyError('n')
19  |\ Spec: 'yyy'
20  |X glom.core.PathAccessError: could not access 'yyy', part 0 of Path('yyy'), got error: KeyError('yyy')
21 glom.core.CoalesceError: no valid values found. Tried (('xxx', 'z', 'n'), 'yyy') and got (PathAccessError, PathAccessError) (at path ['xxx', 'z'])
  • Line 1-10: Standard fare for glom use and error behavior, see “Reading a glom Exception

  • Line 11: We see a “+” when starting a branching spec. Each level of branch adds a “|” on the left to help track nesting level.

  • Line 12: We see a “\” indicating a new branch of the root branching spec.

  • Line 13-17: Traversing downward as usual until…

  • Line 18: We see an “X” indicating our first exception, causing the failure of this branch.

  • Line 19: We see a “\” which starts our next branch.

  • Line 20: We see an “X” indicating our second and last exception, causing the failure of this branch.

  • Line 21: The last line is our root level exception, dedented, same as any other glom error.

Apart from the formatting, error branching doesn’t change any other semantics of the glom exception being raised.

Debugging

Good error messages are great when the data has a problem, but what about when a spec is incorrect?

Even the most carefully-constructed specifications eventually need debugging. If the error message isn’t enough to fix your glom issues, that’s where Inspect comes in.

class glom.Inspect(*a, **kw)[source]

The Inspect specifier type provides a way to get visibility into glom’s evaluation of a specification, enabling debugging of those tricky problems that may arise with unexpected data.

Inspect can be inserted into an existing spec in one of two ways. First, as a wrapper around the spec in question, or second, as an argument-less placeholder wherever a spec could be.

Inspect supports several modes, controlled by keyword arguments. Its default, no-argument mode, simply echos the state of the glom at the point where it appears:

>>> target = {'a': {'b': {}}}
>>> val = glom(target, Inspect('a.b'))  # wrapping a spec
---
path:   ['a.b']
target: {'a': {'b': {}}}
output: {}
---

Debugging behavior aside, Inspect has no effect on values in the target, spec, or result.

Parameters
  • echo (bool) – Whether to print the path, target, and output of each inspected glom. Defaults to True.

  • recursive (bool) – Whether or not the Inspect should be applied at every level, at or below the spec that it wraps. Defaults to False.

  • breakpoint (bool) – This flag controls whether a debugging prompt should appear before evaluating each inspected spec. Can also take a callable. Defaults to False.

  • post_mortem (bool) – This flag controls whether exceptions should be caught and interactively debugged with pdb on inspected specs.

All arguments above are keyword-only to avoid overlap with a wrapped spec.

Note

Just like pdb.set_trace(), be careful about leaving stray Inspect() instances in production glom specs.