Fix Python RecursionError in JSON Serialization

You're calling json.dumps(your_object) and Python throws RecursionError: maximum recursion depth exceeded. This usually hits when you have a nested structure that loops back on itself — say, a Person object with a manager attribute, and that manager has a direct_report pointing back to the first person. Or a Node class with a parent and children list that creates a cycle. I've seen it a hundred times in real codebases, especially with ORM models (SQLAlchemy, Django) where relationships are bidirectional by default.

The root cause is simple: Python's default JSON encoder walks your object graph recursively, and when it hits a cycle, it keeps going until it blows the recursion limit (usually 1000). It's not a bug in your data — it's the encoder not knowing when to stop.

Step 1: Confirm the cycle

Before touching code, verify it's a circular reference. Try this on your object:

import sys
print(sys.getrecursionlimit())  # usually 1000

If your object is deeply nested (list of lists of lists... 1000 layers deep), that's the culprit. But more often it's a cycle. You can trace it by writing a small function that walks the object and prints keys — but honestly, just inspect your classes. Look for attributes that reference back to the same instance or to a parent that references you.

Step 2: Break the cycle (the real fix)

The cleanest solution is to exclude the back-reference when serializing. You have a few options:

Option A: Use a custom encoder

Override default() in a json.JSONEncoder subclass and skip attributes that create cycles. Here's a generic one I use:

import json

class SafeEncoder(json.JSONEncoder):
    def default(self, obj):
        if hasattr(obj, '__dict__'):
            # Skip attributes that point back to the same object or to a parent
            return {k: v for k, v in obj.__dict__.items()
                    if not isinstance(v, type(obj)) and not isinstance(v, list)}
        return str(obj)

This is blunt — it skips any list attribute (which often contains children) and any attribute of the same type. Tune it to your needs. For a Person with a manager, you'd skip manager or store only the ID.

Option B: Clean up before serializing

If you control the input, strip the back-references before calling json.dumps. For example:

def to_serializable(person):
    return {
        'name': person.name,
        'manager_id': person.manager.id if person.manager else None
    }

This is the most predictable approach — no magic, just explicit data shaping.

Step 3: Use a library that handles cycles

If you can't restructure your objects, try python-json-tricks or blinker? Actually, the best is jsonpickle — it handles circular references out of the box by storing references as keys. Just be aware the output is not standard JSON, it's a special format. For debugging, it's great.

pip install jsonpickle
import jsonpickle
jsonpickle.encode(your_object)

Step 4: Increase recursion limit (temporary fix)

If you're absolutely sure the structure is acyclic but deeply nested, you can bump the limit:

import sys
sys.setrecursionlimit(10000)

But careful — this can crash your Python process with a segfault if you set it too high (above ~100000 on CPython), and it's a band-aid. I'd only use this for quick debugging, not production.

If it still fails

Check these:

• Did you actually apply the encoder? json.dumps(obj, cls=SafeEncoder) — easy to forget.
• Is the cycle inside a list or dict? My generic encoder above skips lists, but if your cycle is through a dict value, you need to handle that too. You might need to track visited object ids in the encoder.
• Are you serializing a generator or a custom iterator? Those also cause recursion errors sometimes — convert to a list first.
• If you're using SQLAlchemy, check for lazy-loaded relationships that trigger during serialization — that's a different beast. Use marshmallow or Pydantic to control output.

Remember, the RecursionError isn't random — it's your data telling you it has a loop. Find the loop and break it, and you'll never see this error again.

Small note: if you're serializing to send over an API, you probably don't want the entire object graph anyway. Design a DTO (Data Transfer Object) that only has the fields you need.

I've fixed this error more times than I can count, and 90% of the time it's a circular relationship in an ORM model. Skip the recursion limit hack — fix the data structure.