Skip to main content

Deep Dive: SSE Data Serialization

In this project, Server-Sent Events (SSE) are implemented with a focus on flexible data serialization. While standard FastAPI responses typically return a single JSON object, SSE requires a stream of specifically formatted text blocks. The implementation in fastapi/sse.py and the integration in fastapi/routing.py provide two distinct paths for serializing this data: automatic JSON wrapping for simple use cases and the ServerSentEvent class for fine-grained control.

The Serialization Pipeline​

The serialization process is triggered when a path operation function uses response_class=EventSourceResponse. This class acts as a marker for the FastAPI router, which then wraps the returned generator in SSE-specific encoding logic.

The core of this logic resides in fastapi/routing.py within the _serialize_sse_item helper function. This function determines how to transform a yielded object into the SSE wire format based on its type.

Automatic JSON Serialization​

For many use cases, developers can yield plain Python objects (such as dictionaries or Pydantic models) directly. When the router encounters a non-ServerSentEvent object, it automatically:

  1. Serializes the object to JSON using jsonable_encoder.
  2. Wraps the resulting JSON string in the data: field format.

As seen in tests/test_sse.py, yielding a Pydantic model like Item(name="Plumbus") results in the following wire format:

data: {"name":"Plumbus"}

Explicit Control with ServerSentEvent​

When metadata like event types (event), unique identifiers (id), or reconnection hints (retry) are required, the ServerSentEvent class is used. This class is a Pydantic model that explicitly defines the structure of an SSE message.

# From fastapi/sse.py
class ServerSentEvent(BaseModel):
    data: Any = None
    raw_data: str | None = None
    event: str | None = None
    id: str | None = None
    retry: int | None = None
    comment: str | None = None

Data vs. Raw Data​

A critical design choice in this implementation is the distinction between the data and raw_data fields. These fields are mutually exclusive, enforced by a @model_validator in the ServerSentEvent class.

The data Field (JSON-First)​

The data field is intended for structured data. The implementation ensures that anything passed to data is JSON-serialized. This includes plain strings, which results in them being quoted on the wire.

For example, yielding ServerSentEvent(data="hello") produces:

data: "hello"

This behavior ensures consistency when the client-side EventSource listener expects to run JSON.parse() on the incoming message.

The raw_data Field (The Escape Hatch)​

The raw_data field allows developers to bypass JSON serialization entirely. This is essential for sending pre-formatted text, HTML fragments, or CSV data where JSON quotes would break the payload.

As demonstrated in tests/test_sse.py:

# Yielding raw_data
yield ServerSentEvent(raw_data="plain text without quotes")

Produces:

data: plain text without quotes

Wire Format Construction​

The final transformation from a ServerSentEvent instance to bytes is handled by format_sse_event in fastapi/sse.py. This utility function iterates through the fields and constructs the multi-line string required by the SSE specification.

One notable detail is the handling of multi-line data. The SSE spec requires that each line of a multi-line payload be prefixed with data: . The format_sse_event function handles this automatically by splitting the input string:

# From fastapi/sse.py
if data_str is not None:
    for line in data_str.splitlines():
        lines.append(f"data: {line}")

Keep-Alive and Timeouts​

To prevent proxies and load balancers from closing idle connections, the implementation includes an automatic keep-alive mechanism.

In fastapi/routing.py, a background task monitors the stream. If the generator does not yield an item within the _PING_INTERVAL (defaulting to 15 seconds), the server automatically injects a "ping" comment:

# From fastapi/sse.py
KEEPALIVE_COMMENT = b": ping\n\n"
_PING_INTERVAL: float = 15.0

This comment is ignored by the browser's EventSource API but keeps the underlying TCP connection active.

Constraints and Validation​

The implementation enforces several constraints to ensure compatibility with the SSE specification:

  • ID Safety: The id field is validated via _check_id_no_null to ensure it contains no null characters (\0), which are prohibited in the SSE wire format.
  • Retry Logic: The retry field must be a non-negative integer representing milliseconds.
  • Exclusivity: A ValueError is raised if both data and raw_data are provided, forcing a clear choice between JSON-managed or manual serialization.