Skip to content

PlanMaterialization

pydantable.PlanMaterialization

Bases: str, Enum

How terminal materialization of a lazy plan is scheduled and consumed.

Values are lowercase strings suitable for stable logging and JSON.

Blocking

Synchronous APIs on the **current thread**:
:meth:`~pydantable.DataFrame.collect`,
:meth:`~pydantable.DataFrame.to_dict`,
:meth:`~pydantable.DataFrame.to_polars`,
:meth:`~pydantable.DataFrame.to_arrow`,
:meth:`~pydantable.DataFrame.collect_batches`, etc.

Async

``await`` coroutines so the event loop can run other tasks while the
engine is awaited (or work is offloaded to a thread pool when the wheel
does not expose native async execution): :meth:`~pydantable.DataFrame.acollect`,
:meth:`~pydantable.DataFrame.ato_dict`,
:meth:`~pydantable.DataFrame.ato_polars`,
:meth:`~pydantable.DataFrame.ato_arrow` (and :class:`~pydantable.DataFrameModel`
mirrors such as ``arows`` / ``ato_dicts``).

Deferred

Start materialization in the background and await the result later:
:meth:`~pydantable.DataFrame.submit` returns a
:class:`~pydantable.dataframe.ExecutionHandle`; ``await handle.result()``
matches :meth:`~pydantable.DataFrame.collect` for the same arguments.

Chunked

After **one** full engine collect, yield **column dict** chunks (same slicing
contract as :meth:`~pydantable.DataFrame.collect_batches`): **sync**
:meth:`~pydantable.DataFrame.stream` or **async**
:meth:`~pydantable.DataFrame.astream`. Not out-of-core Polars streaming;
see **EXECUTION** (Rust engine) in the docs.
Source code in python/pydantable/materialization.py 
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
class PlanMaterialization(str, Enum):
    """How terminal materialization of a lazy plan is scheduled and consumed.

    Values are **lowercase strings** suitable for stable logging and JSON.

    **Blocking**

        Synchronous APIs on the **current thread**:
        :meth:`~pydantable.DataFrame.collect`,
        :meth:`~pydantable.DataFrame.to_dict`,
        :meth:`~pydantable.DataFrame.to_polars`,
        :meth:`~pydantable.DataFrame.to_arrow`,
        :meth:`~pydantable.DataFrame.collect_batches`, etc.

    **Async**

        ``await`` coroutines so the event loop can run other tasks while the
        engine is awaited (or work is offloaded to a thread pool when the wheel
        does not expose native async execution): :meth:`~pydantable.DataFrame.acollect`,
        :meth:`~pydantable.DataFrame.ato_dict`,
        :meth:`~pydantable.DataFrame.ato_polars`,
        :meth:`~pydantable.DataFrame.ato_arrow` (and :class:`~pydantable.DataFrameModel`
        mirrors such as ``arows`` / ``ato_dicts``).

    **Deferred**

        Start materialization in the background and await the result later:
        :meth:`~pydantable.DataFrame.submit` returns a
        :class:`~pydantable.dataframe.ExecutionHandle`; ``await handle.result()``
        matches :meth:`~pydantable.DataFrame.collect` for the same arguments.

    **Chunked**

        After **one** full engine collect, yield **column dict** chunks (same slicing
        contract as :meth:`~pydantable.DataFrame.collect_batches`): **sync**
        :meth:`~pydantable.DataFrame.stream` or **async**
        :meth:`~pydantable.DataFrame.astream`. Not out-of-core Polars streaming;
        see **EXECUTION** (Rust engine) in the docs.
    """

    BLOCKING = "blocking"
    ASYNC = "async"
    DEFERRED = "deferred"
    CHUNKED = "chunked"