Skip to content

docs: add a DataFrame::cache() example in staged and pipelined modes - #2890

Open
akshaychitneni wants to merge 1 commit into
apache:mainfrom
akshaychitneni:docs/pipelined-cache-example
Open

akshaychitneni wants to merge 1 commit into
apache:mainfrom
akshaychitneni:docs/pipelined-cache-example

Conversation

@akshaychitneni

Copy link
Copy Markdown
Contributor

Part of #1829.

Rationale for this change

There is no example of DataFrame::cache(), and none of pipelined mode. The
release presents one engine for ETL and interactive queries, and caching is
where the two meet: prepare data once, then query it interactively.

What changes are included in this PR?

  • examples/examples/standalone-dataframe-cache.rs, in one session:
    • turn caching on and cache a prepared DataFrame;
    • in staged mode, the first query builds the cache and later ones read it;
    • switch to pipelined mode with one SET, and read the same cache with three
      parallel queries that stream their rows as they arrive;
    • check that both modes return the same result.
  • examples/README.md and examples/Cargo.toml: register the example.
  • Tuning guide, "Caching DataFrames": include the example and note that caching
    works in both modes.
  • Pipelined guide: a short section on caching, linking to the example.

Are there any user-facing changes?

Documentation only: a new example and user-guide sections. No code or API
changes.

The example caches a prepared DataFrame in one session, builds the cache with
the first query and reads it in staged mode, then switches the session to
pipelined mode and reads the same cache with three parallel queries that
stream their rows as they arrive.

The tuning guide's "Caching DataFrames" section includes it, and the pipelined
guide notes how caching works there.
@akshaychitneni

Copy link
Copy Markdown
Contributor Author

@andygrove @milenkovicm @comphead Added a small example to showcase df.cache with staged and pipeline mode. please take a look. Let me know if there's another use case or feature you'd like it to highlight. thanks

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant