add tumble feature - #794
Conversation
Signed-off-by: ericyuanhui <285521263@qq.com>
4f01e43 to
1b6214e
Compare
|
This is a proposal to implement a window function similar to time_bucket. From my preliminary understanding, only TigerGraph supports this feature at present. Discussion for development is welcome. Thank you. |
|
Will the duckdb variant work for you? |
Either the streaming engine style or the DuckDB style works for me. What matters is that Ladybug supports this capability. So I need to implement this feature following either the DuckDB syntax or the streaming style, right? |
|
Yes, the feature looks valuable. Let's use the duckdb style for consistency with the rest of UDFs. |
Are there any existing syntax implementations aligned with DuckDB or streaming engines in the current project? I'd like to refer to them. Could you provide an example? |
|
is an example where we implemented functionality to be compatible with duckdb. It was more semantics than syntax. |
Goal
Add a Cypher-native
TUMBLEreading clause that assigns every input row to afixed, non-overlapping event-time window. The clause exposes the window start
and end timestamps so that existing
WITH,GROUP BY-equivalent aggregation,and
CREATE ... TABLE ASfacilities can consume them.The compatibility target is the row-assignment portion of Feldera SQL
TUMBLE(TABLE input, DESCRIPTOR(time_col), interval), not its continuousincremental execution model.
Proposed Syntax
Example:
TUMBLEconsumes the rows produced by the preceding reading clauses and keepsall bindings in scope.
YIELDintroduces exactly two new timestamp bindings.The yielded names must be distinct and must not shadow an in-scope name.
The grammar makes whitespace around the comma optional, so both forms are
equivalent:
Required Semantics
of Ladybug's timestamp types:
TIMESTAMP,TIMESTAMP_SEC,TIMESTAMP_MS,TIMESTAMP_NS, orTIMESTAMP_TZ. The yielded columns have the same logicaltype as the input expression.
TIMESTAMP_TZuses the represented instant and Unix-epoch alignment;interval('1 day')means exactly 24 hours, not a session-local civil day.DATEis not accepted as a time expression in Phase 1 and must be castexplicitly.
INTERVALis accepted only asEVERY.INTERVALof a fixed duration. Phase 1 supports microseconds through days;month and year components are rejected because they are calendar-dependent.
boundary,
TIMESTAMP_SECrequires a whole-second interval andTIMESTAMP_MSrequires a whole-millisecond interval.TIMESTAMP,TIMESTAMP_NS, andTIMESTAMP_TZaccept any fixed microsecond interval.1970-01-01 00:00:00 UTC.tand window sizew, the window start is thegreatest
origin + k*wnot later thant; the end isstart + w.[window_start, window_end). A timestampexactly at a boundary belongs to the later window.
NULLtime expression producesNULLfor both yielded columns. It doesnot remove the row.
rows.
The implementation normalizes timestamp input to the existing canonical
microsecond representation for bucket arithmetic, then converts both results
back to the input logical timestamp type. Existing timestamp conversion rules,
including the existing precision behavior of
TIMESTAMP_NS, remain unchanged.For this operator's floor-bucket calculation, a pre-epoch
TIMESTAMP_NSvaluewith a sub-microsecond remainder is normalized toward negative infinity.
Explicit Non-goals
processing-time trigger.
TABLE(TUMBLE(...))SQL syntax.OVER,PARTITION BY,ORDER BY,ROWS, andRANGE.CREATE NODE TABLE AS/CREATE REL TABLE ASresults,triggers, subscriptions, or change-data capture.
Expected Usage with Existing Ladybug Features
The daily aggregate can be snapshotted with existing CTAS support:
This is a one-time snapshot. Future writes to
EventNetworkOnlinedo notautomatically refresh
DailyFeature; an application or scheduled job mustrebuild or explicitly update affected daily rows.
Acceptance Criteria
input row.
timestamps at midnight.
epoch, null timestamps, invalid types, zero/negative intervals, calendar
intervals, alias conflicts, and chained
MATCH/WITHscopes.EXPLAINidentifies the logical and physicalTUMBLEoperator.tumbleas an unescaped identifier continue toparse.
TUMBLEandEVERYremain contextual/non-reserved keywords outsidethe new clause syntax.