Skip to content

feat(firestore): add DML stages, literals source, and atomic execution option to Android SDK pipelines - #8524

Merged
dlarocque merged 9 commits into
mainfrom
feat-android-pipeline-dml
Oct 9, 2026
Merged

dlarocque merged 9 commits into
mainfrom
feat-android-pipeline-dml

Conversation

@wu-hui

@wu-hui wu-hui commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds full DML stages (Delete, Update, Insert, Upsert), Literals source, and atomic execution option to the Android SDK (firebase-android-sdk) Firestore Pipelines subsystem to achieve parity with Web and Node SDKs.

Key Changes

  • stage.kt: Added DeleteStage, UpdateStage, InsertStage, UpsertStage, and LiteralsSource.
  • options.kt: Added hasAtomic() helper method to InternalOptions.
  • Pipeline.kt:
    • Added withAtomic(atomic: Boolean) method to ExecuteOptions.
    • Updated toExecutePipelineRequest to configure newTransaction (readWrite) and autoCommitTransaction = true when atomic is enabled.
    • Added literals(vararg data: Map<String, Any?>) source builder method to PipelineSource.
    • Added delete(), update(...), insert(...), and upsert(...) stage builder methods to Pipeline.
  • Unit & Integration Tests:
    • Added unit test suite in DmlTests.kt verifying proto generation and options encoding for all stages.
    • Added integration test methods in PipelineTest.java.

Buganizer Ticket

Fixes http://b/545136695 (under umbrella http://b/500350942)

@gemini-code-assist

Copy link
Copy Markdown
Contributor
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

@github-actions

Copy link
Copy Markdown
Contributor

📝 PRs merging into main branch

Our main branch should always be in a releasable state. If you are working on a larger change, or if you don't want this change to see the light of the day just yet, consider using a feature branch first, and only merge into the main branch when the code complete and ready to be released.

@google-oss-bot

Copy link
Copy Markdown
Collaborator

The public api surface has changed for the subproject firebase-firestore:
error: Added method com.google.firebase.firestore.Pipeline.delete() [AddedMethod]
error: Added method com.google.firebase.firestore.Pipeline.insert(String,com.google.firebase.firestore.pipeline.Expression) [AddedMethod]
error: Added method com.google.firebase.firestore.Pipeline.update(com.google.firebase.firestore.pipeline.Selectable...) [AddedMethod]
error: Added method com.google.firebase.firestore.Pipeline.upsert(com.google.firebase.firestore.pipeline.Selectable[],String,com.google.firebase.firestore.pipeline.Expression) [AddedMethod]
error: Added method com.google.firebase.firestore.Pipeline.ExecuteOptions.withAtomic(boolean) [AddedMethod]
error: Added method com.google.firebase.firestore.PipelineSource.literals(java.util.List>) [AddedMethod]
error: Added method com.google.firebase.firestore.PipelineSource.literals(java.util.Map<java.lang.String,?>...) [AddedMethod]
error: Added class com.google.firebase.firestore.pipeline.LiteralsSource [AddedClass]

Please update the api.txt files for the subprojects being affected by this change by running ./gradlew ${subproject}:generateApiTxtFile. Also perform a major/minor bump accordingly.

@google-oss-bot

Copy link
Copy Markdown
Collaborator

The public api surface has changed for the subproject firebase-firestore:
error: Added class com.google.firebase.firestore.pipeline.LiteralsSource [AddedClass]

Please update the api.txt files for the subprojects being affected by this change by running ./gradlew ${subproject}:generateApiTxtFile. Also perform a major/minor bump accordingly.

1 similar comment
@google-oss-bot

Copy link
Copy Markdown
Collaborator

The public api surface has changed for the subproject firebase-firestore:
error: Added class com.google.firebase.firestore.pipeline.LiteralsSource [AddedClass]

Please update the api.txt files for the subprojects being affected by this change by running ./gradlew ${subproject}:generateApiTxtFile. Also perform a major/minor bump accordingly.

@google-oss-bot

Copy link
Copy Markdown
Collaborator

The public api surface has changed for the subproject firebase-firestore:
error: Added method com.google.firebase.firestore.Pipeline.insert(String) [AddedMethod]
error: Added method com.google.firebase.firestore.Pipeline.upsert(com.google.firebase.firestore.pipeline.Selectable...) [AddedMethod]
error: Added method com.google.firebase.firestore.PipelineSource.documents(java.util.List<com.google.firebase.firestore.DocumentReference>) [AddedMethod]
error: Added method com.google.firebase.firestore.PipelineSource.documentsByPath(java.util.List<java.lang.String>) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Expression.as(String) [AddedMethod]
error: Added class com.google.firebase.firestore.pipeline.LiteralsSource [AddedClass]

Please update the api.txt files for the subprojects being affected by this change by running ./gradlew ${subproject}:generateApiTxtFile. Also perform a major/minor bump accordingly.

1 similar comment
@google-oss-bot

Copy link
Copy Markdown
Collaborator

The public api surface has changed for the subproject firebase-firestore:
error: Added method com.google.firebase.firestore.Pipeline.insert(String) [AddedMethod]
error: Added method com.google.firebase.firestore.Pipeline.upsert(com.google.firebase.firestore.pipeline.Selectable...) [AddedMethod]
error: Added method com.google.firebase.firestore.PipelineSource.documents(java.util.List<com.google.firebase.firestore.DocumentReference>) [AddedMethod]
error: Added method com.google.firebase.firestore.PipelineSource.documentsByPath(java.util.List<java.lang.String>) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Expression.as(String) [AddedMethod]
error: Added class com.google.firebase.firestore.pipeline.LiteralsSource [AddedClass]

Please update the api.txt files for the subprojects being affected by this change by running ./gradlew ${subproject}:generateApiTxtFile. Also perform a major/minor bump accordingly.

@google-oss-bot

Copy link
Copy Markdown
Collaborator

The public api surface has changed for the subproject firebase-firestore:
error: Added method com.google.firebase.firestore.Pipeline.insert(String) [AddedMethod]
error: Added method com.google.firebase.firestore.PipelineSource.documents(java.util.List<com.google.firebase.firestore.DocumentReference>) [AddedMethod]
error: Added method com.google.firebase.firestore.PipelineSource.documentsByPath(java.util.List<java.lang.String>) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Expression.as(String) [AddedMethod]
error: Added class com.google.firebase.firestore.pipeline.LiteralsSource [AddedClass]

Please update the api.txt files for the subprojects being affected by this change by running ./gradlew ${subproject}:generateApiTxtFile. Also perform a major/minor bump accordingly.

@wu-hui
wu-hui force-pushed the feat-android-pipeline-dml branch from 46c5156 to 7661b92 Compare October 1, 2026 21:31
@google-oss-bot

Copy link
Copy Markdown
Collaborator

The public api surface has changed for the subproject firebase-firestore:
error: Added method com.google.firebase.firestore.Pipeline.insert(String) [AddedMethod]
error: Added method com.google.firebase.firestore.PipelineSource.documents(java.util.List<com.google.firebase.firestore.DocumentReference>) [AddedMethod]
error: Added method com.google.firebase.firestore.PipelineSource.documentsByPath(java.util.List<java.lang.String>) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Expression.as(String) [AddedMethod]
error: Added class com.google.firebase.firestore.pipeline.LiteralsSource [AddedClass]

Please update the api.txt files for the subprojects being affected by this change by running ./gradlew ${subproject}:generateApiTxtFile. Also perform a major/minor bump accordingly.

1 similar comment
@google-oss-bot

Copy link
Copy Markdown
Collaborator

The public api surface has changed for the subproject firebase-firestore:
error: Added method com.google.firebase.firestore.Pipeline.insert(String) [AddedMethod]
error: Added method com.google.firebase.firestore.PipelineSource.documents(java.util.List<com.google.firebase.firestore.DocumentReference>) [AddedMethod]
error: Added method com.google.firebase.firestore.PipelineSource.documentsByPath(java.util.List<java.lang.String>) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Expression.as(String) [AddedMethod]
error: Added class com.google.firebase.firestore.pipeline.LiteralsSource [AddedClass]

Please update the api.txt files for the subprojects being affected by this change by running ./gradlew ${subproject}:generateApiTxtFile. Also perform a major/minor bump accordingly.

@wu-hui
wu-hui force-pushed the feat-android-pipeline-dml branch from 7661b92 to 3e65b9b Compare October 2, 2026 18:12
wu-hui added 7 commits October 2, 2026 18:39
…erals source, and atomic execution option to Android SDK pipelines

- Implement DeleteStage, UpdateStage, InsertStage, UpsertStage, and LiteralsSource in stage.kt
- Add literals() source method to PipelineSource and delete(), update(), insert(), upsert() to Pipeline in Pipeline.kt
- Add withAtomic() execution option to ExecuteOptions and wire proto request (newTransaction + autoCommitTransaction)
- Add DmlTests.kt unit test suite verifying proto generation for all stages
- Add integration test methods in PipelineTest.java

BUG=b/545136695
TAG=agy
…quest and add serialization tests

- Add bool auto_commit_transaction = 9 to firestore.proto
- Serialize autoCommitTransaction = true when options.hasAtomic() is enabled
- Add unit tests verifying autoCommitTransaction is set for atomic and omitted for non-atomic
- Refine expression aliases and UserDataReader literal map encoding in Pipeline stages

BUG=b/545234002
TAG=agy
…llability, and exclude atomic from options

- In Pipeline.kt, allow nullable Timestamp? in PipelineResultObserver.onComplete and default to Timestamp.now() in ObserverSnapshotTask to prevent NPE and timeout on DML streams.
- Filter out 'atomic' from StructuredPipeline.options to avoid INVALID_ARGUMENT: Unsupported option: atomic.
- In stage.kt, always encode 1 MapValue argument in UpsertStage matching UpdateStage and backend expectations.
- Remove erroneous reserved checks in RemoveFieldsStage that prevented removing document ID (__name__).
- Remove unrequested update(List<Selectable>) overload from Pipeline.kt.
- Regenerate api.txt with generateApiTxtFile and add unit test assertions.
@wu-hui
wu-hui force-pushed the feat-android-pipeline-dml branch from 3e65b9b to 6767828 Compare October 2, 2026 18:44
@wu-hui
wu-hui requested a review from dlarocque October 6, 2026 15:08
Comment thread firebase-firestore/api.txt
Comment thread firebase-firestore/api.txt Outdated
Comment thread firebase-firestore/api.txt Outdated
@dlarocque
dlarocque added this pull request to stack #8688 October 8, 2026 19:07
@dlarocque
dlarocque removed this pull request from stack #8688 October 9, 2026 17:48
## Changes

- Expressions nested in map/list values in `literals()` are now
evaluated (the backend used to reject them as `FUNCTION_VALUE`).
- FieldValue sentinels in `literals()` fail with an error naming
`literals()` and the field path.
- Empty field names in maps inside literal lists are left to the
backend.
- `insert`/`upsert` use explicit overloads taking a String path or
`CollectionReference`; the insert collection is optional.
- Removed the List overloads `upsert(List)`, `literals(List)`,
`documents(List)` and `documentsByPath(List)`.
- Removed `Expression.as()`, an undocumented duplicate of `alias()`.
- KDoc for the DML stages, `literals`, `withAtomic` and `documents`

## API Change Decisions

No released stage method takes a List of inputs (List parameters only
appear for values, like `array()`), so the List overloads are gone and
`update(List)` wasn't added. Callers can spread a list.

The released Pipeline API has no default arguments or nullable stage
parameters, and takes collections by path or reference
(`PipelineSource.collection`). The new overloads follow that:
- `insert()`, `insert(String)`, `insert(CollectionReference)`,
`insert(Expression)`, `insert(String, Expression)`,
`insert(CollectionReference, Expression)`
- `upsert(vararg Selectable)`, `upsert(String)`,
`upsert(CollectionReference)`, `upsert(String, Expression)`,
`upsert(CollectionReference, Expression)`

A target upsert no longer takes extra fields, because an array parameter
breaks the vararg convention and a trailing vararg would collide with
`(String, Expression)`; use `addFields(...).upsert(collection, id)`.
There's no `upsert(Expression)`: `Field` is both an `Expression` and a
`Selectable`, so `upsert(field("x"))` would pick the ID overload. A
reference from another Firestore instance gets the
`PipelineSource.collection` error.
@dlarocque
dlarocque self-requested a review October 9, 2026 17:49
@dlarocque
dlarocque merged commit 3237cbd into main Oct 9, 2026
35 checks passed
@dlarocque
dlarocque deleted the feat-android-pipeline-dml branch October 9, 2026 18:43
@github-actions github-actions Bot mentioned this pull request Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants