Skip to content

[#13085] improvement(server): Allow catalog users to test existing catalog connections with stored configuration - #13086

Merged
jerryshao merged 4 commits into
apache:mainfrom
jerryshao:test-existing-connection-authz
Sep 11, 2026
Merged

jerryshao merged 4 commits into
apache:mainfrom
jerryshao:test-existing-connection-authz

Conversation

@jerryshao

@jerryshao jerryshao commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

  • Authorize testExistingConnection with the load-catalog expression.
  • Add CatalogConnectionTestAuthorizationExecutor (request type TEST_CATALOG_CONNECTION). When the request carries non-empty updates, it switches to the annotation's secondary expression, which requires metalake or catalog ownership (TEST_CATALOG_CONNECTION_WITH_CHANGES_AUTHORIZATION_EXPRESSION, condition HAS_PROPOSED_CHANGES).
  • In GravitinoInterceptionService, report the expression the executor evaluated (recorded in AuthorizationRequestContext) in the authorization denial event and log, falling back to the annotation expression. Before this change, a denial by a secondary expression was reported with the primary expression. This applies to every executor-based endpoint; endpoints that evaluate only the annotation expression report the same value as before.
  • Document the rule in docs/security/access-control.md and declare the 403 response in docs/open-api/catalogs.yaml.

Why are the changes needed?

A user who can load a catalog already reaches the backing system with its stored configuration. Owner-only for that test adds no protection. Proposed changes still require ownership because the caller chooses what the server connects to.

Fix: #13085

Does this PR introduce any user-facing change?

Yes.

  • Users with USE_CATALOG can now test an existing catalog with its stored configuration.
  • Testing with proposed changes still requires metalake or catalog ownership.
  • Authorization denial audit events report the expression that was actually evaluated.

How was this patch tested?

  • TestCatalogConnectionTestAuthorizationExecutor: expression selection for no body, empty or null updates, proposed changes, and the NEVER condition, plus creation through AuthorizeExecutorFactory.
  • TestGravitinoInterceptionService: the real CatalogOperations annotations through the interceptor. A USE_CATALOG user can test the stored configuration and is denied for proposed changes (the denial event reports the owner expression); the catalog owner can test proposed changes.
  • TestCatalogAuthorizationExpression: the primary and secondary expressions on testExistingConnection.
  • CatalogAuthorizationIT#testTestExistingCatalogConnection: a user with USE_CATALOG and the owner end to end.
  • Python test_error_handler: the catalog error handler raises ForbiddenException for a forbidden response.
  • ./gradlew :server:test --tests 'org.apache.gravitino.server.web.filter.*' --tests '*TestCatalogAuthorizationExpression' --tests '*TestCatalogOperations' :docs:build -PskipITs

🤖 Generated with Claude Code

…ing catalog connections with stored configuration

Authorize testing an existing catalog with its stored configuration the
same as loading the catalog. Testing with proposed changes still requires
metalake or catalog ownership, and a denial now returns HTTP 403 instead
of 500.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 11, 2026 03:56
@jerryshao
jerryshao requested a review from roryqi September 11, 2026 03:58
@jerryshao jerryshao self-assigned this Sep 11, 2026
@jerryshao jerryshao added the branch-1.3 Automatically cherry-pick commit to branch-1.3 label Sep 11, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The OpenAPI operation still lacks an explicit 403 response matching the documented and implemented behavior.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Updates catalog connection testing authorization: stored configurations follow USE_CATALOG, while proposed changes require ownership.

Changes:

  • Added conditional authorization and HTTP 403 handling.
  • Added unit and integration coverage.
  • Updated access-control and OpenAPI documentation.
File summaries
File Description
server/src/test/java/org/apache/gravitino/server/web/rest/TestCatalogOperations.java Tests authorization behavior.
server/src/test/java/org/apache/gravitino/server/web/rest/authorization/TestCatalogAuthorizationExpression.java Tests authorization expressions.
server/src/main/java/org/apache/gravitino/server/web/rest/ExceptionHandlers.java Maps forbidden errors to HTTP 403.
server/src/main/java/org/apache/gravitino/server/web/rest/CatalogOperations.java Applies conditional authorization.
server-common/src/main/java/org/apache/gravitino/server/authorization/expression/AuthorizationExpressionConstants.java Adds the ownership authorization expression.
docs/security/access-control.md Documents access rules.
docs/open-api/catalogs.yaml Documents endpoint authorization behavior.
clients/client-java/src/test/java/org/apache/gravitino/client/integration/test/authorization/CatalogAuthorizationIT.java Adds end-to-end authorization coverage.
Review details
  • Files reviewed: 8/8 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread docs/open-api/catalogs.yaml
jerryshao and others added 3 commits September 11, 2026 14:01
…ting catalog connection tests

Replace the in-method ownership check with CatalogConnectionTestAuthorizationExecutor,
which switches to the annotation's secondary expression when the request carries
proposed changes. Revert the test-connection 403 mapping that is no longer needed,
and declare the 403 response in the OpenAPI spec.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…uthorization through the interceptor

Add interceptor tests that use the real CatalogOperations annotations to verify
USE_CATALOG can test the stored configuration, proposed changes are denied without
ownership, and the catalog owner can test proposed changes. Also cover the executor
factory and a request without updates.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…n authorization denial events

An authorization executor can evaluate a request-specific expression, such as a
secondary expression. Report the expression it recorded in the request context in
the denial event and log, and fall back to the annotation expression otherwise.
Also cover the forbidden error mapping of the Python catalog error handler.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@roryqi roryqi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@github-actions

Copy link
Copy Markdown

Code Coverage Report

Overall Project 69.84% +0.24% 🟢
Files changed 90.56% 🟢

Module Coverage
aliyun 19.74% 🔴
api 51.59% 🟢
authorization-common 85.96% 🟢
authorization-ranger 4.38% 🔴
aws 53.54% 🟢
azure 32.1% 🔴
catalog-common 28.4% 🔴
catalog-fileset 82.17% 🟢
catalog-glue 71.52% 🟢
catalog-hive 83.07% 🟢
catalog-jdbc-common 45.05% 🟢
catalog-jdbc-doris 82.69% 🟢
catalog-jdbc-mysql 81.8% 🟢
catalog-jdbc-postgresql 83.89% 🟢
catalog-jdbc-starrocks 79.16% 🟢
catalog-kafka 75.21% 🟢
catalog-lakehouse-generic 61.16% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 85.97% 🟢
catalog-lakehouse-paimon 84.29% 🟢
catalog-model 77.99% 🟢
cli 44.62% 🟢
client-java 77.91% 🟢
common 58.09% 🟢
core 84.33% +0.15% 🟢
filesystem-hadoop3 76.48% 🟢
flink 0.0% 🔴
flink-common 53.63% 🟢
flink-runtime 0.0% 🔴
gcp 32.2% 🔴
hadoop-auth 68.0% 🟢
hadoop-common 17.84% 🔴
hive-metastore-common 54.41% 🟢
iceberg-aliyun-bundle 0.0% 🔴
iceberg-common 66.16% 🟢
iceberg-rest-server 77.02% 🟢
idp-basic 86.77% 🟢
integration-test-common 0.0% 🔴
jobs 62.92% 🟢
lance-common 34.02% 🔴
lance-rest-server 69.12% 🟢
lineage 59.39% 🟢
optimizer 83.17% 🟢
optimizer-api 21.95% 🔴
server 90.1% +0.48% 🟢
server-common 81.87% -0.24% 🟢
spark 56.37% 🟢
tencent 81.78% 🟢
trino-connector 58.57% 🟢
Files
Module File Coverage
core JobMetaBaseSQLProvider.java 100.0% 🟢
JobMetaPostgreSQLProvider.java 100.0% 🟢
JobTemplateMetaPostgreSQLProvider.java 100.0% 🟢
MetalakeMetaService.java 100.0% 🟢
JobMetaSQLProviderFactory.java 95.83% 🟢
JobMetaService.java 94.59% 🟢
JobTemplateMetaSQLProviderFactory.java 87.5% 🟢
JobManager.java 86.73% 🟢
JobTemplateMetaBaseSQLProvider.java 84.62% 🟢
JobTemplateMetaService.java 81.08% 🟢
JobMetaMapper.java 0.0% 🔴
JobTemplateMetaMapper.java 0.0% 🔴
server CatalogConnectionTestAuthorizationExecutor.java 100.0% 🟢
CatalogOperations.java 100.0% 🟢
GravitinoInterceptionService.java 93.71% 🟢
AuthorizeExecutorFactory.java 60.0% 🟢
server-common ExpressionCondition.java 100.0% 🟢
AuthorizationRequest.java 0.0% 🔴
AuthorizationExpressionConstants.java 0.0% 🔴

@jerryshao
jerryshao merged commit bb9a7e5 into apache:main Sep 11, 2026
38 checks passed
jerryshao added a commit that referenced this pull request Sep 11, 2026
…og users to test existing catalog connections with stored configuration (#13086) (#13088)

**Cherry-pick Information:**
- Original PR: #13086 (still open; this backport is raised ahead of its
merge, and must be kept in sync with any review changes there)
- Original issue: #13085
- Target branch: `branch-1.3`
- Status: ✅ Conflicts resolved manually

**Resolution:**
- Applied the final #13086 change as one commit instead of replaying its
individual review commits.
- `TestGravitinoInterceptionService`: `branch-1.3` does not have the
table existence-probe tests or their helpers, so only the new catalog
connection tests, their helpers, and their imports are added.
- `TestCatalogConnectionTestAuthorizationExecutor`:
`AuthorizeExecutorFactory.create` on `branch-1.3` has no
`allowCheckExistenceExpression` parameter, so the factory test passes
nine arguments.
- Production code, docs, the integration test, and the Python test are
the same as #13086.

**Validation:**
- Server unit tests on `branch-1.3`: all
`org.apache.gravitino.server.web.filter.*` tests (including the new
interceptor and executor tests), `TestCatalogAuthorizationExpression`,
and `TestCatalogOperations`.
- Compiled the Java client test sources, including
`CatalogAuthorizationIT`. The Docker integration test and the Python
unit test were not run locally.
- `./gradlew spotlessApply`, `:server-common:javadoc`, `:docs:build`
- Conflict-marker scan and `git diff --check`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement] Allow catalog users to test an existing catalog connection with its stored configuration

3 participants