Describe the proposal
EntityStore is the only way core, catalogs and the server reach Gravitino metadata, and gravitino.entity.store accepts any implementation class. Recent concurrency fixes keep adding one-off capability interfaces (SupportsIdentityFencedDelete in #13198, SupportsConditionalCatalogDelete in #13499) or bypass the store with direct mapper calls. The root cause is the interface: parts of its contract are unwritten or wrong, and some capabilities callers need cannot be expressed. This epic fixes the interface first; #13172 and #13176 then become ordinary users of it.
Principles:
- The interface states semantics in domain terms: identity, write intent, read freshness, and the outcome of a conflict. How they are achieved (row versions, CAS, row locks, transactions) stays inside the implementation, as
OccWriteSupport does today.
- Additions are
default methods, so existing implementations keep working and reject what they cannot honor.
- Checks the backend can make on its own (relation endpoint fences, import id checks, statistic CAS) stay inside the backend and are out of scope.
Audit 1: the written contract differs from the behavior callers rely on
| # |
Problem |
Evidence |
| C1 |
No concurrency contract. update throws OptimisticLockException on a conflict, but the Javadoc lists only NoSuchEntityException and EntityAlreadyExistsException. Which methods are atomic, and what "should handle concurrent store" means for put, is unstated. |
OperationDispatcher, TableOperationDispatcher, JobManager, TagManager and Lance catch it; a Lance test re-implements the behavior in a CasEntityStore test double. |
| C2 |
put(e) is documented as overwrite but calls put(e, false), a strict insert. |
EntityStore.put(E) |
| C3 |
delete is documented to return false for a missing entity, and RelationalEntityStore maps NoSuchEntityException to false, yet callers still catch NoSuchEntityException around it. |
CatalogManager, ManagedFunctionOperations and others |
| C4 |
batchGet is documented to throw NoSuchEntityException for a missing entity, but skips it; USER, ROLE and VIEW are fetched one by one. Callers cannot tell a missing entity from a failed read. |
JDBCBackend.batchGet; the JCasbin authorizer falls back to per-entity get when the batch comes back empty. |
| C5 |
cascade semantics per entity type are undocumented: which children make a non-cascade delete fail, and which relations are removed. |
Only visible in each meta service. |
| C6 |
update may rename, but it is not stated which fields may change (id, parent) or who invalidates the old and new names' cache entries. |
#13303 (stale cache entry after a rename by import); #13197 adds new-name invalidation. |
| C7 |
executeInTransaction is declared, but the only implementation throws UnsupportedOperationException and nothing calls it. |
RelationalEntityStore.executeInTransaction |
| C8 |
Reads may be served from the per-node cache, which is not stated, and callers cannot request the current stored state. |
#13198 adds getEntityId to bypass the cache. |
Audit 2: capabilities callers need but cannot express
| # |
Missing capability |
Current workaround |
Related |
| G1 |
Identity addressing. Every operation resolves the name when it runs, so a delayed write can hit a newer entity under the same name. |
#13198 adds deleteIfIdMatches; MetadataIdConverter loads a whole entity to get its id; owner-by-object-id and id→name lookups go around the store. |
E1/F5, E2/F6 in the concurrency review |
| G2 |
Write intent. One overwrite flag stands for create, import and reconcile. |
All 8 put(..., true) call sites are external create/import paths; a natural-key conflict keeps the old id on MySQL and fails on PostgreSQL. |
M7, #13303 |
| G3 |
Lifecycle preconditions. A delete has only a cascade flag; conditions such as "non-force, only these children may go" or "parent must be in use/enabled" cannot be decided with the write. |
#13499 adds deleteCatalogWithAllowedSchemas; managers check first and write later. |
M4, M5, #13176 |
| G4 |
Version and change awareness. Callers cannot get an entity's version or last update, so derived caches cannot tell when they are stale. |
The JCasbin authorizer probes user_meta, group_meta and role_meta.updated_at and prefetches subjects with its own SQL. |
Authorization caches |
| G5 |
Relation change notifications. The entity change log covers entities, not relations such as owners and grants. |
JcasbinChangeListener polls owner_meta on its own. |
Multi-node authorization |
| G6 |
Pagination and counting. |
UserGroupManager calls UserMetaService / GroupMetaService directly. |
— |
| G7 |
Multi-entity atomicity. |
Not possible (see C7). |
M5 in-use propagation; replacing a stale registration in #13198 |
Task list
P0: fix the contract (documentation and behavior of existing methods, no new methods)
P1: identity and write intent
P2: preconditions, change awareness and read freshness
P3: relation changes, pagination and multi-entity atomicity
Downstream consumers
Visitar URL original
Describe the proposal
EntityStoreis the only way core, catalogs and the server reach Gravitino metadata, andgravitino.entity.storeaccepts any implementation class. Recent concurrency fixes keep adding one-off capability interfaces (SupportsIdentityFencedDeletein #13198,SupportsConditionalCatalogDeletein #13499) or bypass the store with direct mapper calls. The root cause is the interface: parts of its contract are unwritten or wrong, and some capabilities callers need cannot be expressed. This epic fixes the interface first; #13172 and #13176 then become ordinary users of it.Principles:
OccWriteSupportdoes today.defaultmethods, so existing implementations keep working and reject what they cannot honor.Audit 1: the written contract differs from the behavior callers rely on
updatethrowsOptimisticLockExceptionon a conflict, but the Javadoc lists onlyNoSuchEntityExceptionandEntityAlreadyExistsException. Which methods are atomic, and what "should handle concurrent store" means forput, is unstated.OperationDispatcher,TableOperationDispatcher,JobManager,TagManagerand Lance catch it; a Lance test re-implements the behavior in aCasEntityStoretest double.put(e)is documented as overwrite but callsput(e, false), a strict insert.EntityStore.put(E)deleteis documented to returnfalsefor a missing entity, andRelationalEntityStoremapsNoSuchEntityExceptiontofalse, yet callers still catchNoSuchEntityExceptionaround it.CatalogManager,ManagedFunctionOperationsand othersbatchGetis documented to throwNoSuchEntityExceptionfor a missing entity, but skips it; USER, ROLE and VIEW are fetched one by one. Callers cannot tell a missing entity from a failed read.JDBCBackend.batchGet; the JCasbin authorizer falls back to per-entitygetwhen the batch comes back empty.cascadesemantics per entity type are undocumented: which children make a non-cascade delete fail, and which relations are removed.updatemay rename, but it is not stated which fields may change (id, parent) or who invalidates the old and new names' cache entries.executeInTransactionis declared, but the only implementation throwsUnsupportedOperationExceptionand nothing calls it.RelationalEntityStore.executeInTransactiongetEntityIdto bypass the cache.Audit 2: capabilities callers need but cannot express
deleteIfIdMatches;MetadataIdConverterloads a whole entity to get its id; owner-by-object-id and id→name lookups go around the store.overwriteflag stands for create, import and reconcile.put(..., true)call sites are external create/import paths; a natural-key conflict keeps the old id on MySQL and fails on PostgreSQL.cascadeflag; conditions such as "non-force, only these children may go" or "parent must be in use/enabled" cannot be decided with the write.deleteCatalogWithAllowedSchemas; managers check first and write later.user_meta,group_metaandrole_meta.updated_atand prefetches subjects with its own SQL.JcasbinChangeListenerpollsowner_metaon its own.UserGroupManagercallsUserMetaService/GroupMetaServicedirectly.Task list
P0: fix the contract (documentation and behavior of existing methods, no new methods)
EntityStore(C1)put,delete,batchGet,cascadeandupdatewith their documentation (C2–C6)executeInTransaction(C7)P1: identity and write intent
overwriteflag with explicit write intents (G2)P2: preconditions, change awareness and read freshness
P3: relation changes, pagination and multi-entity atomicity
Downstream consumers
SupportsIdentityFencedDeleteSupportsConditionalCatalogDelete