Skip to content

Endorsement Store Interface Implementation - #435

Open
shefali-kamal wants to merge 8 commits into
veraison:mainfrom
MonakaResearch:main
Open

shefali-kamal wants to merge 8 commits into
veraison:mainfrom
MonakaResearch:main

Conversation

@shefali-kamal

Copy link
Copy Markdown

This PR implements following:

  • contains the implementation of endorsement store plugin interface
  • converted coserv proxy plugins to endorsement store plugins
  • make changes in vts to use the endorsement store plugins
  • vts now uses a composite store made up of corimstore and store plugins as fallback

Address issue #431

@setrofim setrofim left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the notion of a "media type" makes sense in the context of the store backend. We don't really want to be selecting the backend based on it. I also don't like the somewhat arbitrary distinction between "primary" and "fallback" backends.

Suggestion: store backends are identified solely by name; for the sake of reusing existing plugin loader/manager the media type APIs implemented via a shim to to just return the name, and this is hidden as much as possible. Store configuration contains a list of backend names, which specifies which frontend plugins will actually be actively used by the store frontend. When servicing requests, the frontend tries the backends in the order specified until.


func NewStore() *DefaultStore {
logger := log.Named(PluginName)
logger.Debug("initializing default store")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This log line belongs inside DefaultStore.Init() below, not here.


func (s *DefaultStore) Init(params *plugin.Parameters) error {
if params == nil {
panic("parameters are required for corimstore")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return an error rather than panic here.


store, err := corimstore.Open(context.Background(), cfg.StoreConfig())
if err != nil {
panic(err)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return the error rather than panic.

// initialize it here.
if strings.Contains(cfg.DSN, ":memory:") {
if err := store.Init(); err != nil {
panic(err)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return the error rather than panic.

return SchemeName
}

func (s *DefaultStore) GetSupportedMediaTypes() map[string][]string {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the concept of a "supported media type" really makes sense for a store backend. I think we may want a different plugin interface for this.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since store plugins can also serve CoSERV queries, the supported media types can be the CoSERV media types they support.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not really clear from the API though, and even with the old CoSERV implementations this was a bit of a hack. Now that we're officially adding a new type of plugins, unrelated to attestation schemes, I think the general plugin API needs to be rethought, rather than piggybacking on the exiting IPluggable definition that was created with attestation schemes in mind.

return res, err
}

func (s *DefaultStore) ExecuteCoservQuery(mediaType, query string) (*coserv.Coserv, error) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this take a mediaType? It does not appear to be used, and also clashes with remaining store API that takea label instead. The media type should be resolved to a label by the time the request reaches the store.

}

func (s CoservProxyHandler) GetEndorsements(tenantID string, query string) ([]byte, error) {
func (s CoservProxyHandler) ExecuteCoservQuery(mediaType, query string) (*coserv.Coserv, error) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto here. the media type should be resolved by this point.


if err := o.CoservProxyPluginManager.Close(); err != nil {
o.logger.Errorf("coserv plugin manager shutdown failed: %v", err)
// FIXME: close stores while closing store manager

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need Line 174..?

Should not the Sore manager be responsible for closing the Stores!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's correct. But with the current implementation of the store manager, it is not possible. This is because the pluggable interface lacks a Close method, which can be called at the end (similar to the Init method that gets called at the start). The StoreManager.Close method calls the plugin.Client.Kill method on the store plugin client, which terminates the plugin process, without closing the file descriptors and other resources that are in use by the plugin process. The note here was for myself and this should be fixed before the pull request can be merged.


coservProxyDerived := c.assembleCoservMediaTypes(
c.CoservProxyPluginManager.GetRegisteredMediaTypes(),
c.StoreManager.GetRegisteredMediaTypes(),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with Sergei, the Store should not have a Media Type.

The functionality must be changed!

@yogeshbdeshpande yogeshbdeshpande left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have started reviewing it, will complete by the end of the day today!

@THS-on
THS-on requested a review from thomas-fossati August 11, 2026 15:37

@thomas-fossati thomas-fossati left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this!

I have inlined a few comments.

Comment on lines +526 to +527
if err != nil {
o.logger.Warnw("could not find in store", "valID", valID, "error", err)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When joining the dots between L526-527 and L532-533, it looks like any store errors will be seen as ENOTFOUND from the pov of the caller. If that's the case, the caller (L446) will log and move on. So, imagine the case where the DB is down, valueTriples stays empty, getValueTriples returns ENOTFONUD, the caller logs a warning and proceeds with nil endorsements into AppraiseEvidence. I don't think it's what we want, right?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All errors other than ENOTFOUND from all the stores are logged as warning (/vts/trustedservices/store.go:122). The reason was to avoid the operation from failing some of the stores are down. But now I realized that this is a bad idea. Will update to return on unexpected store errors.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking: do you still plan to address this?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error is propagated till vts/endorsementstore/store.go, forgot to update the final layer. Will update to throw an error on any store errors other than ENOTFOUND.

Comment on lines +175 to +180
if err := CloseCorimStore(o.StoreManager); err != nil {
o.logger.Errorf("failed to close corim store: %v", err)
}

if err := o.Store.Close(); err != nil {
o.logger.Errorf("store closure failed: %v", err)
if err := o.StoreManager.Close(); err != nil {
o.logger.Errorf("store plugin manager shutdown failed: %v", err)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential ENOCOFFEE :-) Would this effectively result in a double close?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, StoreManager.Close does not properly close the corim-store (see #435 (comment)). This is a hacky fix and should be fixed before merging the pr.

Comment on lines +666 to +667
if err != nil {
o.logger.Infof("could not find coserv result in store")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reasoning behind masking any error as ENOTFOUND/404?
From a ReST perspective, we want to be able to tell our clinets that there is something wrong with the service (5xx) rather than with their query (4xx).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API layer seems to treats all errors from VTS as Internal Server Error.

coserv:

// Forward query to VTS
res, err := o.EndorsementDistibutor.GetEndorsements(tenantID, coservQuery, mediaType)
if err != nil {
status := http.StatusBadRequest
reportProblem(c, status, err.Error())
return
}

verification:

// Forward the evidence to the verifier. We expect the verifier to be
// able to cope with bad evidence, so the error here should only be
// reported if something in the verifier or the connection goes wrong.
// Any problems with the evidence are expected to be reported via the
// attestation result.
attestationResult, err := o.Verifier.ProcessEvidence(tenantID, session.Nonce,
evidence, mediaType)
if err != nil {
o.logger.Error(err)
session.SetStatus(StatusFailed)
mustStoreSession(o.SessionManager, session, id, tenantID)
ReportProblem(c,
http.StatusInternalServerError,
"error encountered while processing evidence",
)
return
}

But more context can be added to the error. I will try to do that.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, thanks for checking.
I think that's a problem with the frontends that we'll need to fix.
However, first they need to get the right signal from VTS :-)


func (s *DefaultStore) ExecuteCoservQuery(mediaType, query string) (*coserv.Coserv, error) {
s.logger.Infof("got coserv query: %v", query)
fallbackAuthority, err := comid.NewCryptoKeyTaggedBytes([]byte("dummyauth"))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

curious: why we need to use this "dummyauth"?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a copy-paste of an older version of the code (before #420). I should update this with the latest changes. Thanks for pointing it out.

Comment thread proto/endorsement_store.proto Outdated
option go_package = "github.com/veraison/services/proto";

message GetEndorsementsArgs {
bytes environment = 1 [json_name = "environemnt-map"];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
bytes environment = 1 [json_name = "environemnt-map"];
bytes environment = 1 [json_name = "environment-map"];

Comment thread vts/cmd/vts-service/main.go Outdated
if subs["coserv"].IsSet("signer") {
coservContext, err = coserv.NewCoservContextFromViper(subs["coserv"])
if err != nil {
log.Fatal("CoSERV config initialization: %v", err)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
log.Fatal("CoSERV config initialization: %v", err)
log.Fatalf("CoSERV config initialization: %v", err)

@DhanusML

Copy link
Copy Markdown

Major changes:

  • VTS config now includes an active-stores section containing a list of store plugins that will be used (see /vts/endorsementstore for details)
  • Added Fini method in IPluggable

@thomas-fossati thomas-fossati left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I have re-reviewed and added a few comments.

Comment thread vts/cmd/vts-service/main.go Outdated
maps.Copy(pluginConfig, endorsementStoreConfig)

log.Debug("loading scheme and endorsement store plugins")
psubs, err := config.GetSubs(subs["plugin"], "*go-plugin", "*builtin", "go-plugin-stores")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be made conditional instead?

I suspect this breaks when the services are built with make SCHEME_LOADER=builtin

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config for endorsement store plugins are copied to pluginConfig in line 93.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The go-plugin should be renamed to go-plugin-schemes to be consistent with go-plugin-stores.

Comment on lines +89 to +92
if err := s.Store.Close(); err != nil {
s.logger.Errorf("Failed to close corim-store: %v", err)
return err
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could panic if Init failed before assigning s.Store:

Suggested change
if err := s.Store.Close(); err != nil {
s.logger.Errorf("Failed to close corim-store: %v", err)
return err
}
if s.Store != nil {
if err := s.Store.Close(); err != nil {
s.logger.Errorf("Failed to close corim-store: %v", err)
return err
}
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, Fini() should only be called if Init() succeeds, so it's reasonable to assume that Store has been initialized inside Fini().

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, then let's panic() on a nil s.Store if that's an invariant

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually in the current implementation, Fini is also called if Init fails (Close is called to terminate the plugin server if Init fails (goplugin_loader.go:177)).

I will make changes there and also panic on a nil s.Store here

Comment on lines +526 to +527
if err != nil {
o.logger.Warnw("could not find in store", "valID", valID, "error", err)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking: do you still plan to address this?

Comment on lines +666 to +667
if err != nil {
o.logger.Infof("could not find coserv result in store")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, thanks for checking.
I think that's a problem with the frontends that we'll need to fix.
However, first they need to get the right signal from VTS :-)

Comment thread vts/endorsementstore/store.go Outdated
store := VtsEndorsementStore{
logger: logger,
}
for _, pl := range plugins {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to explicitly check that the plugins list is not empty here.

Comment thread handler/endorsementstore_rpc.go Outdated
}
var unused any
if err := c.client.Call("Plugin.AddCorimBytes", &args, &unused); err != nil {
return err

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return err
return ParseError(err)

@shefali-kamal
shefali-kamal force-pushed the main branch 2 times, most recently from f32357f to c27bcbb Compare September 3, 2026 07:33
Comment thread deployments/native/README.md Outdated
├── bin
├── certs
├── config
├── endorsementstore

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the name should make it clearer that the directory contains plugins (rather than the actual store database). Also rather than increasing the number of top-level directories, it would good to move this under plugin/ so that it looks something like

├── plugins
    ├── schemes
    │    ├── arm-cca.plugin
    │    └── ...
    └── stores
         ├── corim-store.plugin
         └── ...

@@ -73,14 +84,10 @@ func (s CoservProxyHandler) addTrustAnchorForInstance(i *coserv.StatefulInstance
return err
}

// TODO(paulhowardarm) - This authority is a dummy value.
// TODO(paulhowardarm)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this point, probably remove the TODO as well

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only the dummy authority issue is resolved. Rest of the points are still todo


.DEFAULT_GOAL := test

GOPKG := github.com/veraison/services/endorsementstore/amd-kds-coserv

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to directory naming in the deployment, I would change the package name to make it clearer that this is a plugin maybe store-plugin instead of endorsementstore?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit, but for consistency call the package corim-store rather than corimstore.

)

// implement the IEndorsementStore interface for corimstore.Store
type DefaultStore struct {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change the name to just Store since this isn't really default in the context of this module.

Comment on lines +89 to +92
if err := s.Store.Close(); err != nil {
s.logger.Errorf("Failed to close corim-store: %v", err)
return err
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, Fini() should only be called if Init() succeeds, so it's reasonable to assume that Store has been initialized inside Fini().

return SchemeName
}

func (s *DefaultStore) GetSupportedMediaTypes() map[string][]string {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not really clear from the API though, and even with the old CoSERV implementations this was a bit of a hack. Now that we're officially adding a new type of plugins, unrelated to attestation schemes, I think the general plugin API needs to be rethought, rather than piggybacking on the exiting IPluggable definition that was created with attestation schemes in mind.

Comment thread store-plugin/nvidia-coserv/coserv_handler.go
Comment thread plugin/goplugin_loader.go Outdated
if err := pluginContext.Handle.Init(params); err != nil {
o.logger.Errorf("plugin q: %s", pluginName, err.Error())
o.logger.Errorf("plugin %q: %s", pluginName, err.Error())
pluginContext.Close()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Close calls Fini and then Client.Kill. But as per https://github.com/veraison/services/pull/435/changes#r3923652647, Fini should only be called during graceful exit. So, this should be changed to pluginContext.client.Kill().

@@ -0,0 +1,148 @@
// Copyright 2025-2026 Contributors to the Veraison project.
// SPDX-License-Identifier: Apache-2.0
package corimstore

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the package name should be corim-store

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

package name should be a valid identifier, so it cannot contain - character (https://go.dev/ref/spec#Package_clause)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The convention is to replace - with _ when mapping directories to package names; so this should be corim_store


const (
PluginName = "corim-store"
SchemeName = "ANY"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it mandatory to have the schemename in the plugin, in theory this is not needed..?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the struct Store should implement IPluggable, which has a method GetAttestationScheme. Right now, scheme name is only used to stub this method.

return err
}

// The store must be initialized before it may be used. In general, we

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// The store must be initialized before it may be used. In general, we
// The store must be initialized before it can be used. In general, we

@yogeshbdeshpande yogeshbdeshpande left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

started reviewing the changes, will complete it by tomorrow

Comment thread scheme/Makefile
@@ -1 +1 @@
# Copyright 2021-2026 Contributors to the Veraison project.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file, is not pertaining to your changes, perhaps, your changes needs re-base so that this file does not appear in this PR.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume the comment is on /scheme/tpm-enacttrust/test/cmd/gen-token/main.go. The change was done because copyright check (part of make test) was failing because the file was recently updated (in #449) but the year in the header was not updated.

@@ -0,0 +1,148 @@
// Copyright 2025-2026 Contributors to the Veraison project.
// SPDX-License-Identifier: Apache-2.0
package corimstore

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The convention is to replace - with _ when mapping directories to package names; so this should be corim_store

Comment thread vts/cmd/vts-service/main.go Outdated
maps.Copy(pluginConfig, endorsementStoreConfig)

log.Debug("loading scheme and endorsement store plugins")
psubs, err := config.GetSubs(subs["plugin"], "*go-plugin", "*builtin", "go-plugin-stores")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The go-plugin should be renamed to go-plugin-schemes to be consistent with go-plugin-stores.

Comment thread vts/cmd/vts-service/main.go Outdated
}

// for builtin loader
builtinPluginConfig := maps.Clone(pluginConfig)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would drop builtinPluginConfig and just merge endorsementStorePluginConfig into pluginConfig, and later on, pass pluginConfig to all loaders.

We don't want builtin configuration to be in any way different from plugin configuration (the loading method should be as transparent as possible).

As in builtin configuration the same loader is used for both plugin types, it also means that the two types of plugins share a namespace (we cannot have a scheme plugin and a store plugin with the same name). The loader will raise an error (though, as builtin loader is rarely exercised we might miss it), however when copying parameters below, rather an just using maps.Copy(), you need to make sure there are no clashing keys and raise an error.

An alternative, and I think this is the better solution, would be to ensure that store and scheme plugins live in different namespaces. This means using separate loaders, even in built in configuration. This would require modifying the builtin loader to take a list plugins as an argument rather than assuming github.com/veraison/builtin.plugins list. That list can than be split in two -- one for schemes and one for stores.

return SchemeName
}

func (s *Store) GetSupportedMediaTypes() map[string][]string {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So as discussed elsewhere, I don't think media types make sense the context of the store and this should be removed.

The existing plugin code needs to be rejigged to not assume all plugins have associated media types. The simplest, bare minimum, solution would be to return nil here and make sure that gets interpreted correctly by the loader.

A better solution would be to refactor the plugin code so that instead of just

type GoPluginLoader struct {
	Location string

	logger            *zap.SugaredLogger
	loadedByName      map[string]IPluginContext
	loadedByMediaType map[string]IPluginContext

	pluginMap map[string]plugin.Plugin

	pluginParams map[string]*Parameters

	registeredPluginTypes map[string]string
}

that we have now, we would have

type GoPluginLoader struct {
	Location string

	logger            *zap.SugaredLogger
	loadedByName      map[string]IPluginContext

	// This gets specified as Plugins when creating a new go-plugin client.
	pluginMap map[string]plugin.Plugin

	pluginParams map[string]*Parameters

	registeredPluginTypes map[string]string
}

type MediaTypeGoPluginLoader struct {
	GoPluginLoader

	loadedByMediaType map[string]IPluginContext
}

And all media type related methods moved into MediaTypeGoPluginLoader which would then correspond a new IMediaTypeLoader interface which contains and extends the existing ILoader. Similar modifications would need to be done to IPluggable and IManager interfaces and their implementations. So the media type functionally is layered on top of general plugin functionality. The schemes will then use the new media type based API, and the stores can use the old API that no longer requires a vestigial media type.

// AggregateStoreParams aggregates different configurations to create
// a consolidated parameters map, that can be passed to the store plugin
// manager. The configurations come from store plugin configuration
// section, vts configuration section and coserv configuration section.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than doing this, I think it would make more sense to restructure configuration so that all endorsement-store related configuration is in a single section. In particular, the old CoSERV section does not need to exist

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

coserv configuration should not be merged with the endorsement store
configuration because coserv service is the consumer of coserv configuration.
This pattern is followed in all the services except coserv section, which
contains max-expiry and signer sub-sections, which are used by vts. Maybe this
should be first separated as

# used by coserv service
coserv:
  listen-addr: ...
  protocol: ...
  cert: ...
  cert-key: ...

# used by vts
coserv-signer:
  alg: ...
  key: ...
  max-expiry: ...

(expiry in the coserv-signer section because max-expiry is also (indirectly)
the validity period of the signature).

Now, the common coserv configuration which all the stores use are max-expiry and
the public part of signing key. Since vts is the primary owner of these
parameters, I decided to let vts parse these configurations and then extract
them from the signer object (coserv.Context) and pass them to the store.

It is possible to move just the expiry parameter to a unified store config, but
the fallback authority will have to be extracted from the signer.

@setrofim setrofim Sep 17, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

coserv configuration should not be merged with the endorsement store
configuration because coserv service is the consumer of coserv configuration.

Yes, you're right, sorry, I was over-eager with that because I was just looking at vts/cmd/vts-service/config.yaml.

which contains max-expiry and signer sub-sections, which are used by vts. Maybe this should be first separated as [...]

Yes, moving that is probably the right thing to do. Actually, I think both singer and max-expiry can just move under endorsement-store; and active-store (currently under vts) should move there as well. This will remove the need for AggregateStoreParams() as everything will be in a single sub and can be loaded using the normal config loading mechanism.

EDIT: the names should probably change to make it clearer what the configs control; so the config would look something like

endorsement-store:
  coserv-signer:
    alg: ES256
    key: ./skey.jwk
  coserv-max-expiry: 1m
  active-plugins:
    - corim-store
  plugin-parameters:
    corim-store:
      dbms: sqlite3
      dsn: file::memory:?cache=shared
      trace-sql: false

}

// Create StoreCommonParams from a coservContext
func CreateStoreCommonParams(coservContext *vtscoserv.Context) (*plugin.Parameters, error) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems.. backwards. The context should be created from the config parameters and not the other way round.

Comment thread vts/endorsementstore/store.go Outdated
func CreateEndorsementStore(plugins []string, manager StoreManager, logger *zap.SugaredLogger) (handler.IEndorsementStore, error) {
if len(plugins) == 0 {
err := errors.New("no plugins in `active-stores` list")
logger.Errorf("could not create endorsementstore: %v", err)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not log here. The error is returned and should be handled (including logging it) by the calling context.

Comment thread vts/endorsementstore/store.go Outdated

func CreateEndorsementStore(plugins []string, manager StoreManager, logger *zap.SugaredLogger) (handler.IEndorsementStore, error) {
if len(plugins) == 0 {
err := errors.New("no plugins in `active-stores` list")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
err := errors.New("no plugins in `active-stores` list")
err := errors.New("no names in `active-stores` list")

Comment thread vts/endorsementstore/store.go Outdated
return s.stores, nil
}

func CreateEndorsementStore(plugins []string, manager StoreManager, logger *zap.SugaredLogger) (handler.IEndorsementStore, error) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename plugins to pluginNames

@thomas-fossati thomas-fossati left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few more comments

Comment thread vts/cmd/vts-service/config-docker.yaml Outdated
Comment on lines 6 to 7
go-plugin:
dir: ../../../scheme/bin/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should go, right?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. go-plugin should be changed to go-plugin-schemes. Thanks for pointing it out. Same shoudl be done in vts/cmd/vts-service/config.yaml.

// CoSERV interface would be disabled.
if s.CoservCfg == nil {
s.logger.Errorf("store is not configured for CoSERV")
return nil, errors.New("missing configurations for CoSERV service")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this return one of the errors handled by logOrErr() (i.e., errnotfound/errunsupported)? I think we want ExecuteCoservQuery to skip to the next active store rather than stop the loop.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is neither of those errors. Instead this is a misconfiguration because this happens when trying to execute coserv query when the store is not configured for coserv. This can be caught in Init by throwing an error if coserv config is not being passed but that would cause store initialization to fail on deployments that do not enable coserv service.

Note that CoservCfg is passed from vts to all store plugins if the config.yaml contains coserv configurations.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is neither of those errors. Instead this is a misconfiguration because this happens when trying to execute coserv query when the store is not configured for coserv.

ok, but if this is a misconfiguration, it shouldn't reach this deep.

This can be caught in Init by throwing an error if coserv config is not being passed but that would cause store initialization to fail on deployments that do not enable coserv service.

tricky.

Note that CoservCfg is passed from vts to all store plugins if the config.yaml contains coserv configurations.

so, is the error even possible? If not, panic'ing is probably a better option.

Comment thread store-plugin/corim-store/corimstore_enstore.go
}
resp, err := o.endorsementStore.ExecuteCoservQuery(profile, queryIn.Query)
if err != nil {
o.logger.Infof("could not find coserv result in store")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please attach the original error to the log message or it'll get lost. (there may be other errors than ENOTFOUND that are worth surfacing.)

s.logger.Errorf("failed to fetch store list: %v", err)
return err
}
store := stores[0] // stores is guaranteed to contain at least one entry

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: how can we be sure that stores[0] is not read-only?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If stores[0] does not support write operation, in the current implementation, endorsement provisioning will fail. So the user should make sure that the first store in the list is not read-only. Since the default configurations had corim-store as the first entry, this was fine. Maybe a better approach would be to try AddCorimBytes until a store does not return ErrUnsupported.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If stores[0] does not support write operation, in the current implementation, endorsement provisioning will fail. So the user should make sure that the first store in the list is not read-only. Since the default configurations had corim-store as the first entry, this was fine.

ok. Is this assumption clearly documented?

Maybe a better approach would be to try AddCorimBytes until a store does not return ErrUnsupported.

definitely :-)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a better approach would be to try AddCorimBytes until a store does not return ErrUnsupported.

definitely :-)

Actually, I'm not sure that's necessarily a great idea, since it's not immediately obvious where writes will be directed to. I think a better solution would be to ensure that the first (highest priority) store is writable. And rather than relying to catching and handling ErrNotSupported, the store API should be extended with a method for querying its capabilities (e.g. if it's writable); that way, this can be validated during initialization.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a better approach would be to try AddCorimBytes until a store does not return ErrUnsupported.

definitely :-)

Actually, I'm not sure that's necessarily a great idea, since it's not immediately obvious where writes will be directed to. I think a better solution would be to ensure that the first (highest priority) store is writable. And rather than relying to catching and handling ErrNotSupported, the store API should be extended with a method for querying its capabilities (e.g. if it's writable); that way, this can be validated during initialization.

that's an even better approach. Though it's a bit more intrusive.

func (s *CoservProxyHandler) Init(params *plugin.Parameters) error {
var cfg vtsstore.StoreCommonParams
if err := (&cfg).FromParams(params); err != nil {
s.StoreCommonParams = nil

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add a warning-level log here

If plugin initialization fails, kill the plugin process.
Otherwise, the plugin server process runs in the background and
is not terminated even when the service exits.
Also fix typo in logging (q -> %q)

Signed-off-by: Dhanus M Lal <Dhanus.MLal@fujitsu.com>
Add method to join two parameter objects

Signed-off-by: Dhanus M Lal <Dhanus.MLal@fujitsu.com>
Add Fini method in IPlugable that can be called to free the
resources used by the plugin. The plugin loader now calls this method
before killing the plugin process.

Signed-off-by: Dhanus M Lal <Dhanus.MLal@fujitsu.com>
Definition of protobuf messages used as arguments for
endorsement store interface.

Signed-off-by: Dhanus M Lal <Dhanus.MLal@fujitsu.com>
The store plugin interface definition and implementation
of the RPC layer (for go-plugin client and server).
Endorsement store interface contains methods for
1. Fetching key and value triples using environment map
2. Fetching CoSERV result by passing a CoSERV query as input
3. Adding CoRIM bytes to the store

Signed-off-by: Dhanus M Lal <Dhanus.MLal@fujitsu.com>
CreateGoPluginManager now takes pluginClass as an additional input,
which specifies the subsection under plugin that should be used by
the manager to load plugins from.

Signed-off-by: Dhanus M Lal <Dhanus.MLal@fujitsu.com>
VTS uses a store composed of multiple store plugins. For each store
operation (except addition), the operation is iterated over the component
stores until a match is found. The store plugins to be used by vts are
passed as a list in the vts configuration in the active-stores section.
The store plugins may take initialization parameters, these are passed
to them via the plugin parameters. Some configurations are common for all
plugins. Such parameters are broadcasted to all the active store plugins.
Examples of such config parameters include the CoSERV configuration
(default expiry time of the results and a fallback authority for adding the
service's authority in the quads).

This commit converts the existing corim-store into a store plugin. The coserv
proxy plugins have also been converted into store plugins, with only the
ExecuteCoservQuery method implemented.

The following store plugins are now available
* corim-store
* amd-kds-coserv-proxy-handler
* nvidia-coserv-proxy-handler

The native and docker deployments are updtated to use all these stores
in this order in the active-stores list.

BREAKING CHANGE: Existing configurations won't work anymore because the
store section of the config (which had corim-store configuration) is now
part of the corim-store plugin parameter. For the native and docker
deployments, the directory structure of plugin binaries have also
changed. In the plugins section, go-plugins subsection containing the
plugin loading configuration has been renamed to go-plugin-schemes (now
for scheme plugins) and a new section go-plugin-stores has been added
for store plugins.

Signed-off-by: Dhanus M Lal <Dhanus.MLal@fujitsu.com>
Signed-off-by: Dhanus M Lal <Dhanus.MLal@fujitsu.com>
subs, err := config.GetSubs(v, "store", "po-store",
"*po-agent", "plugin", "*vts", "ear-signer", "*coserv", "*logging", "*scheme")
subs, err := config.GetSubs(v, "po-store",
"*po-agent", "plugin", "*vts", "ear-signer", "*coserv-signer",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the coserv-signer sub is ever used?

Comment thread vts/cmd/vts-service/main.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants