Add provide_any module to core

[nrc]

Rendered

---

This RFC proposes adding a provide_any module to the core library. The module provides a generic API for objects to provide type-based access to data. (In contrast to the any module which provides type-driven downcasting, the proposed module integrates downcasting into data access to provide a safer and more ergonomic API).

[pachi]

Add link to Rendered version

[zkat]

If it helps at all, this looks like it would be amaaaazing for miette. I believe that with this in place, miette could stop defining its own Diagnostic type and "simply" get users to provide various bits of context that it knows about. That means miette and its reporters would be able to use Eyre directly, instead of a bad fork of it :)

Living under any rather than in a new top-level module seems like a more natural home for this to me. There's overlap between type tags and type ids, and Any and Provider that I think we could give proper treatment together under the same umbrella.

that makes sense! I'm a bit hesitant because type ids and type tags are so obviously similar but have nothing in common in their interfaces or how they are used, so I thought it might be a bit confusing to have them together

Do you think trying to come up with a module-level doc for a hypothetical combined any module might be a good exercise for identifying those cases where that confusion can come from?

Why is Ref not a type tag combinator, where if T has a type tag then Ref can be used as type tag for references to T (somewhat similar to the Option combinator that does seem to exist)?

My guess is convenience. I think if Ref was implemented the same way Optional is you'd have to invoke it with foo.get_by_tag::<Ref<Value<MyType>>>(). Thinking about it this doesn't seem too bad, since we already have get_context_ref that handles this case, so we may never need to use the Ref type tag directly as is, which seems like a good justification for updating it to compose with other TypeTags.

One reason is that type tags represent Sized types, so using the compositional form for Ref would mean we lose the ability to represent references to ?Sized types. You can add the ?Sized bound to the Type associated type, but that means a lot of I::Type: Sized bounds all over the place, so it seems sub-optimal.

I found a nice usecase for something like this in my day-to-day codebase, which is a storage engine, that I thought I'd share. Individual documents can be cached for better performance. Caching is fine-grained, so for each document we can choose a different caching strategy. We've got a trait for these cache entries that looks like this:

pub trait CacheEntry: Any + Send + Sync + Display {
    /**
    The raw on-disk payload bytes.
    */
    fn bytes_from_disk(&self) -> Option<BytesFromDisk> {
        None
    }

    /**
    The payload bytes that are yielded to callers.

    This may involve decompression or other transformations over the raw on-disk payload.
    */
    fn bytes(&self) -> Option<Bytes> {
        None
    }

    /**
    The approximate size of this entry in-memory.
    */
    fn approximate_size(&self) -> usize;
}

The combination of Any as a supertrait plus some optional getters are a pretty clear smell for Provider. The reason this type is Any at all is so different layers of the storage engine can cache differently shaped things, like a fully deserialized value for frequently accessed values. The Any trait just gives us a single extensibility point that only supports a single specialized usecase at a time, with Provider we'd simply provide another kind of thing a caller can ask for.

Suggested change

	pub trait TypeTag<'a>: Sized + 'static {
	type Type: 'a;
	}
	pub trait TypeTag: Sized + 'static {
	type Type<'a>: 'a;
	}

This could be made nicer with GATs, which should be stabilizing soon.

This change doesn't seem to make things much nicer, it just moves the lifetime parameter from the type to the associated type, it still needs to be specified in most cases. I think GATs make this harder to understand, and the original is closer to my mental model of a type tag in any case (i.e., the tag includes the lifetime bound of the type it represents as part of its type, rather than the tag represents a type constructor that can be parameterised by any lifetime).

RFC updated to take into account comments - thanks for the feedback!

One thing that has been floating in my mind is – what's the complexity of a function that provides a huge number of different types, and can this be made to be better than O(n) over the number of provide calls (possibly with some help from the compiler)? Can this be done with the API as proposed?

Given the particularities of phf and how match codegen works, I would intuit that it is indeed possible, but implementation in dyno and the shape of the proposed API in general would suggest to me that O(n) is the best possible. Is that true, or am I missing something?

It seems O(n) is the best we can do with this API. However, I don't think n should ever be large, in practice it will be limited by the number of fields (I guess you could be providing a whole bunch of new data, but it seems like you should have a dedicated, named API for that). I couldn't work out a way to have O(1) perf here, though I agree it seems like it should be possible some how.

It seems O(n) is the best we can do with this API. However, I don't think n should ever be large, in practice it will be limited by the number of fields (I guess you could be providing a whole bunch of new data, but it seems like you should have a dedicated, named API for that). I couldn't work out a way to have O(1) perf here, though I agree it seems like it should be possible some how.

Not sure if this is satisfying, but because of the flexibility of the provider API I think you could pretty much always just provide a more performant interface that does have O(1) perf, though ofc with the possibly chained nature on something like the error trait you might still be doing an O(N) lookup of that other trait interface. Admittedly though if that perf is important that seems like a good argument to figure out what that API would look like before stabilizing the current provider API, not after.

Isn't this unsound now (after Requisition<'a, '_> -> &mut Requisition<'a> change)?

Since Requisition is actually similar to (TypeId, Option<T>) for some erased T, this theoretically allows writing a safe_transmute by

1. Requesting type U
2. In provider replacing request for U with a request for T via mem::swap
3. In provider satisfying the request with T-value
4. Getting U from the request, but since the request was changed to a request for T, this reads T as U

I couldn't figure out how to get two requests with the same lifetime, but I'm sure there is a way/it's easy to accidentally allow something like this.

It seems like this API should either

1. Use Pin<&mut Requisition<'a>>, like the object_provider crate
2. Use Requisition<'a, '_> and provide a reborrow API to change the '_ and allow delegating

I don't think so because the implementation of request_by_type_tag will use the original Requisition (for U) not the new one (for T) so the user will always get back None.

Note that the old and new types are equivalent, with the lifetimes fully explicit they are Requisition<'a, 'b> and &'b mut Requisition<'a>

Doesn't request_by_type_tag call provide with a reference to it's Requisition? I imagined it's implemented in a similar facion:

#[repr(C)]
struct Requisition<'a> {
    id: TypeId,
    ph: PhantomData<&'a ()>,
}

pub fn request_by_type_tag<'a, I: TypeTag<'a>>(provider: &'a dyn Provider) -> Option<I::Type> {
    let mut req = (TypeId::of::<I>(), None::<I::Type>); // this should be a `repr(C)` struct, not a tuple
    let req = transmute::<_, &mut Requisition<'a>>(&mut req);
    provider.provide(req);
    req.1
}

In this example provider can change the TypeId that Requisition checks, by replacing it with another Requisition with a different TypeId, but the slot stays the same.

If I understand correctly, Requisition<'a, 'b> was wrapping *mut inside and so replacing it wouldn't bring inconsistency between TypeId and the slot.

Ah, I see what you mean, sorry.

Implementation is here: https://github.com/rust-lang/rust/pull/91970/files#diff-0752889661748b8a15a597d7156127b6fb90fdeda0627be50e07f3f785bd0f4dR798-R805

Requisition never stores the type id, it is synthesised as needed.

Its not actually possible to create a new Requisition since TagValue is private to the any module. In addition, Requisition is unsized so I think that even if you could create one, you can't assign it into the mutable reference.

Its not actually possible to create a new Requisition since TagValue is private to the any module

If Requisition was Sized, the only thing you'd need would be &mut Requisition, since you could use mem::swap.

Implementation is here: https://github.com/rust-lang/rust/pull/91970/files#diff-0752889661748b8a15a597d7156127b6fb90fdeda0627be50e07f3f785bd0f4dR798-R805

I see. Interesting, thanks!

If Requisition was Sized, the only thing you'd need would be &mut Requisition, since you could use mem::swap

I don't understand, how would you create the new Requisition to swap in? Requisition has a TagValue field and you need to name that to create it and thus create the Requisition.

You could do another request_by_type_tag, but I suppose there's no way to get the lifetimes to match up.

I've significantly reworked the RFC and draft implementations (https://github.com/nrc/provide-any and rust-lang/rust#91970). The main change is to remove type tags from the API. This lets us iterate on that part of the proposal without breaking changes. It also means the surface area of the RFC is much smaller. I have removed handling of mutable references for now (though they are still present in https://github.com/nrc/provide-any so you can see the changes requires) to further shrink the surface area. There are some further changes to the implementation to simplify it and provide better encapsulation of the implementation.

I think this resolves all open questions, though since the RFC is now somewhat different there may be new ones :-)

The new version makes much more sense to me!

@rfcbot concern lang-feature

I agree with Mara that it feels like there's a lang feature that might help here.

@rfcbot concern in-rfc-example-for-non-error

I also feel like, if we're going to have a more generalized API with type tags to enable use cases that are not Error, the text of the RFC needs an example that doesn't involve Error, to help motivate that.

@joshtriplett are these concerns resolved now that the type tags have been turned into an implementation detail?

Yes, I think these have both been resolved, and the new version looks great!

@rfcbot resolved lang-feature
@rfcbot resolved in-rfc-example-for-non-error

@rfcbot resolved typetag

Is it possible to request/provide with an extra const string FIELD_NAME? (may be blocked by const_adt_params) In many cases, there may exists multiple fields with same type in some traits. Although newtype is a good pattern, rust doesn’t has a good support for it now.

Is it possible to request/provide with an extra const string FIELD_NAME? (may be blocked by const_adt_params) In many cases, there may exists multiple fields with same type in some traits. Although newtype is a good pattern, rust doesn’t has a good support for it now.

imho it's better to use a struct type tag, rather than a const str as the disambiguating parameter, since that handles inadvertent typos or name clashes much better.

🔔 This is now entering its final comment period, as per the review above. 🔔

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

when this can be merged?

when this can be merged?

Please note that this is only an RFC, not an implementation.

Huzzah! The @rust-lang/libs-api team has decided to accept this RFC.

To track further discussion, subscribe to the tracking issue¹.

Footnotes

1. https://github.com/rust-lang/rust/issues/96024 ↩