Breaking changes to consider for foundations 6

[TheJokr]

This is a tracking issue to compile breaking changes we want to consider before releasing foundations 6.

[TheJokr]

foundations::security::allow_list generates once_cell::sync::Lazy instances in user code. We should consider switching this macro to use std::sync::LazyLock instead, which allows us to drop our once_cell dependency completely. (See also #157)

[TheJokr]

Metric functions with and without labels have a subtle difference in behavior: unlabeled metrics are emitted with their default value immediately, while labeled metrics are only created on first usage. (Naturally, since we can't know the labels ahead of time.)

This can cause a lot of uninteresting metrics to be emitted when only part of a metrics-enabled library is actually executed.

Making unlabeled metrics behave like labeled ones, by initializing them on first use, would make foundations' metrics handling more consistent and avoid wasting storage in such cases. However, zero values in prometheus are useful (e.g. for rate() calculations), so maybe we should offer both options?

[tsheinen]

re: #116 and #166 we could consider changing the API of the telemetry server so that it can be used without other crates? right now it's not usable without hyper and http-body-util (both now exported) but if we're considering breaking changes we could change the API to not use them directly

[TheJokr]

We should forward add_docs on enums and newtype structs. There is no technical blocker, but it is a breaking change since we technically don't require T: Settings for a struct Newtype(T) today.

[TheJokr]

Changing the default sampling rate is also a breaking change. I agree with Anthony that the default shouldn't be 100%: #103

IMO, the default should be for tracing to be disabled entirely. Sending jaeger UDP to localhost:6831 doesn't make sense for most users these days.

[TheJokr]

We should upgrade sentry to the latest major version. Since we expose a sentry before_send hook, this needs to happen in a major release.

E: We decided to extract the sentry functionality into a separate crate instead. This crate can make major releases more often. #203

[TheJokr]

Our current metrics implementation is tightly coupled to a specific (pre-release) prometheus-client version. This has caused us some hassle in the recent past already (see #213). Instead, it would be nice if we had a dedicated foundations-metrics crate which contains our own global singleton registry. This can (hopefully) be stable across many foundations major versions, such that they can coexist in the same binary.

foundations-metrics should have a minimal API:

• A trait that metrics implement to encode into an output format
• A function to register a metric in the global registry (either a single Box<dyn OurMetricTrait> or a vec of boxed metrics)
• A function to collect all registered metrics for encoding

(Writing this down now, it seems very similar to the API of the non-official prometheus crate: https://docs.rs/prometheus. Maybe we just switch to that as our backing store?)

With this breaking change we also want to support protobuf metrics output (required for native histograms), so the output format should be protobuf. We can convert from protobuf to OpenMetrics text on the fly.

[TheJokr]

Similar to the above, we should make it easy for libraries to use foundations' logging and tracing features across major versions. This requires splitting the APIs for those into separate crates (e.g., foundations-logging, foundations-tracing), or explicitly documenting external facades that can be used (slog, cf-rustracing).

As with metrics, the API for these crates needs to be very stable, so should be kept limited. For example, output handling should stay in foundations.

[TheJokr]

jemalloc has the unprefixed_malloc_on_supported_platforms feature, which makes jemalloc the default allocator for non-Rust code as well. Its easy for our users to enable themselves (with an explicit tikv-jemalloc-sys dependency), but making this the default would avoid surprises (as well as making jemalloc compatible with ASAN!)