Struct EnvelopeSpool

pub struct EnvelopeSpool {
    pub path: Option<PathBuf>,
    pub max_disk_size: ByteSize,
    pub batch_size_bytes: ByteSize,
    pub max_envelope_delay_secs: u64,
    pub disk_usage_refresh_frequency_ms: u64,
    pub max_backpressure_envelopes: usize,
    pub max_backpressure_memory_percent: f32,
    pub partitions: NonZeroU8,
}

Persistent buffering configuration for incoming envelopes.

Fields

path: Option<PathBuf>

The path of the SQLite database file(s) which persist the data.

Based on the number of partitions, more database files will be created within the same path.

If not set, the envelopes will be buffered in memory.

max_disk_size: ByteSize

The maximum size of the buffer to keep, in bytes.

When the on-disk buffer reaches this size, new envelopes will be dropped.

Defaults to 500MB.

batch_size_bytes: ByteSize

Size of the batch of compressed envelopes that are spooled to disk at once.

Note that this is the size after which spooling will be triggered but it does not guarantee that exactly this size will be spooled, it can be greater or equal.

Defaults to 10 KiB.

max_envelope_delay_secs: u64

Maximum time between receiving the envelope and processing it.

When envelopes spend too much time in the buffer (e.g. because their project cannot be loaded), they are dropped.

Defaults to 24h.

disk_usage_refresh_frequency_ms: u64

The refresh frequency in ms of how frequently disk usage is updated by querying SQLite internal page stats.

Defaults to 100ms.

max_backpressure_envelopes: usize

The amount of envelopes that the envelope buffer can push to its output queue.

Defaults to 500.

max_backpressure_memory_percent: f32

The relative memory usage above which the buffer service will stop dequeueing envelopes.

Only applies when Self::path is set.

This value should be lower than Health::max_memory_percent to prevent flip-flopping.

Warning: This threshold can cause the buffer service to deadlock when the buffer consumes excessive memory (as influenced by Self::batch_size_bytes).

This scenario arises when the buffer stops spooling due to reaching the Self::max_backpressure_memory_percent limit, but the batch threshold for spooling (Self::batch_size_bytes) is never reached. As a result, no data is spooled, memory usage continues to grow, and the system becomes deadlocked.

Example

Suppose the system has 1GB of available memory and is configured to spool only after accumulating 10GB worth of envelopes. If Relay consumes 900MB of memory, it will stop unspooling due to reaching the Self::max_backpressure_memory_percent threshold.

However, because the buffer hasn’t accumulated the 10GB needed to trigger spooling, no data will be offloaded. Memory usage keeps increasing until it hits the Health::max_memory_percent threshold, e.g., at 950MB. At this point:

• No more envelopes are accepted.
• The buffer remains stuck, as unspooling won’t resume until memory drops below 900MB which will not happen.
• A deadlock occurs, with the system unable to recover without manual intervention.

Defaults to 90% (5% less than max memory).

partitions: NonZeroU8

Number of partitions of the buffer.

A partition is a separate instance of the buffer which has its own isolated queue, stacks and other resources.

Defaults to 1.

Trait Implementations

impl Debug for EnvelopeSpool

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for EnvelopeSpool

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for EnvelopeSpool

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for EnvelopeSpool

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.