Struct TieredStorage 

pub struct TieredStorage { /* private fields */ }

Two-tier storage backend that routes objects by size.

TieredStorage implements Backend and is intended to be used inside a StorageService, which wraps it with task spawning and panic isolation.

Size-Based Routing

Objects are routed at write time based on their size relative to a 1 MiB threshold:

• Objects ≤ 1 MiB go to the high_volume backend — optimized for low-latency reads and writes of small objects (e.g. BigTable).
• Objects > 1 MiB go to the long_term backend — optimized for cost-efficient storage of large objects (e.g. GCS).

Redirect Tombstones

Because the ObjectId is backend-independent, reads must be able to find an object without knowing which backend stores it. A naive approach would check the long-term backend on every read miss in the high-volume backend — but that is slow and expensive.

Instead, when an object is stored in the long-term backend, a redirect tombstone is written in the high-volume backend. It acts as a signpost: “the real data lives in the other backend at this target.” On reads, a single high-volume lookup either returns the object directly or follows the tombstone to long-term storage, without probing both backends.

How tombstones are physically stored is determined by the HighVolumeBackend implementation — refer to the backend’s own documentation for storage format details.

Consistency

Consistency across the two backends is maintained through compare-and-swap operations on the high-volume backend (see HighVolumeBackend::compare_and_write), not distributed locks. Each mutating operation reads the current high-volume revision, performs its work, and then atomically swaps the high-volume entry only if the revision is still current — rolling back on conflict. Cleanup of unreferenced LT blobs runs in background tasks so the caller returns as soon as the commit point is reached. Call Backend::join during shutdown to wait for outstanding cleanup.

See the module-level documentation for per-operation diagrams.

Usage

TieredStorage handles only the routing and consistency logic. Wrap it in a StorageService to add task spawning, panic isolation, and concurrency limiting.

Implementations

impl TieredStorage

pub fn new( high_volume: Box<dyn HighVolumeBackend>, long_term: Box<dyn Backend>, changelog: Box<dyn ChangeLog>, ) -> Self

Creates a new TieredStorage with the given backends and change log.

Trait Implementations

impl Backend for TieredStorage

fn name(&self) -> &'static str

The backend name, used for diagnostics.

fn put_object<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, id: &'life1 ObjectId, metadata: &'life2 Metadata, stream: ClientStream, ) -> Pin<Box<dyn Future<Output = Result<PutResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Stores an object at the given path with the given metadata.

fn get_object<'life0, 'life1, 'async_trait>( &'life0 self, id: &'life1 ObjectId, ) -> Pin<Box<dyn Future<Output = Result<GetResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Retrieves an object at the given path, returning its metadata and a stream of bytes.

fn get_metadata<'life0, 'life1, 'async_trait>( &'life0 self, id: &'life1 ObjectId, ) -> Pin<Box<dyn Future<Output = Result<MetadataResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Retrieves only the metadata for an object, without the payload.

fn delete_object<'life0, 'life1, 'async_trait>( &'life0 self, id: &'life1 ObjectId, ) -> Pin<Box<dyn Future<Output = Result<DeleteResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Deletes the object at the given path.

fn join<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = ()> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Waits for any outstanding background operations to complete before shutdown.

impl Debug for TieredStorage

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.