Expand description
Local filesystem backend for development and testing.
Stores each object as one file below the configured root directory. The file holds the object’s metadata as a single JSON line, followed by the raw payload.
§Layout
{root}/
├── {usecase}/{scopes}/objects/{key}.obj object files
├── .uploads/{upload_id} in-progress resumable uploads
└── .locks/{xx}/{yy} lock slotsInternal directories start with ., which usecase names cannot contain, so they never collide
with objects.
Each scope becomes one directory level named {name}.{value}, in scope order. Objects without
scopes are stored directly under {usecase}/objects/.
Each /-separated segment of the key becomes one directory level, and the last segment gets a
.obj suffix. Segments are encoded first. For example, the key docs/report
in usecase attachments with scopes org=17;project=42 is stored at
attachments/org.17/project.42/objects/docs/report.obj. Because of the suffix, keys like a
and a/b can coexist.
§Key encoding
Key segments are stored mostly verbatim, so that paths stay readable. To keep every key inside
the root and every key mapped to a distinct path, these characters are percent-encoded as
%XX:
%itself and ASCII control characters, including NUL.- Characters reserved on Windows:
\ < > : " | ? *. - A
.at the start of a segment, so no segment is.or.., or a hidden file. - A
.or space at the end of a segment, which Windows would strip. - The
.of a trailing.obj(in any case), so no directory looks like an object file.
Empty segments, such as in a//b, are stored as %.
§Concurrency
Complete object files are published by atomically renaming same-directory drafts, so readers observe either the previous or next complete file. Unpublished drafts are removed automatically when dropped.
Deleting an object also removes its directories while they are empty. This is not synchronized with writers: directory removal only succeeds on empty directories, and writers recreate a directory removed before they placed their file in it.
To avoid races on metadata, expiry, and upload updates, this backend uses locks placed under
.locks/ to synchronize mutations across backend instances and cooperating processes. The first
two bytes of the BLAKE3 hash of an object’s storage path select one of 65,536 permanent lock
slots under .locks/<first byte>/<second byte> (lowercase hexadecimal). An object and all of its
resumable uploads share the same lock.
Shared filesystems are supported only when locks propagate across the cluster, pathname visibility is coherent, and rename is atomic.
§Limitations
- Keys that differ only in case or Unicode normalization collide on filesystems that ignore those differences, such as the macOS default.
- Each encoded key segment must fit the filesystem’s file name limit, typically 255 bytes.
- Windows additionally rejects reserved device names such as
CONorNUL. These are stored unescaped.
§Permissions
Newly created files and directories are owner-only on Unix (0600 and 0700 respectively, further restricted by umask). Existing paths retain their permissions.
Structs§
- File
System Config - Configuration for
LocalFsBackend. - Local
FsBackend - Local filesystem backend for development and testing.