SceneDB is a Cargo workspace with three crates. Add pulsar_scenedb for the core library, pulsar_scenedb_derive for the derive macros. Replication is always available, no feature gate. Everything GPU-related sits behind the gpu feature, off by default. C0: the core has zero graphics dependencies. Enable it for SceneGpuStore, the World mirror, or GPU asset storage.
1[workspace.dependencies]2pulsar_scenedb = { git = "https://github.com/Far-Beyond-Pulsar/SceneDB" }3pulsar_scenedb_derive = { git = "https://github.com/Far-Beyond-Pulsar/SceneDB" }45[dependencies]6pulsar_scenedb = { workspace = true, features = ["gpu"] }7pulsar_scenedb_derive = { workspace = true }89# "gpu" is opt-in (off by default) -- enables wgpu and every #[gpu]-mirrored10# path (SceneGpuStore, the World mirror, GPU asset storage). The storage,11# spatial, streaming, and replication layers need nothing beyond the12# default feature set.13#14# "telemetry" (also opt-in; pulls in "gpu") enables TelemetryServer -- the15# TCP monitoring socket that streams world/gpu snapshots to dashboards such16# as the companion scenedb_dashboard crate.
Every row gets a Handle, a packed u64 with a slot index, generation counter, and type tag. Storage lives in fixed-capacity SoA pages (256 rows default, 1024 max) with 64-byte aligned columns and a 128-byte per-element stride ceiling. Frame-boundary compaction is swap-and-pop: physical rows rearrange without breaking handles. Generation counters guarantee no dangling pointers.
Fields live in CPU columns by default. Adding #[gpu] creates an additional GPU-side mirror, an SSBO column updated by delta-sync. Only rows changed since the last sync upload. CPU-only fields consume no VRAM and generate no dirty words. They still participate in replication and spatial queries.
Two independent storage models share the same #[gpu] attribute. Inside a SpatialCell/CellStorage page, the mirror is written explicitly through the phase machine's witnesses. Inside a World (the archetype ECS, see ECS · World and World ↔ GPU Mirror below) with a mirror attached, the same attribute is written automatically inside world.insert() itself. No explicit write call.
World is SceneDB's archetype ECS. Every Entity lives in exactly one archetype, its exact set of component types. Inserting a component migrates the entity to a different archetype and moves only that entity's data. Queries iterate archetypes directly over dense per-component column slices.
1use pulsar_scenedb::{World, Entity};23struct Pos(f32, f32, f32);4struct Vel(f32, f32, f32);5struct Health(u32);67let mut world = World::new();89let e = world.spawn();10world.insert(e, Pos(0.0, 0.0, 0.0));11world.insert(e, Vel(1.0, 0.0, 0.0));1213for (entity, (pos, vel)) in world.query::<(&Pos, &Vel)>() {14 // entity: Entity, pos: &Pos, vel: &Vel15}1617// Don't need the entity handle? query_items skips fetching it.18for (pos, vel) in world.query_items::<(&Pos, &Vel)>() {19 // pos: &Pos, vel: &Vel20}2122// Bundle: one destination archetype, every column written directly.23world.reserve_bundle::<(Pos, Vel, Health)>(10_000);24let e = world.spawn_bundle((Pos(0.0, 0.0, 0.0), Vel(1.0, 0.0, 0.0), Health(100)));2526// get_mut returns a Mut guard. On a mirrored World its #[gpu] fields27// write through to the GPU when the guard drops.28{29 let mut health = world.get_mut::<Health>(e).unwrap();30 health.0 = 50;31}
get_mut returns a Mut guard. On a mirrored World, a #[gpu] field written through that guard writes through to the GPU when the guard drops, exactly like insert. query_items drops the entity handle from the iteration entirely when you don't need it.
A spatial cell wraps a page with six dedicated f32 columns for AABB min/max per axis. Queries scan the column arrays directly. No per-entity iteration, no allocation in the hot path. The SIMD layer accelerates with AVX2 (x86) and NEON (ARM), and a scalar reference matches them bit-for-bit.
1use pulsar_scenedb::{SpatialCell, Aabb, Handle};23// A spatial cell is a page of entities with six dedicated f32 columns4// for AABB min/max per axis. No per-entity iteration in queries.5let mut cell = SpatialCell::new(256).unwrap();67let handle: Handle = cell.alloc(Aabb {8 min: [0.0, 0.0, 0.0],9 max: [1.0, 1.0, 1.0],10}).unwrap();1112// Query scans directly over the column arrays, no allocation.13let mut results = vec![0u32; cell.rows_in_use() as usize];14let hit_count = cell.query_aabb(15 &Aabb { min: [-1.0; 3], max: [2.0; 3] },16 &mut results,17);18// results[0] == 0 (the handle's row passed the query)
The streaming grid classifies cells into Outer, Margin, or Inner domains using a concentric distance model with hysteresis bands. The bands damp boundary jitter. You pass a slice of observer AABBs, so overlapping players work correctly: a cell promotes if any player is close enough, and demotes only when all players have left. Cells can be pinned to any domain directly, bypassing distance rules.
1use pulsar_scenedb::gpu::grid::{StreamingGrid, GridConfig, CellCoord, Domain, StreamingBudget};23let mut grid = StreamingGrid::new(4 GridConfig {5 cell_width: 100.0,6 margin_radius: 150.0,7 pad_fraction: 0.10,8 hysteresis: 20.0,9 },10 StreamingBudget {11 vram_hlod_budget: 256_000_000,12 vram_geometry_budget: 1_000_000_000,13 max_materialized_cells: 1024,14 proxy_mesh_bytes: 4096,15 mean_cell_geometry_bytes: 1_048_576,16 },17 &[], // inner region classes18).unwrap();1920grid.materialize(CellCoord { x: 0, z: 0 });2122// Two players: overlapping load areas work correctly. A cell promotes23// if ANY player is close enough and demotes only when ALL have left.24grid.classify(&[25 Aabb { min: [-10.0, -10.0, -10.0], max: [10.0, 10.0, 10.0] },26 Aabb { min: [490.0, -10.0, -10.0], max: [510.0, 10.0, 10.0] },27]);2829let transitions = grid.take_transitions();3031// Pin a cell to keep it loaded regardless of player positions.32grid.pin(CellCoord { x: 5, z: 3 }, Domain::Inner);33grid.unpin(CellCoord { x: 5, z: 3 });
Alongside the paged SpatialCell/CellStorage model above sits a second, independent storage model: World, an archetype ECS (Entity, Component, archetype migration on insert). It works standalone, no GPU dependency. Attach a GpuMirrorHandle via World::new_with_gpu_mirror at construction, or World::attach_gpu_mirror later, and every #[gpu] field mirrors automatically. Until then, World::insert behaves as if the gpu feature were disabled.
Each #[gpu] field declares its own mirror mode. #[gpu(mirror = Once)] writes on the entity's first insert of that component, never again. The right choice for static data, a base transform or a mesh id. Plain #[gpu] (DirtyTracked, the default) marks the row dirty on every insert, writing nothing immediately. Nothing reaches the GPU until world.flush_gpu_mirror, once per frame. Both modes coalesce a frame's worth of writes. Adjacent rows upload as one contiguous range. Scattered rows take a different path: a GPU-side scatter-write compute pass instead of one upload call per row. Scattering is the common shape at scale, when entities churn (despawn/respawn) and a recycled entity index bears no relation to physical row adjacency. Either way the flush cost stays roughly constant.
1use pulsar_scenedb::{World, gpu::{GpuMirrorHandle, SceneGpuStore}};2use pulsar_scenedb_derive::SceneStore;3use std::sync::Arc;45/// #[gpu(layout = packed)] interleaves every #[gpu] field into one SSBO6/// row instead of one buffer per field.7#[derive(SceneStore, Clone, Copy)]8#[gpu(layout = packed)]9struct Instance {10 #[gpu(mirror = Once)] // written on first insert, never again11 model: [f32; 16],12 #[gpu(mirror = Once)]13 normal_mat: [f32; 16],14 #[gpu] // DirtyTracked (the default): re-synced on change15 mesh_id: u32,16}1718// Setup, once. No register_gpu_columns call needed -- the first insert of19// a #[gpu]-bearing type registers its columns for you.20let store = Arc::new(SceneGpuStore::new(&ctx, cfg));21let mut world = World::new_with_gpu_mirror(GpuMirrorHandle::new(Arc::clone(&store), queue.clone()));2223let entity = world.spawn();24world.insert(entity, Instance { model, normal_mat, mesh_id: 7 });25// Every #[gpu] field above already wrote or dirty-marked itself inside26// insert() -- no manual dispatch call.2728// Once per frame, after your simulation step:29world.flush_gpu_mirror(&queue);
Growth is lazy and unbounded by default. The first insert whose entity index doesn't fit the current buffer grows it, a real GPU-to-GPU copy. Reserve capacity up front when a batch size is known ahead of time. Symmetrically, shrink_gpu_mirror_to_fit reclaims capacity after a load spike settles.
1// Ahead of a known-size batch (streaming a sublevel, spawning a wave):2world.reserve_gpu_mirror_capacity(&queue, 10_000)3 .expect("mirror attached")4 .expect("reserve succeeds");56// At a natural boundary after a peak-then-drop (not every frame --7// this is a real GPU-to-GPU copy, same cost as growth):8world.shrink_gpu_mirror_to_fit(&queue, highest_live_entity_index, 1.5);
A GPU-resident generation buffer (GpuMirrorHandle::generations()) tracks entity liveness automatically, in lockstep with World::is_alive's own CPU-side check. A shader holding a captured (row, generation) pair detects a stale reference the same way the CPU does. Entities with no #[gpu] field pay nothing: the liveness entry is written lazily, on the entity's first #[gpu]-bearing insert.
#[derive(SceneStore)] generates a Pod impl, the SceneColumnSet column layout, GpuColumnSet GPU write dispatch, and MirrorMode wiring from a repr(C) struct. It only processes #[gpu(...)] attributes. Any other attribute passes through unmodified.
1use pulsar_scenedb_derive::SceneStore;23/// A material component with mixed storage locations:4/// - color, roughness, metallic → CPU + GPU (dirty-tracked mirror)5/// - name → CPU only (no GPU mirror, no VRAM cost)6#[derive(SceneStore)]7#[repr(C)]8pub struct Material {9 #[gpu] // CPU + GPU, DirtyTracked10 pub albedo: [f32; 4],1112 #[gpu(mirror = DirtyTracked)] // CPU + GPU, explicit13 pub roughness: f32,1415 #[gpu] // CPU + GPU, DirtyTracked16 pub metallic: f32,1718 // No #[gpu]: CPU only. No VRAM, no dirty tracking.19 pub name: [u8; 64],20}
#[derive(Replicate)] reads #[replicate(...)] attributes and generates a register_replication function. It registers a real per-named-field accessor with the ReplicationRegistry, driving delta encoding and interest management. The two derives are orthogonal. Stack them on the same struct, even the same field, since each only reads its own attributes.
1use pulsar_scenedb_derive::Replicate;2use pulsar_scenedb::ReplicationEncoding::{self, *};3use pulsar_scenedb::ReplicationCondition::{self, *};45/// A player state component with per-field replication control.6#[derive(Replicate, Default)]7struct PlayerState {8 /// Full transform: replicated every frame as raw Pod bytes.9 #[replicate(encoding = Pod, condition = Always)]10 position: [f32; 3],1112 /// Health: only sent to non-owning simulated proxies.13 #[replicate(encoding = DeltaCompressed, condition = SimulatedOnly)]14 health: f32,1516 /// Ammo: only relevant to the owning client.17 #[replicate(encoding = Pod, condition = AutonomousOnly)]18 ammo: u32,1920 /// Sent once at spawn, never again.21 #[replicate(encoding = Serialized, condition = InitialOnly)]22 inventory: Vec<Item>,2324 /// One-shot event, delivered via the RPC channel.25 #[replicate(encoding = Event, condition = Multicast)]26 on_damage_taken: DamageEvent,27}2829let mut registry = ReplicationRegistry::new();30PlayerState::register_replication(&mut registry);
SceneDb is the frame facade. It owns a World, a SubsystemRegistry, and a FrameDriver. Subsystems are named, phase-gated plugins whose hooks (simulate_a/simulate_b, harvest, boundary) all default to no-ops. Implement only what you need. db.step() runs SimulateA → SimulateB across every subsystem and flushes the World mirror when one is attached. step_gpu additionally runs the harvest and boundary stages against a caller-supplied SceneGpuStore.
1use std::any::Any;2use pulsar_scenedb::{SceneDb, Subsystem, World};3use pulsar_scenedb::gpu::{SimulateA, SimulateB, RetiredPhase};45struct PhysicsSubsystem { gravity: [f32; 3] }67impl Subsystem for PhysicsSubsystem {8 fn name(&self) -> &'static str { "physics" }910 fn simulate_a(&mut self, _world: &mut World, _witness: &SimulateA) {11 // gameplay mutation is permitted here12 }1314 fn simulate_b(&mut self, _world: &mut World, _witness: &SimulateB) {15 // physics writeback16 }1718 fn boundary(&mut self, _phase: &RetiredPhase) {19 // after retire, before compact20 }2122 fn as_any(&self) -> &dyn Any { self }23 fn as_any_mut(&mut self) -> &mut dyn Any { self }24}2526let mut db = SceneDb::new();27db.register_subsystem(PhysicsSubsystem { gravity: [0.0, -9.8, 0.0] });2829db.step(); // SimulateA -> SimulateB across every subsystem; flushes the World mirror if attached3031let physics = db.subsystem_mut::<PhysicsSubsystem>().unwrap();32physics.gravity = [0.0, -1.6, 0.0];3334// By-name path for scripts/events: invoke a #[subsystem_method] through35// the reflection registry.36db.dispatch("physics", "apply_impulse", vec![Box::new(42u64), Box::new([1.0f32, 0.0, 0.0])])37 .expect("dispatch succeeds");
Subsystems are addressable two ways. Typed via subsystem_mut::<T> for ordinary Rust. By registered name via dispatch / subsystem_by_name_mut for scripts and editor tooling. #[scenedb_subsystem] marks an impl block, #[subsystem_method] marks the callable methods on it.
RelationIndex turns a component's cross-entity links into dense, columnar buffers. Rebuild it once per boundary; harvest reads borrow the result with no allocation. A pair is confirmed only when both sides link back to each other, and is then emitted exactly once. Everything else lands in unmatched (the target has no link back) or conflicts (the target reciprocates with someone else). Each conflict carries what the target links to instead, so the caller decides how to resolve.
1use pulsar_scenedb::{RelationIndex, Entity};23struct PortalLink { linked_to: Entity }45// Rebuild once per boundary: scan every PortalLink in the world and6// classify each into a confirmed pair, unmatched, or a conflict.7let mut index = RelationIndex::new();8index.build::<PortalLink>(&world, |link| link.linked_to);910let view = index.view();11// view.pairs: &[(Entity, Entity)] -- reciprocal links, emitted once each12// view.unmatched: &[Entity] -- target has no link back (or no component)13// view.conflicts: &[ConflictEntry] -- target reciprocates with someone else;14// each entry carries source, target, and ConflictReason::NotReciprocated(15// what_the_target_links_to_instead)
A compile-time frame phase machine turns the frame's phase into a type. Holding a SimulateA/SimulateB permits mutation (A = gameplay, B = physics writeback), a HarvestPhase permits read-back, and a RetiredPhase permits compaction. Each transition consumes the previous witness, so reordering or skipping a phase won't compile. No runtime checks, no lock contention, no phase-order bugs. A FrameDriver owns the frame's progression; SceneDb::step/step_gpu drive it for you.
1// FrameDriver owns one frame's progression. Each transition consumes the2// previous witness, so reordering or skipping a phase won't compile.34let sim_a = driver.begin(); // SimulateA -- gameplay mutation5let sim_b = sim_a.end(); // SimulateB -- physics writeback6let harvest = sim_b.end(); // HarvestPhase -- read-only snapshots7let boundary = harvest.end(); // BoundaryPhase8let (retired, _drained) = boundary.retire(store, cells); // RetiredPhase9let stats = retired.compact(store, cells).sync(store, cells); // SyncStats1011// store/cells are the caller's SceneGpuStore and CellSlot slices.12// SceneDb's step()/step_gpu() drive exactly this chain for you.
SceneDB records every mutation during Simulate (change tracking), encodes field deltas per a component schema (delta encoding), filters which client sees what (interest management + conditions), resolves who is allowed to write what (authority table), handles one-shot RPCs (event channel), and supports client-side prediction with server reconciliation (snapshots + reconciler).
Delta::apply carries no ordering guard. Frame ordering belongs to the transport. SceneDB does not own transport, encryption, or asset streaming. It produces Delta and EventBatch byte payloads and specifies per-field encodings.
1use pulsar_scenedb::{World, ChangeTracker, CpuSimulateWitness};23let mut world = World::new();4let mut tracker = ChangeTracker::new();5let witness = CpuSimulateWitness::new();67let delta = witness.run_tracked(&mut world, &mut tracker, |world, tracker| {8 // Systems write to the world and track changes here.9 let entity = world.spawn_tracked(tracker);10 world.insert_tracked(entity, 100.0f32, tracker);11});1213// delta contains spawned entities, despawned entities, and component14// changes, each already encoded via the field's Replicable impl.
1use pulsar_scenedb::{Snapshot, RelevanceSet};23// Full world state.4let full = Snapshot::capture_full(&world, ®istry, current_frame);56// Only entities relevant to a specific client.7let relevant = Snapshot::capture_relevant(&world, ®istry, &relevance, current_frame);89// Restore into a World, e.g. a client resyncing after a connection gap.10let mut client_world = pulsar_scenedb::World::new();11full.restore_to_world(&mut client_world, ®istry).unwrap();
Every layer is a bounded unit with a single responsibility:
| Layer | Location | Types | Responsibility |
|---|---|---|---|
| Storage | CPU | CellStorage, Page, PageLayout, LivenessMask | SoA pages, alloc/free, swap-and-pop compaction, handle→row indirection |
| ECS | CPU | World, Entity, Bundle, WorldQuery, QueryIter, QueryItemsIter, Mut | Archetype storage with edge-cached migration, bundle spawn/insert, typed queries, GPU write-through on Mut drop |
| Spatial | CPU | SpatialCell, Aabb, Frustum | Six bounds columns, AABB + frustum queries, scalar + SIMD |
| Streaming | CPU | StreamingGrid, CellCoord, Domain, GridConfig | Concentric classification, hysteresis, cross-fade, persistent pinning |
| GPU store | GPU | SceneGpuStore, RegionPool, SceneBuffer, CellGpuState | Region-partitioned SSBOs, delta-sync, generation validation, device loss rebuild |
| GPU buffers | GPU | GpuBufferRegistry, DynamicGpuBuffer, GrowableSceneBuffer | One keyed registry for every GPU buffer; pipeline-owned dynamic/growable SSBOs; explicit register/set/flush path for tooling |
| World mirror | CPU+GPU | World, Entity, GpuMirrorHandle, MirrorMode, Mut, DirtyTrackedSceneBuffer, GenerationMirror | Automatic per-field write/dirty-mark on insert, write-through on Mut drop, batched flush, GPU scatter-write for scattered churn, GPU liveness mirror |
| Harvest | CPU→GPU | HarvestPipeline, HarvestStaging, View, MeshClass | Per-view spatial queries, DEI compact, per-class token routing, upload to VRAM |
| SceneDb | CPU+GPU | SceneDb, Subsystem, SubsystemRegistry, FrameDriver | Frame facade: owns World + subsystem registry + frame driver; step()/step_gpu(); typed or by-name subsystem access; reflection dispatch |
| Relations | CPU | RelationIndex, RelationView, ConflictEntry, ConflictReason | Columnar relational view over World component links; reciprocal pairs + unmatched/conflict buffers |
| Scheduler | CPU | Schedule, SystemFn | Ordered per-tick systems receiving (&mut World, GameTime) |
| Actors | CPU | Actor, ActorRegistry | Lifecycle-driven autonomous objects (begin_play/tick/end_play) backed by one Entity each |
| Telemetry | CPU | TelemetryServer, TelemetrySnapshot | TCP monitoring snapshots of world/gpu state (feature: telemetry) |
| Phase machine | CPU | FrameDriver, SimulateA, SimulateB, HarvestPhase, BoundaryPhase, RetiredPhase | Compile-time frame phase guards |
| Assets | GPU | GeometryArena, MeshRegistry, ClusterBuffer, TextureStore, MeshletBuffer | GPU-side asset storage with suballocation |
| Lease | CPU | Lease, LeaseMask, Scratchpad | RAII read leases, decaying per-frame scratch buffers |
| Replication | CPU | ChangeTracker, CpuSimulateWitness, Delta, Replicable, ReplicationRegistry, SchemaBuilder, RelevanceSet, EntityCellMap, AuthorityTable, EventBatch, Snapshot, Reconciler, DeltaCompressor | Change tracking, delta encoding, interest management, ownership, condition filtering, RPC channel, snapshots + resync, prediction reconciliation |