FJoinStreamReader

The joiner's half: takes the stream in pieces of any size and keeps the save as a sparse file.

Declaration

class FJoinStreamReader

Header: #include "Native/Net/NetJoin.h"
Module: PublicDependencyModuleNames.Add("OrreryTerrainCore"); in your module’s Build.cs.
Stability: Experimental: supported, may change in a minor version, with a changelog line.
Thread: Feed, TakeDrain, TakeCacheFill and ResidentBytes on one thread (the game thread); Drain on any thread, one call at a time; the file is thread-safe, for the page jobs.

Description

The file answers IoError for bytes that have not arrived, so a world loaded from it with a page store pages its codes in as they come (a cold tile is answered from its summary meanwhile).

Keeping the save. The file knows which runs of the save arrived and where each one is. Without a store (FJoinReaderConfig::MakeStore empty) every run stays in memory. With one, Drain appends each run to the store and the run leaves memory once written; until LoadJoin has loaded from the file every run also stays in memory, so the load reads nothing from the store on the game thread. After the load the joiner holds the runs not yet written and the index of runs, not the save.

Joining from a kept save. A stream whose head says WholeSave or HeldCodes leaves out bytes the joiner holds in FJoinReaderConfig::Cache. TakeCacheFill says when, and FillFromCache adds them, off the game thread: a code joins the file only once its bytes hash to the server's index, and it is then read from the kept save, never copied. A HeldCodes stream waits for CacheAnswer before it sends the rest.

Tier 1. Feed, TakeDrain, TakeCacheFill and ResidentBytes on one thread (the game thread); Drain on any thread, one call at a time; the file is thread-safe, for the page jobs.

Functions

FJoinStreamReader (2 overloads)
operator=
Feed
Take the next bytes of the stream.
TakeDrain
Whether the caller owes a Drain: bytes arrived for the store and no drain is under way.
Drain
Write every run that arrived and is not in the store yet to it, making the store first; returns once none is left, including runs that arrive meanwhile.
TakeCacheFill
Whether the caller owes a FillFromCache: true once per stream, when a WholeSave head arrived, or when a HeldCodes stream's load set did.
FillFromCache
Add the bytes this joiner holds from its kept save to the file.
CacheFilled
FillFromCache finished, whatever it answered.
CachedBytes
Save bytes FillFromCache added so far.
CacheAnswer
What a HeldCodes stream's writer needs to hear (FJoinStreamWriter::TakeHeldCodes): U64 the head's FileHash, U32 the index's tile count, then one bit a tile in index order, lowest bit first, set when the joiner holds its code.
Error
Also StoreFailed once a Drain failed. Feed's thread.
HasHead
Head
Valid with HasHead.
CanLoad
The head, every start, every mask and the whole load set arrived (from a WholeSave stream's kept save, once FillFromCache put it in): LoadJoin may run in a world with a page store.
Masks
The brush masks the mask frames carried so far, decoded and checked, in the head's order.
Complete
The end frame arrived and, with a WholeSave stream, FillFromCache finished: every byte of the save is there.
StreamEnded
The end frame arrived, whatever is still to come from the kept save. Feed's thread.
FileBytes
Save bytes received over the stream; the kept save's are CachedBytes.
ResidentBytes
The joiner's own heap for the join, bytes: the frame being received, the save runs held in memory and the index of runs.
File
The save as received, for LoadJoin (and FTerrainSerialization::LoadFromFile).
Verify
Read and hash every byte (FTerrainSerialization::VerifySave) and hold the trailer to the head's FileHash: the join's last check, once Complete.