OpenMax
Free Trial

Gap fill

createChatGapFillScheduler with its options, read methods, entry points for each moment and results.

Gap fill

Exports of the @openmaxai/chat-store root entry point for the gap-fill scheduler. Status, wiring it into a host and an index of every export are on the Overview.

The gap-fill scheduler decides when, and from which seq, to fetch forward, and writes what comes back through the store's write entry point. It sends no requests (the read methods are injected by the caller), does not touch the DOM, and does not read the clock; its only timer is the backoff before resending a failed fetch by id (see Catching up on changes). When and how gaps are filled is on Realtime and sync · Gap-fill.

One gap-fill follows the conversation's state in the store at that moment:

  • The range reaches the latest, or a fill is halfway (an earlier gap-fill attached a batch from the watermark and the server said there is more): fetch forward from the watermark and attach after the range.
  • The range was told it reached the latest but ends below the watermark (a late window, or the message at the watermark was deleted): fetch by seq forward from the end of the range.
  • Otherwise (never loaded, no watermark yet, a window around a message, the newer end unloaded by scrolling up): no fill. While the window does not reach the latest, only newerPending says there are new messages; with latestWindowOnGap set, a messages signal during that time loads the latest window once instead.

Every round passes the watermark at request time (requestWatermark). One gap-fill of a conversation calls loadThreadSince at most 10 times; if it is still not done at that limit, it loads the latest window once and this gap-fill ends there: when that window connects to the range (the oldest message's seq is at most the start of this round plus one: the start is usually the watermark, or the end of the range when the range stops below the watermark), it is appended after the range, and messages in the overlap that were there at request time but are missing from the window are treated as deleted; only when it does not connect (a gap remains) is the whole window swapped in. "Halfway" is recorded in the store, not in the scheduler: if the previous gap-fill failed midway, or the scheduler was stopped and a new one created on the same store, the next gap-fill still continues from the watermark.

Loading the latest window while the window does not reach the latest

When the scheduler is created with latestWindowOnGap, and a messages signal arrives while the conversation has a range that does not reach the latest (a window around a message, or the newer end unloaded by scrolling up; not a fill that is halfway, and not a range ending below the watermark), this gap-fill loads the latest window once instead: source.loadThread(id, undefined, { limit }). It still never fetches by seq into the gap.

  • It connects (the oldest message's seq is at most the end of the range plus one): it is appended after the range, messages in the overlap that were there at request time but are missing from the window are treated as deleted, and the range reaches the latest. Another gap-fill follows, fetching from the watermark what arrived after the window was loaded. The result is reloaded with rounds 0.
  • It does not connect (a gap remains): nothing is written; it is kept aside per conversation (only the most recently loaded one), and the result is skipped. A thread loader on the same store checks it against the end of the range before paging newer (loadNewer): if it connects, it is appended on the spot without a request; otherwise one page is loaded as usual and it is checked again after that page. Appending it is likewise followed by another gap-fill.
  • A window that comes back with a newer cursor (newerCursor) is not the latest window and is dropped.
  • If the conversation was closed, the window reached the latest, or the conversation was cleared (clearConversation) while the request was in flight, the window is dropped (interrupted). close on that conversation and stopping the scheduler also drop the window kept aside; so does clearing the conversation (it is not appended later even if the conversation stays open and loads a window around a message again).
  • Reconnecting, returning to the foreground, opening a conversation and sending do not load it. The rule that one conversation never fills concurrently still holds: signals arriving while the request is in flight merge into one more fill.

Without latestWindowOnGap nothing changes: a messages signal during that time gets skipped and nothing is fetched.

Catching up on changes

Gap-fill only fetches messages newer than the watermark, so changes other people made to existing messages while you were away or offline (edits, recalls, deletes, reactions added or removed, cards settled) are not among them. When the read methods include loadChangesSince and getMessagesByIds, every gap-fill goes on to catch up on changes after fetching new messages, and the result goes in the changes of ChatGapFillResult:

  1. Fetch the change log forward from the conversation's change watermark (see Write events · The change watermark and change numbers), page after page, until hasMore is false.
  2. Pick the changed messages inside the range held: those older than both the range and the stretch on the device are not fetched, and those newer than the watermark are not held and not fetched either. Dedupe, fetch them by id in batches of 100, and put ids returned as deferredIds at the back to fetch again.
  3. Write what came back into the store: messages in the range are merged by id, messages not in memory but in the stretch on the device are written to the device, and missingIds are handled as deletes.
  4. Only after that does the change watermark move, to the newest number at the end of the change log. The stretch's change watermark is kept in its segment record and goes through the same write queue as the messages, after them, so the device's change watermark never runs ahead of its messages.

Exceptions:

  • Too old. When the change log says tooOld (past retention, or the change watermark is above the newest number), or one round would take more than 10 pages: drop the stretch on the device and swap in the latest window, whose headChangeSeq becomes the change watermark. When the window does not reach the latest (you jumped to a message), the window you are looking at is not replaced; only the stretch on the device and the change watermark are dropped.
  • A newer push arrived during the fetch. The copy fetched by id is a snapshot from request time and may overwrite a newer push already written: then the change watermark only moves to the number at fetch time and another round fetches those again, up to 3 rounds; after 3 rounds it stops (incomplete) and waits for the next moment.
  • The window was replaced or the conversation closed meanwhile: what came back is not written (interrupted).
  • A fetch by id failed. When a batch rejects with an error carrying retryable: true (for example a ChatSdkError from @openmaxai/chat-sdk for a network failure, timeout, temporarily unavailable service or rate limit), the same batch is sent again after 1 second, and after another 2 seconds if it fails again: at most 3 sends per batch. Other errors (unauthorized, forbidden, not found, anything without retryable: true) are not retried. If the scheduler is disposed (dispose), the conversation is closed (close) or a window is swapped in during the backoff, nothing more is sent. When no send succeeds, this catch-up ends with the last error (failed) and the change watermark does not move: the next gap-fill lists those changes again from the same change watermark and fetches them again.
  • The user switched: what came back still goes into memory, but not onto the new user's device, and the device's change watermark does not move.

Realtime pushes with a change number move the change watermark as in Write events · The change watermark and change numbers; when pushes skip a number (the missing ones have not arrived), call changes.

createChatGapFillScheduler

function createChatGapFillScheduler(options: ChatGapFillSchedulerOptions): ChatGapFillScheduler;

Creates a gap-fill scheduler. Throws TypeError when options.store was not created by createChatStore.

ChatGapFillSchedulerOptions

interface ChatGapFillSchedulerOptions {
  store: ChatStore;
  source: ChatGapFillSource;
  latestWindowOnGap?: {
    limit?: number;
  };
}
Field Description
store The store to write back to; must be created by createChatStore. Halfway progress is recorded in the store, so stopping the old scheduler and creating a new one on the same store continues the fill unaffected.
source The read methods.
latestWindowOnGap Load the latest window once when a messages signal arrives while the window does not reach the latest; see Loading the latest window while the window does not reach the latest. limit is passed to source.loadThread as the third argument { limit } (an upper-bound hint the source may ignore); without it the third argument is omitted. Without this option the window is not loaded.

ChatGapFillSource

interface ChatGapFillSource {
  loadThreadSince?(conversationId: string, afterSeq: number): Promise<readonly ChatMessage[] | ChatGapFillSincePage>;
  loadThread(conversationId: string, cursor?: undefined, options?: {
    limit?: number;
  }): Promise<ChatGapFillLatestPage>;
  loadChangesSince?(conversationId: string, sinceChangeSeq: number): Promise<ChatGapFillChangesPage>;
  getMessagesByIds?(conversationId: string, messageIds: readonly string[]): Promise<ChatGapFillMessagesPage>;
}

The two read methods gap-fill needs, injected by the caller (the store sends no requests and does not know the host). They are shaped like the host's messaging, so you can pass the host's messaging in directly.

Member Description
loadThreadSince Messages with seq above afterSeq, ascending. May return an array or a ChatGapFillSincePage; for an array, an empty one means there is nothing newer. Reject on failure: this gap-fill gives up quietly, and the next moment continues from the watermark. Without it there is no gap-fill.
loadThread The latest window (no cursor). Loaded when 10 rounds did not finish (appended when it connects to the range, swapped in when it does not), and swapped in when the change log says it is too old. With latestWindowOnGap it is also loaded while the window does not reach the latest, with undefined as the second argument and limit (when given) in the third; a window that comes back with newerCursor is dropped. Implementations taking a single argument still work.
loadChangesSince One page of the change log with numbers above sinceChangeSeq, like GET /conversations/{id}/changes. sinceChangeSeq is exclusive; 0 means from the first change. Catching up on changes happens only when both this and getMessagesByIds are given; one alone counts as neither.
getMessagesByIds The current version of messages by id, like POST /conversations/{id}/messages/batch-get. At most 100 distinct ids per call: the server counts its limit before deduplication, so the scheduler dedupes first and then splits into batches. Rejects on failure; when the error carries retryable: true, the scheduler backs off and sends the same batch again, at most 3 sends (see Catching up on changes).

ChatGapFillSincePage

interface ChatGapFillSincePage {
  messages: readonly ChatMessage[];
  hasNewer: boolean;
  headChangeSeq?: number;
}

A batch fetched forward from the watermark, the object form returned by the host's messaging.loadThreadSince. messages are those with seq above the requested afterSeq, ascending by seq; hasNewer can only come from the server; give true when unsure. headChangeSeq is the change number read in the same snapshot as the batch (head_change_seq of the list response); when it is below the change watermark, the change watermark moves back to it. Without it the change watermark stays.

ChatGapFillLatestPage

interface ChatGapFillLatestPage {
  messages: readonly ChatMessage[];
  nextCursor?: string;
  headChangeSeq?: number;
}

The latest window, as returned by the host's messaging.loadThread (no cursor). messages ascend by seq; nextCursor is given when there are older messages, and left out at the beginning. headChangeSeq is the change number read in the same snapshot as the window (head_change_seq of the list response): when this window is swapped in, it becomes the conversation's change watermark; without it there is no change watermark and the conversation does not catch up on changes.

ChatGapFillChange

interface ChatGapFillChange {
  changeSeq: number;
  messageId: string;
  messageSeq: number;
}

One row of the change log: an existing message in the conversation was changed. changeSeq is the change's number, rising per conversation; messageSeq is that message's seq, so changes older than the range held are not fetched.

ChatGapFillChangesPage

interface ChatGapFillChangesPage {
  changes: readonly ChatGapFillChange[];
  nextSinceChangeSeq: number;
  hasMore: boolean;
  headChangeSeq: number;
  tooOld: boolean;
}

One page of the change log, like the response of GET /conversations/{id}/changes.

Field Description
changes Changes numbered above the requested sinceChangeSeq, ascending; a message appears at most once per page (its last change). Empty when tooOld.
nextSinceChangeSeq The highest number scanned in this page; the next page starts after it. Equal to the requested sinceChangeSeq on an empty page. If hasMore is true but it did not move forward, this catch-up fails.
hasMore There is more.
headChangeSeq The newest number, read in the same snapshot as changes. After the catch-up the change watermark moves to this number of the last page.
tooOld The requested position has been purged (past retention), or is above the newest number (it came from elsewhere): no incremental fetch is possible; refetch the whole stretch.

ChatGapFillMessagesPage

interface ChatGapFillMessagesPage {
  messages: readonly ChatMessage[];
  missingIds: readonly string[];
  deferredIds: readonly string[];
  headChangeSeq: number;
}

A batch fetched by id, like the response of POST /conversations/{id}/messages/batch-get. messages are the current versions of the messages found (a recalled one is its tombstone), missingIds are ids that no longer exist (deleted, or not in this conversation), deferredIds are ids not returned this time that should be fetched again (beyond the per-response card cap), and headChangeSeq is the change number read in the same snapshot as messages. When a batch includes cards, the server returns only part of it each time and puts the rest in deferredIds; that is normal paging, and the scheduler keeps fetching for as many calls as it takes. Only when more than 10 calls in a row return nothing (no messages and no missingIds) does this catch-up fail.

ChatGapFillScheduler

interface ChatGapFillScheduler {
  open(conversationId: string): Promise<ChatGapFillResult>;
  close(conversationId: string): void;
  reconnected(): Promise<ChatGapFillResult[]>;
  foreground(): Promise<ChatGapFillResult[]>;
  catchUp(): Promise<ChatGapFillResult[]>;
  messages(conversationId: string): Promise<ChatGapFillResult | undefined>;
  sent(conversationId: string): Promise<ChatGapFillResult>;
  changes(conversationId: string): Promise<ChatGapFillResult | undefined>;
  dispose(): void;
}

No concurrency within a conversation: a trigger that arrives while a fill is running waits for it to finish and then fills once more from the new watermark; further triggers meanwhile join that one and get the same result. The returned promises never reject. Every gap-fill fetches new messages first and then catches up on changes (when the read methods have a change log).

Method Description
open(conversationId) Opening a conversation, or switching back to one opened before: mark it open and fill once.
close(conversationId) Switching away from or closing a conversation: reconnects, foregrounding and messages signals no longer fill it; a fill in flight still completes, except that a fetch by id waiting to be resent after a failure is not resent.
reconnected() The realtime connection reconnected: fill each open conversation once.
foreground() The page came back from the background: fill each open conversation once.
catchUp() The host asks for a catch-up although nothing in particular happened (a timer while the connection looks open, coming back online): fill each open conversation that is not filling right now, once. One already filling is skipped, not queued, and is not in the returned list. It does not mean there are new messages. Meant for a timer, it keeps to one request per conversation when nothing changed: when the fetch of new messages reaches the latest and its headChangeSeq is not above the change watermark, the change log is not read (changes is caughtUp with rounds 0).
messages(conversationId) The conversation has new messages (messages signal): fill once if it is open; if not, no fill and undefined, it is filled when switched back to.
sent(conversationId) A send succeeded but the server message was not handed back (sendSettled without message): fill once from the watermark, whether or not the conversation is open. This fill sends its request after the call, so once it reaches the latest (caughtUp, reloaded) the server message is in the range, and the local messages already confirmed but not matched at the time of the call are removed through the write entry point; when it does not reach the latest, they stay, and any later fill that reaches the latest (whatever triggered it) removes the local messages already confirmed but not matched when it started.
changes(conversationId) Realtime pushes skipped a change number (one above the change watermark plus one, with the missing ones not arrived): fill once if the conversation is open and the store still has the gap; otherwise do nothing and return undefined. Pushes are routinely out of order across messages, so wait 1–2 seconds after a push with a change number before calling it; if the missing numbers arrive meanwhile, nothing is fetched.
dispose() Stop: no more requests, and results of requests in flight are not written to the store. The fill in flight checks its request first: it gets disposed if the request succeeds, and still gets failed if the request fails. The open conversations are cleared as well, so afterwards open gets disposed right away, reconnected and foreground return [], and the messages signal returns undefined.

ChatGapFillResult

interface ChatGapFillResult {
  conversationId: string;
  outcome: ChatGapFillOutcome;
  rounds: number;
  error?: unknown;
  changes?: ChatGapFillChangesResult;
}

The result of one gap-fill of one conversation. rounds is how many times this gap-fill called loadThreadSince (not counting loading the latest window), at most 10; error is present only when outcome is failed, and is the reason the request rejected. outcome describes only the step that fetches new messages; the result of catching up on changes is in changes, present when the read methods have a change log and the scheduler has not been stopped.

ChatGapFillOutcome

type ChatGapFillOutcome =
  | "caughtUp"
  | "reloaded"
  | "skipped"
  | "interrupted"
  | "failed"
  | "disposed";
Value Description
caughtUp The server said there is nothing newer; the range reached the latest.
reloaded After 10 rounds the server still said there is more (or the range still stopped below the watermark); the latest window was loaded: appended after the range when it connects, swapped in whole when it does not. With latestWindowOnGap, also when the latest window loaded while the window did not reach the latest was appended after the range (rounds is 0).
skipped Nothing to fill: never loaded, closed (conversationClosed), no watermark yet, the window does not reach the latest, or no loadThreadSince. With latestWindowOnGap, also when the latest window loaded does not connect and is kept aside (or is not the latest window and is dropped).
interrupted Halfway through, the window was replaced by one that does not reach the latest (for example a jump to a message); the batch in flight (or the latest window) is not written, and that window stays as it is. With latestWindowOnGap, also when the conversation was closed, the window reached the latest, or the conversation was cleared while the latest window was being loaded.
failed The request failed; see error.
disposed The scheduler was stopped. A fill in flight when it stopped gets this only if its request succeeds; if the request fails it gets failed.

ChatGapFillChangesResult

interface ChatGapFillChangesResult {
  outcome: ChatGapFillChangesOutcome;
  rounds: number;
  error?: unknown;
}

The result of one catch-up on changes. rounds is how many rounds of the change log were fetched (each round from the change watermark at the time to the end, then by id), at most 3; error is present only when outcome is failed, and is the reason the request rejected (for a fetch by id that was resent, the last one's; or that the change log did not move forward, or more than 10 calls in a row by id returned nothing).

ChatGapFillChangesOutcome

type ChatGapFillChangesOutcome =
  | "caughtUp"
  | "reloaded"
  | "skipped"
  | "incomplete"
  | "interrupted"
  | "failed"
  | "disposed";
Value Description
caughtUp The change log was fetched to the end, the changed messages were fetched and written to the store, and the change watermark moved to the number fetched.
reloaded The change log said it was too old (or one round would take more than 10 pages): the stretch on the device was dropped and the latest window swapped in.
skipped Nothing to catch up on: never loaded, closed, or no change watermark (the window swapped in had no headChangeSeq).
incomplete Three rounds ran and pushes newer than the fetched copy kept arriving first; the change watermark stays at the last number fetched and waits for the next moment.
interrupted The window was replaced or the conversation closed meanwhile, so what came back is not written; or the change log said it was too old while the window does not reach the latest: only the stretch on the device was dropped, and the window you are looking at was not replaced.
failed A request failed; see error. A retryable fetch by id was already sent 3 times (or the backoff ended because the scheduler was disposed, the conversation closed, or a window was swapped in). The change watermark does not move.
disposed The scheduler has stopped.