Skip to main content

Update recovery

The plain dispatcher delivers updates as they arrive, but Telegram's update stream is stateful: each update carries a sequence number, and a client that was briefly offline can miss updates or receive them out of order. Recovering those gaps means tracking state and calling updates.getDifference / updates.getChannelDifference at the right moments.

The updates package — the "gap engine" — does this for you.

Wiring it up​

The manager sits between the client and your dispatcher. It needs to both receive updates (as the UpdateHandler) and observe RPC responses (as a middleware), so it is installed in two places:

import (
"github.com/gotd/td/telegram"
"github.com/gotd/td/telegram/updates"
updhook "github.com/gotd/td/telegram/updates/hook"
"github.com/gotd/td/tg"
)

d := tg.NewUpdateDispatcher()
gaps := updates.New(updates.Config{
Handler: d, // your dispatcher receives recovered, ordered updates
Logger: logzap.New(log.Named("gaps")),
})

client, err := telegram.ClientFromEnvironment(telegram.Options{
UpdateHandler: gaps,
Middlewares: []telegram.Middleware{
updhook.UpdateHook(gaps.Handle),
// Keep local pts in sync after self-initiated reads/deletes.
updhook.AffectedHook(gaps),
},
})

Self-initiated reads and deletes​

UpdateHook forwards tg.UpdatesClass results to the manager, but some methods return a different shape. messages.readHistory, messages.deleteMessages, channels.deleteMessages, messages.readMentions and messages.deleteHistory return messages.affectedMessages / messages.affectedHistory — results that carry a pts increment the client must apply, yet are not updates and so never reach the gap engine.

If you skip them, the server's pts advances while your local pts stays behind, so the next genuine update looks like a gap and is buffered. A common symptom: you read or delete a message in a handler, and a later edit in the same chat is then postponed until some unrelated pts-changing update arrives (gotd/td#1382).

updhook.AffectedHook closes this gap. It inspects each RPC result, and when it sees an affected-pts result it calls Manager.HandleAffected to advance the local pts (releasing any updates postponed behind it), routing channel methods to the right channel sequence and everything else to the common one. It is opt-in, mirrors TDLib's behavior, and is a no-op before gaps.Run — so install it alongside UpdateHook as shown above. See gotd/td#1782.

Running the engine​

After authenticating, start the engine with your user ID. It blocks, keeping the client alive and processing updates until the context is cancelled:

return client.Run(ctx, func(ctx context.Context) error {
// ... authenticate ...
self, err := client.Self(ctx)
if err != nil {
return err
}
return gaps.Run(ctx, client.API(), self.ID, updates.AuthOptions{
IsBot: self.Bot,
OnStart: func(ctx context.Context) {
log.Info("recovery started")
},
})
})

Your dispatcher handlers (OnNewMessage, OnNewChannelMessage, …) are registered exactly as before — the difference is they now receive a gap-free, correctly ordered stream.

Persisting state​

By default the engine keeps state in memory, so a restart re-syncs from scratch. For a long-running service, give it a Storage and access-hash storage so it resumes from where it left off: without stored channel access hashes it can't fetch channel differences after a restart. The gotd/contrib module provides bbolt- and pebble-backed implementations:

import "github.com/gotd/contrib/pebble"

gaps := updates.New(updates.Config{
Handler: d,
Storage: pebble.NewStateStorage(db),
AccessHasher: pebble.NewAccessHasher(db),
UserAccessHasher: pebble.NewAccessHasher(db),
})

The bbolt equivalents are bbolt.NewStateStorage(db) and bbolt.NewAccessHasher(db).

Tracking channels​

The engine tracks channels with a stored state, and learns about other channels from their next update. To track channels joined while the client was offline (or reported via OnLoadChannelStateFailed), pass them with the pts of their dialog. Stored pts take precedence, so missed updates are fetched:

gaps.Run(ctx, api, self.ID, updates.AuthOptions{
OnStart: func(ctx context.Context) {
for _, d := range dialogs { // e.g. from messages.getDialogs
_ = gaps.TrackChannel(ctx, d.ChannelID, d.AccessHash, d.Pts)
}
},
})

Updates before Run​

Updates can arrive before the manager's Run has loaded the state, e.g. while the client connects. By default they are handled right away (BeforeRunPass). To handle them in order once the state is loaded, buffer them, or drop them and let gap recovery fetch them again:

updates.Config{
BeforeRun: updates.BeforeRunBuffer, // or updates.BeforeRunDrop
BeforeRunBufferSize: 100, // batches; extra ones are dropped
}

Channel gaps​

When a channel gap is too long to recover (updates.channelDifferenceTooLong), the engine moves to the new state and reports the gap with the latest messages returned by the server. They are not passed to your handler, because they may include messages it has already received. Backfill what you missed with messages.getHistory:

updates.Config{
OnChannelGap: func(gap updates.ChannelGap) {
// gap.ChannelID, gap.AccessHash, gap.LocalPts, gap.RemotePts, gap.TopMessage
// gap.Messages are the latest messages, with gap.Users and gap.Chats.
},
}

The userbot example uses persistent state; the updates example is a minimal in-memory version.

Known limits

The engine relies on the server for getDifference correctness, and messages missed in a ChannelDifferenceTooLong gap must be fetched by the application (see Channel gaps). Stateless updates can't be ordered. These are inherent to the MTProto update model.