Skip to main content

gotd v0.164.0: built for long-running clients

· 9 min read
Aleksandr Razumov
Aleksandr Razumov
TeamLead @ VK

gotd v0.164.0 and gotd/contrib v0.26.0 are about clients that run for weeks: archivers, userbots and services with many accounts in one process. Most changes came from running gotd in such an archiver. They close gaps where gotd went quiet, waited forever or lost data, and add the metrics needed to see that it works. Many fixes follow what Telegram Desktop and TDLib do; we checked them against both, and against the live Telegram servers.

Highlights​

Start, Stop and Wait​

A client no longer has to live inside a Run callback. Start connects and returns once the client is ready; Wait blocks until it stops and tells you why; Stop, Shutdown and Close end it:

client := telegram.NewClient(appID, appHash, opts)
if err := client.Start(ctx, telegram.WithLifetime(ctx)); err != nil {
	return err
}
defer func() { _ = client.Close() }()

return client.Wait(context.Background())

telegram.Connect creates and starts a client in one call. Run works as before. See Running without Run.

Bots in one call​

The new telegram/bot package starts a bot with everything a long-running bot needs: the session and the update state in a directory, so updates sent while it was stopped are delivered after a restart; a check that the stored session belongs to the token's bot; flood waits on sign in waited out; and a peer manager fed from every update.

b, err := bot.Start(ctx, appID, appHash, token, bot.Options{Dir: "state", Handler: d})

auth.ParseBotToken validates tokens with the Bot API server's rules, without putting the secret in errors, and updates.FileStorage keeps the update state in a JSON file. See Bot authentication and the echo bot tutorial.

Know when something is wrong​

  • Lost authorization. OnAuthLost is called once when an authorized session is terminated (AUTH_KEY_UNREGISTERED, SESSION_REVOKED and similar). Return an error to stop Run with *telegram.AuthLostError, so one account can be stopped and reported while others keep running.
  • Banned, deleted and frozen accounts. AuthLostError.Kind() tells a ban or a deleted account from a terminated session. AUTH_KEY_DROP_* service notifications are reported through OnAuthLost. OnFrozen and Client.FreezeState report a frozen account, checked through the app config as the official apps do.
  • Connection events with a DC. OnConnectionStateEvent and OnConnDead say which DC and which kind of connection (primary, data, CDN) an event is about.
  • Stuck requests. RequestTimeout sets a per-request deadline. When it passes, the client reconnects and resends the request once, like Telegram Desktop. New PingInterval and PingTimeout options control liveness checks.
  • Metrics and logs. Pass a MeterProvider to the client and the updates manager for OpenTelemetry metrics of connections, requests, pools, update recovery and downloads, with debug logs at each event. Nothing is recorded unless you set it. See Observability.

Connections that recover​

  • Lost connection state. After the server drops a connection's state (seen when an account signs in on another device), requests failed with CONNECTION_NOT_INITED or CONNECTION_LAYER_INVALID until a restart. The client now sends initConnection again with the failed request.
  • Slow answers. A request the server didn't acknowledge after all resends failed with "retry limit reached", and its answer was dropped when it came. On a busy DC, answers after 30–60 seconds happen. The client now keeps waiting for the answer, bounded by your context, as TDLib and Telegram Desktop do.
  • Misconfiguration. API_ID_INVALID and other permanent errors of the connection setup made the client reconnect every few seconds forever. Run now returns *telegram.ConfigError.
  • Key regeneration. After the server forgot the auth key, the new key reused the old session's state; it now starts a clean session and resends pending requests in it.
  • Acks. Messages that arrived inside a container were never acknowledged, so the server kept them; they are now acked like top-level ones.
  • DC transfer. Connecting to another DC could hang when its key had to be recreated.

Updates that don't go missing​

  • Difference contents. Updates returned inside getDifference and getChannelDifference (other_updates: edits, deletions, read marks…) went through the gap check and could be dropped. They are now applied in TDLib's order and passed to your handler. This includes fixes and tests by @igitur and @vzd3v from #1854, #1864, #1866 and #1867 — thank you.
  • Errors in handlers. With StopOnApplyError, an error from your handler or from the state storage stops Run instead of being logged, and the state isn't advanced past the failed updates, so they are delivered again after a restart.
  • Messages returned with channelDifferenceTooLong were dropped. They are now passed to OnChannelGap together with the gap range. They're not sent to the handler, since they may include messages it already received.
  • Manager.TrackChannel tracks channels joined while offline, and channels reported by OnLoadChannelStateFailed.
  • BeforeRun decides what happens to updates that arrive before the manager starts: pass them, buffer them, or drop them for recovery to fetch again.
  • UpdateDispatcher handlers registered on one copy are now seen by every copy, so OnFallback is no longer lost when the dispatcher is passed by value. The new OnAny* handlers cover private chats, groups and channels in one place, including forum topics. The OnBefore hook sees every update.

Downloads for archivers​

  • Resume with WithOffset, a known WithSize and WithProgress. Builder.Verify checks a local file against the server's hashes; it no longer fails on files whose size is a multiple of the hash window.
  • WithRefresh gets a fresh location when a file reference expires. The new fileref package builds one from a message, a story or a profile photo.
  • Scheduler is an adaptive window of part requests shared between downloads, like Telegram Desktop's. It grows on success and halves on flood waits. Set Options.DownloadScheduler to get one per DC.
  • Client.FileDC gives a cached connection pool for a DC and follows FILE_MIGRATE. Client.DC to the home DC no longer fails with DC_ID_INVALID. CDN downloads now work through any invoker (DownloadVia).

Queries​

  • Iterators request full server pages by default (100 messages or dialogs, 200 participants) and no longer stop early on a short page.
  • MinID/MaxID on history iterators.
  • Resumable cursors for messages and dialogs.
  • ForEachFolder over all dialog folders.
  • GetMessagesWithEntities.
  • A new query/stories package.
  • DialogKey.InputPeer().
  • ExtractPeer no longer builds unusable peers from min users. Min users and channels are merged into full ones the way TDLib does it.

Wire traces​

Options.WireTracer sees every MTProto message after decryption and every connection event, on all connections of a client. The wiretrace package and the cmd/wiretrace tool decode such traces and the ones written by a traced Telegram Desktop build, and compare them: we used it to check gotd's traffic against Telegram Desktop's, step by step. See Debugging.

Also​

  • Layer 233.
  • HTML widget messages for bots: message.Widget, shared widgets and widget queries. See Sending messages.
  • QR login completes 2FA through Options.Password, keeps a scan that arrives while the code is being shown, retries token refresh errors, and completes the login on the account's DC when it's another one.
  • Message entities are sorted like TDLib does; the old order was wrong for some inputs.
  • Takeout sessions can be resumed with takeout.NewClient and handle TAKEOUT_INIT_DELAY.
  • tg.EntitiesFrom(users, chats).

gotd/contrib v0.26.0​

  • Storage: persistent updates state and access hashes for pebble, bbolt and redis. pebble and bbolt also implement peers.Storage for telegram/peers.
  • Fewer writes: pebble honours its write options and writes a whole update's peers in one batch instead of one fsync per peer. Unchanged peers aren't rewritten.
  • Encoding: a compact binary peer encoding, opt-in with WithBinaryEncoding(). Versions before v0.26.0 can't read it, so turn it on only once you won't roll back.
  • Min users: they no longer overwrite stored full users.
  • floodwait gains:
    • concurrent dispatch, so a slow request no longer blocks the others;
    • adaptive pacing, policies, exempt methods and retries of 5xx errors;
    • a snapshot for metrics.
  • Fixes: pebble skips only the peers it can't encode in a batch, bbolt iterates channels in a read-only transaction, and floodwait uses its clock everywhere and decays the wait after errors.

Upgrading​

Most changes are additions. Check these:

  • Iterators use the server's page size by default (it was 1 for some generated iterators) and read until the last page. Set BatchSize if you relied on the old values.
  • Updates: updates inside differences now reach your handler, so you may see edits, deletions and read marks you didn't get before. An updatePtsChanged inside a difference makes the manager fetch the difference once more.
  • Run returns *telegram.ConfigError on API_ID_INVALID, API_ID_PUBLISHED_FLOOD and other permanent errors of the connection setup, where it used to reconnect forever.
  • Slow requests are no longer failed with "retry limit reached": rpc.Engine.Do doesn't return RetryLimitReachedErr any more, and waits for the answer until your context is done. Use a context deadline or RequestTimeout to bound a request.
  • ExtractPeer returns peer.ErrMinPeer for min users and channels, where it used to build an input peer from the min access hash. The iterators handle this. Check your own callers.
  • After Run stops, updates.Manager.Handle and TrackChannel return updates.ErrStopped instead of blocking.
  • Start and Run are exclusive, and a client can be started once.
  • contrib storage:
    • storage.Peer.FromUser returns false for min users;
    • CreatedAt now means "first stored with this content";
    • pebble writes without sync if you passed NoSync write options, which used to be ignored.
  • contrib floodwait now runs requests concurrently; it used to send one per tick.
  • contrib/bg still works, but Client.Start replaces it.

Thanks to everyone who reported issues in @gotd_en and @gotd_ru. Full list of changes: gotd/td, gotd/contrib.