When a client is being actively used, events will occur that affect the current user and that they must learn about as soon as possible, e.g. when a new message is received. To eliminate the need for the client itself to periodically download these events, there is an update delivery mechanism in which the server sends the user notifications over one of its available connections with the client.
Update events are sent to an authorized user into the last active connection (except for connections needed for downloading / uploading files).
So to start receiving updates the client needs to init connection and call API method, e.g. to fetch current state.
Make sure to always ignore updates received from unencrypted connections (i.e. before the handshake is completed).
If the connection is encrypted, but the session isn't logged in yet or was logged out, only the following updates may be handled:
All events are received from the socket as a sequence of TL-serialized Updates objects, which might be optionally gzip-compressed in the same way as responses to queries.
Each Updates object may contain single or multiple Update objects, representing different events happening.
In order to apply all updates in precise order and to guarantee that no update is missed or applied twice there is seq attribute in Updates constructors, and pts (with pts_count) or qts attributes in Update constructors. The client must use those attributes values in combination with locally stored state to correctly apply incoming updates.
When a gap in updates sequence occurs, it must be filled via calling one of the API methods. More below »
As said earlier, each payload with updates has a TL-type Updates. It can be seen from the schema below that this type has several constructors.
updatesTooLong#e317af7e = Updates;
updateShort#78d4dec1 update:Update date:int = Updates;
updateShortMessage#313bc7f8 flags:# out:flags.1?true mentioned:flags.4?true media_unread:flags.5?true silent:flags.13?true id:int user_id:long message:string pts:int pts_count:int date:int fwd_from:flags.2?MessageFwdHeader via_bot_id:flags.11?long reply_to:flags.3?MessageReplyHeader entities:flags.7?Vector<MessageEntity> ttl_period:flags.25?int = Updates;
updateShortChatMessage#4d6deea5 flags:# out:flags.1?true mentioned:flags.4?true media_unread:flags.5?true silent:flags.13?true id:int from_id:long chat_id:long message:string pts:int pts_count:int date:int fwd_from:flags.2?MessageFwdHeader via_bot_id:flags.11?long reply_to:flags.3?MessageReplyHeader entities:flags.7?Vector<MessageEntity> ttl_period:flags.25?int = Updates;
updateShortSentMessage#9015e101 flags:# out:flags.1?true id:int pts:int pts_count:int date:int media:flags.9?MessageMedia entities:flags.7?Vector<MessageEntity> ttl_period:flags.25?int = Updates;
updatesCombined#725b04c3 updates:Vector<Update> users:Vector<User> chats:Vector<Chat> date:int seq_start:int seq:int = Updates;
updates#74ae4240 updates:Vector<Update> users:Vector<User> chats:Vector<Chat> date:int seq:int = Updates;
updatesTooLong indicates that there are too many events pending to be pushed to the client, so one needs to fetch them manually.
Events inside updateShort constructors, normally, have lower priority and are broadcast to a large number of users, i.e. one of the chat participants started entering text in a big conversation (updateChatUserTyping).
The updateShortMessage, updateShortSentMessage and updateShortChatMessage constructors are redundant but help significantly reduce the transmitted message size for 90% of the updates. They should be transformed to updateShort upon receipt.
Two remaining constructors updates and updatesCombined are part of the Updates sequence. Both of them have the seq attribute, which indicates the remote Updates state after the generation of the Updates, and seq_start indicates the remote Updates state after the first of the Updates in the packet is generated. For updates, the seq_start attribute is omitted, because it is assumed that it is always equal to seq.
Each event related to a message box (message created, message edited, message deleted, etc) is identified by a unique auto-incremented pts, or qts in case of secret chat updates, certain bot updates, etc.
Each message box can be considered as some server-side DB table that stores messages and events associated with them. All boxes are completely independent, and each pts sequence is tied to just one box (see below).
The Update object may contain info about multiple events (for example, updateDeleteMessages).
That's why all single updates might have pts_count parameter indicating the number of events contained in the received update (with some exceptions, in this case, the pts_count is considered to be 0).
Each channel and supergroup has its message box and its event sequence as a result; private chats and basic groups of one user have another common event sequence.
Secret chats, certain bot events and other kinds of updates have yet another common secondary event sequence.
To recap, the client has to take care of the integrity of the following sequences to properly handle updates:
// Message constructors
message#95ef6f2b flags:# out:flags.1?true mentioned:flags.4?true media_unread:flags.5?true silent:flags.13?true post:flags.14?true from_scheduled:flags.18?true legacy:flags.19?true edit_hide:flags.21?true pinned:flags.24?true noforwards:flags.26?true invert_media:flags.27?true flags2:# offline:flags2.1?true video_processing_pending:flags2.4?true paid_suggested_post_stars:flags2.8?true paid_suggested_post_ton:flags2.9?true id:int from_id:flags.8?Peer from_boosts_applied:flags.29?int from_rank:flags2.12?string peer_id:Peer saved_peer_id:flags.28?Peer fwd_from:flags.2?MessageFwdHeader via_bot_id:flags.11?long via_business_bot_id:flags2.0?long guestchat_via_from:flags2.19?Peer reply_to:flags.3?MessageReplyHeader date:int message:string media:flags.9?MessageMedia reply_markup:flags.6?ReplyMarkup entities:flags.7?Vector<MessageEntity> views:flags.10?int forwards:flags.10?int replies:flags.23?MessageReplies edit_date:flags.15?int post_author:flags.16?string grouped_id:flags.17?long reactions:flags.20?MessageReactions restriction_reason:flags.22?Vector<RestrictionReason> ttl_period:flags.25?int quick_reply_shortcut_id:flags.30?int effect:flags2.2?long factcheck:flags2.3?FactCheck report_delivery_until_date:flags2.5?int paid_message_stars:flags2.6?long suggested_post:flags2.7?SuggestedPost schedule_repeat_period:flags2.10?int summary_from_language:flags2.11?string = Message;
messageService#7a800e0a flags:# out:flags.1?true mentioned:flags.4?true media_unread:flags.5?true reactions_are_possible:flags.9?true silent:flags.13?true post:flags.14?true legacy:flags.19?true id:int from_id:flags.8?Peer peer_id:Peer saved_peer_id:flags.28?Peer reply_to:flags.3?MessageReplyHeader date:int action:MessageAction reactions:flags.20?MessageReactions ttl_period:flags.25?int = Message;
messageEmpty#90a6ca84 flags:# id:int peer_id:flags.0?Peer = Message;
// Updates related to messages in the common message queue
updateNewMessage#1f2b0afd message:Message pts:int pts_count:int = Update;
updateEditMessage#e40370a3 message:Message pts:int pts_count:int = Update;
updateDeleteMessages#a20db0e5 messages:Vector<int> pts:int pts_count:int = Update;
// Updates related to channel/supergroup messages
updateNewChannelMessage#62ba04d9 message:Message pts:int pts_count:int = Update;
updateEditChannelMessage#1b3f4df7 message:Message pts:int pts_count:int = Update;
updateDeleteChannelMessages#c32d5b12 channel_id:long messages:Vector<int> pts:int pts_count:int = Update;
// Updates related to scheduled messages
updateNewScheduledMessage#39a51dfb message:Message = Update;
updateDeleteScheduledMessages#f2a71983 flags:# peer:Peer messages:Vector<int> sent_messages:flags.0?Vector<int> = Update;
// Updates related to quick-reply messages
updateQuickReplyMessage#3e050d0f message:Message = Update;
updateDeleteQuickReplyMessages#566fe7cd shortcut_id:int messages:Vector<int> = Update;
updateDeleteQuickReply#53e6f1ec shortcut_id:int = Update;
// Updates related to messages received via Telegram business connections
updateBotNewBusinessMessage#9ddb347c flags:# connection_id:string message:Message reply_to_message:flags.0?Message qts:int = Update;
updateBotEditBusinessMessage#7df587c flags:# connection_id:string message:Message reply_to_message:flags.0?Message qts:int = Update;
updateBotDeleteBusinessMessage#a02a982e connection_id:string peer:Peer messages:Vector<int> qts:int = Update;
updateBusinessBotCallbackQuery#1ea2fda7 flags:# query_id:long user_id:long connection_id:string message:Message reply_to_message:flags.2?Message chat_instance:long data:flags.0?bytes = Update;
// E2E message updates
updateNewEncryptedMessage#12bcbd9a message:EncryptedMessage qts:int = Update;
Among the many updates that can be received from the various message boxes », the most important updates are the ones related to messages.
Messages also have their own ID sequences (id field), independent from the pts/qts sequences of the message boxes that contain updates about them, and independent from each other:
Common message ID sequence.
Monotonically increasing, applies to all Message constructors located in private chats and basic groups.
The sequence is shared by all private chats and basic group messages within the current account, meaning that:
Here are the most important updates about messages belonging to the common message ID sequence:
Also note that the following updates may be received from users connected to a bot via a business connection »: in this case, messages located in private chats and basic groups will use the connected user's common message ID sequence, not the bot's common message ID sequence.
Channel/supergroup message ID sequences.
Monotonically increasing, applies to all Message constructors located in private chats and basic groups.
Each channel/supergroup has its own sequence.
Here are the most important updates about messages belonging to channel/supergroups:
Also note that the following updates may be received from users connected to a bot via a business connection »: in this case, messages located in channels/supergroups will use the same ID sequence used by the bot (and by everyone else on Telegram).
Secret chat message ID sequence.
Not monotonically increasing: each secret chat message (represented by a DecryptedMessage constructor, not the containing EncryptedMessage) is uniquely identified by its random_id, a completely random 64-bit long integer chosen by the sender.
Each secret chat has its own sequence (one sequence, shared for both ends of the chat); ordering of secret chat messages is guaranteed by the sequence number, not the message ID.
Secret chat message IDs are obviously the same for the two ends of the secret chat.
Updates about sent/received encrypted messages are contained in updateNewEncryptedMessage.
Scheduled message ID sequence.
Monotonically increasing, applies to all scheduled messages only while they are in the schedule queue; when sent, the message ID will change to an ID from the common message ID sequence or a channel/supergroup ID sequence.
Each channel, supergroup, basic group and private chat has its own sequence (unlike for normal messages, where basic groups and private chats share the same sequence).
Here are the most important updates about messages belonging to the schedule queue:
Quick reply shortcut message » ID sequence.
Monotonically increasing, applies to all quick reply shortcut messages located in quick reply shortcuts; when sent, the message will be duplicated, obtaining a new message ID that will be generated from the common message ID sequence or a channel/supergroup ID sequence.
Note that sending a quick reply shortcut will not delete messages from the shortcut: they will remain in the shortcut, keeping the same shortcut message ID.
All quick reply shortcut messages from all shortcuts share the same ID sequence.
Here are the most important updates about messages belonging to a quick reply shortcut:
Message IDs are used in many places across the API to point to messages, and only channel/supergroup message IDs can also be used to generate message deep links » pointing to the message.
Message IDs are also used to order messages in chats (except for secret chat messages, where the seqno is used instead).
Editing a message does not change its ID.
The common update state is represented by the updates.State constructor. When the user logs in for the first time, a call to updates.getState has to be made to store the latest update state (which will not be the absolute initial state, just the latest state at the current time). The common update state can also be fetched from updates.differenceTooLong.
The channel update state is represented simply by the pts of the event sequence: when first logging in, the initial channel state can be obtained from the dialog constructor when fetching dialogs, from the full channel info, or it can be received as an updateChannelTooLong update.
The secondary update state is represented by the qts of the secret event sequence, it is contained in the updates.State of the common update state.
The Updates sequence state is represented by the date and seq of the Updates sequence, it is contained in the updates.State of the common update state.
Update handling in Telegram clients consists of receiving events, making sure there were no gaps and no events were missed based on the locally stored state of the correspondent event sequence, and then updating the locally stored state based on the parameters received.
When the client receives payload with serialized updates, first of all, it needs to walk through all of the nested Update objects and check if they belong to any of message box sequences (have pts or qts parameters). Those updates need to be handled separately according to corresponding local state and new pts/qts values. Details below »
After message box updates are handled, if there are any other updates remaining the client needs to handle them with respect to seq. Details below »
pts: checking and applyingHere, local_pts will be the local state, pts will be the remote state, pts_count will be the number of events in the update.
local_pts + pts_count === pts, the update can be applied.local_pts + pts_count > pts, the update was already applied, and must be ignored.local_pts + pts_count < pts, there's an update gap that must be filled.For example, let's assume the client has the following local state for the channel 123456789:
local_pts = 131
Now let's assume an updateNewChannelMessage from channel 123456789 is received with pts = 132 and pts_count=1.
Since local_pts + pts_count === pts, the total number of events since the last stored state is, in fact, equal to pts_count: this means the update can be safely accepted and the remote pts applied:
local_pts = 132
Since:
pts indicates the server state after the new channel message events are generatedpts_count indicates the number of events in the new channel updatepts_before = pts - pts_count = 131, which is, in fact, equal to our local state.Now let's assume an updateNewChannelMessage from channel 123456789 is received with pts = 132 and pts_count=1.
Since local_pts + pts_count > pts (133 > 132), the update is skipped because we've already handled this update (in fact, our current local_pts was set by this same update, and it was resent twice due to network issues or other issues).
Now let's assume an updateDeleteChannelMessages from channel 123456789 is received with pts = 140 and pts_count=5.
Since local_pts + pts_count < pts (137 < 140), this means that updates were missed, and the gap must be recovered.
The whole process is very similar for secret chats and certain bot updates, but there is a qts instead of pts, and events are never grouped, so it's assumed that qts_count is always equal to 1.
seq: checking and applyingOn the top level when handling received updates and updatesCombined there are four possible cases:
seq_start === 0, the updates can be applied: this is a special case for updates that aren't ordered and should just be applied immediately.local_seq + 1 === seq_start, the updates can be applied.local_seq + 1 > seq_start, the updates were already applied, and must be ignored.local_seq + 1 < seq_start, there's an updates gap that must be filled (updates.getDifference must be used as with common and secret event sequences).If the updates were applied, local Updates state must be updated with seq (unless it's 0) and date from the constructor.
For all the other Updates type constructors there is no need to check seq or change a local state.
version: checking and applyingSome updates related to basic groups or group calls also have a version integer identifier, which should be used similarly to pts values to deduplicate/update outdated chat information as specified here (basic groups) » and here (group calls) ».
To do this, updates.getDifference (common/secret state) or updates.getChannelDifference (channel state) with the respective local states must be called.
Manually obtaining updates through the above methods is required in the following situations:
When calling updates.getDifference if the updates.differenceSlice constructor is returned in response, the full difference was too large to be received in one request. The intermediate status, intermediate_state, must be saved on the client and the query must be repeated, using the intermediate status as the current status.
To fetch the updates difference of a channel, updates.getChannelDifference is used.
If the difference is too large to be received in one request, the final flag of the result is not set (see docs).
The intermediate status, represented by the pts, must be saved on the client and the query must be repeated, using the intermediate status as the current status.
For performance reasons and for a better user experience, clients can specify the number of updates to be returned by each call during pagination using the pts_total_limit parameter of updates.getDifference and limit parameter for updates.getChannelDifference.
It is recommended to use a limit equal to 10-100 for channels and 1000-10000 otherwise.
As mentioned above, the specified limit does not limit the total number of updates that can be fetched with getChannelDifference/getDifference, it just limits the number of updates returned by each individual getDifference call while paginating through the message box, each time passing the pts/qts returned by the previous call until a final/non-slice result is returned, indicating no more updates are available.
Do not re-invoke updates.getChannelDifference if the returned difference is final, unless the user has opened the channel/supergroup ».
The various message boxes (the common message box, channel message boxes, etc) have a fixed size: the exact size of message boxes is a server-side implementation detail that clients should not rely on, but it is usually very large, 100000 for channels and 5000000 for the common message box.
Very old messages with a pts < (latestPts - size) will be deleted from a message box and will not be fetchable anymore using updates.getChannelDifference/updates.getDifference: invoking those methods with a pts < (latestPts - size) will return a updates.channelDifferenceTooLong/updates.differenceTooLong constructor, and clients have to handle this by re-fetching the latest state for that message box and re-start fetching updates from that state, and filling any gaps in older messages manually by using channels.getMessages and messages.getMessages, if necessary (i.e. if the user scrolls back far enough in the message/chat history).
Please note that the updatesTooLong/updateChannelTooLong updates, on the other hand, do not necessarily indicate the message box size limit was reached, they simply indicate that the number of queued updates for a message box is too large to be delivered passively through the socket, and updates.getDifference/updates.getChannelDifference can be invoked normally to fetch the difference.
Note the subtle difference between filling gaps normally using getChannelDifference/getDifference, and filling gaps for really old messages (< (latestPts - size)) using channels.getMessages/messages.getMessages:
With the normal gap filling logic with getChannelDifference/getDifference, it's enough to simply store the last known PTS, and request new updates by passing that PTS to getDifference, eventually paginating through updates until the client catches up.
If the client has a message database, it can be simply populated by the update handling logic (either through passive updates, or through the getDiff logic), applying updates one by one as they arrive (after reordering and deduplication by the usual update handling logic described above), simply storing messages to the DB, using message ID (common message box) or peer ID+message ID (channel message boxes, secret chats) as primary key.
With the normal gap filling logic, there's no need to worry if gaps in the message ID sequence are encountered in the local message database, as it's guaranteed that getDifference will deliver all relevant updates needed to fill the message database, and message ID gaps are caused (for example) by deleted messages, and must not be "filled".
With the edge case gap filling logic with channels.getMessages/messages.getMessages, suddenly message ID gaps encountered in the local message database become a problem: are they legitimate gaps caused by deleted messages that should be ignored, or gaps caused by an unrecoverable getDifference hole that must be recovered using channels.getMessages/messages.getMessages because they may contain actual, non-deleted messages?
One of the many ways to handle this is to keep track of known message ID ranges that do not contain gaps using a data structure like a segment tree, or any other data structure that allows to efficiently check if a message ID is contained by any range.
All messages belonging to private chats and basic groups » are stored in the one single instance of this datastructure (because they share the same id sequence).
Messages belonging to supergroup and channels are stored each in their own instance of the datastructure, one per channel/supergroup (as channels and supergroups each have their own independent id sequence).
For example, a somewhat naïve and inefficient implementation could store [start_msg_id, end_msg_id] message IDs of valid ranges in a simple list structure:
struct {
list: List<Pair[int, int]>,
pending: bool,
}
The structure is first initialized with an empty list and pending set to true.
Receiving a message via the socket or via getDifference/getChannelDifference will generate the following effects:
pending is equal to true, the first processed message will atomically append a new Pair{} to list (with both range elements equal to the message ID) and set pending to false.pending is equal to true, incoming messages will atomically update the end_msg_id of the last Pair in list (setting it to the message ID).pending is set to true, the update state (pts) is refetched from scratch, and getChannelDifference/getDifference is re-invoked with the new pts.When scrolling back in the message history or viewing context around individual messages, always keep track of the range of the messages being displayed, and if a gap is detected (either because there are no more ranges, or because the range changed), fill it by invoking:
Messages returned by those methods should be appropriately loaded into the message database and update existing ranges (or create new ones, if the returned range does not touch other existing ranges).
These methods will return placeholder messageEmpty constructors for deleted or otherwise non-representable messages, so that the entire fetched range is returned, in one way or another.
Messages obtained via other methods (meaning, not just those that return Updates, piped into the usual update deduplication/handling logic) may also extend/generate ranges.
The API will usually send passive updates (i.e. as standalone Updates constructors in the socket) for channels/supergroups the user/bot is a member of.
However, in some cases the API may stop sending updates (or send fewer updates) for some channels/supergroups: thus clients (user accounts only) should also additionally invoke updates.getChannelDifference periodically for channels and supergroups the user is currently viewing (i.e. explicitly opened channels/supergroups in one or more tabs/windows).
If the returned difference is non-final, the method should be called immediately with the new parameters as usual.
If the returned difference is final, and the user is still viewing the messages of the supergroup/channel (i.e. via distinct tabs/windows), updates.getChannelDifference should be re-invoked after timeout seconds (if the flag is specified, otherwise after 1 second).
This mechanism may also be used to enable passive reception of updates from channels or supergroups we're not a member of: if the specified channel or supergroup is public, or is private but temporarily available for a limited time thanks to a chatInvitePeek, the API will start passively sending updates (i.e. as standalone Updates constructors in the socket, as is already the case for normal channels/supergroups we've already joined) to all logged-in sessions, as long as any of the sessions continues to periodically invoke updates.getChannelDifference every timeout seconds (returned by the method, or every second if the timeout flag is absent from the return value of the method, or immediately with the new parameters if the returned difference is non-final).
Clients should stop updates.getChannelDifference polling once the user closes the channel/supergroup: the API may continue emitting passive updates only if the user is a member of the channel/supergroup.
Clients should also limit to 10 the maximum number of channels/supergroups short-polled using the above mechanism (i.e. if the user opens 11 windows on 11 different channels, short-poll with updates.getChannelDifference only the first 10).
Implementations also have to take care to postpone updates received via the socket while filling gaps in the event and Update sequences, as well as avoid filling gaps in the same sequence.
Example implementations: tdlib, MadelineProto.
An interesting and easy way this can be implemented, instead of using various locks, is by running background loops, like in MadelineProto ».
If a client does not have an active connection at the time of an event, PUSH Notifications will also be useful.