File references

File references are strings of bytes that can be encountered in the file_reference fields of document and photo objects.

They must be cached by the client, along with the source where the document/photo object was found, in order to be refetched when the file reference expires.

Example implementation of a reference database: MadelineProto, android, telegram desktop, tdlib.

Automatic implementation

Implementation and maintenance of the file reference database may be fully automated by using the file reference database map file.

To start, set up the following two tables:

Then, download and implement the map file ».

File reference table

The file reference table can be represented by the following type:

HashMap<FileId, bytes>

This table maps a file ID » (which uniquely identifies a photo or a document) to a file reference ».

This table is populated by the incoming traversers » contained in the map file ».

The file reference can alternatively be stored, for example, in the document and photo DBs along with other metadata, or in any other way that uniquely associates a FileId to a file reference.

File source table

HashMap<FileId, Vector<FileSource>>

It maps a file ID » to one or more file sources ».

This table is populated by the incoming traversers » contained in the map file ».

To reduce the size of the file source table, it's possible to only allow storing one file source per source type per file ID (max N file sources per file ID, for N file source types), or even just one file source per file ID (max 1 source per file ID).

Map file

Download latest JSON file reference map for the current layer » - The constructor predicate is serialized under the _ key, bytes fields are serialized as {"_": "bytes", "bytes": "<base64 encoded value>"}

Download latest serialized TL version of the file reference map for the current layer »

See here » for the full TL schema of the map file.

Subset of the schema: 

fileReferenceMap#72a99250 layer:int db_schema:string db_schema_json:string traversers_incoming:Vector<TraverserIncoming> traversers_outgoing:Vector<TraverserOutgoing> refresh_actions:Vector<RefreshAction> skipped_incoming_sources:Vector<SkippedSource> = FileReferenceMap;

skippedSource#1d53cd15 flags:# predicate:string is_constructor:flags.0?true why:string = SkippedSource;

The map file is composed of a single fileReferenceMap constructor, which contains:

  • In the layer field, the layer version from which the map file was generated.

  • db_schema and db_schema_json fields with the TL database schema of FileIds and FileSources.
    db_schema is text TL schema, db_schema_json is the same TL schema in the JSON format also used for the API schema ».

    This DB schema is used for:

    These TL schemas can be used as a guideline for the file reference and file source table DBMS schema, or by directly serializing file IDs and file source vectors if a simple string => string KV DB is used.

    See here » for the current TL schema of the file reference database.

  • traversers_incoming contains incoming traversers », used to extract and store file references » from incoming media constructors.

  • traversers_outgoing contains outgoing traversers », used to swap expired file references » from outgoing media method calls.

  • refresh_actions contains refresh actions », instructions used to refresh expired file references according to file sources » stored in the file source table.

  • skipped_incoming_sources contains skipped sources.
    These constructors indicate that the source should be ignored completely (including during codegen): skippedSource sources are used internally to make sure all file reference paths are still covered in some way during validation, including paths for ephemeral media like inline results, or for media without any associated source (for example, media uploaded using messages.uploadMedia but not yet sent anywhere obviously does not have any associated source).

    • skippedSource.predicate - Indicates the name of the ignored constructor or method.
    • skippedSource.is_constructor - If set, predicate points to a constructor, otherwise it points to a method.
    • skippedSource.why - A human-readable reason as to why this source should be ignored.

    Each skipped predicate has exactly one associated reason.

The map file is automatically generated and validated by the file reference generator tool »; this tool can be manually run on newer or experimental layers, to generate the file reference database map file for a given API schema: see here » for more info.

File IDs

A file ID (in the context of the file reference DB) is an object that, when serialized to a string, can easily and uniquely identify a media file.

Currently, the following file ID objects are available (taken from the DB schema inside of the map file):

fileIdPhoto#47a0bd49 id:long = FileId;
fileIdDocument#461b1d89 id:long = FileId;

This TL schema can be used as a guideline for the file reference and file source table DBMS schema, or by directly serializing file IDs and file source vectors if a simple string => string KV DB is used.

Outgoing traversers » and incoming traversers » specify how to generate a file ID object from all media objects.

File references

A file reference is a string of bytes that can be encountered in the file_reference fields of

  • Incoming media constructors, such as document and photo objects.
  • Outgoing media constructors like inputDocument, inputPhoto, etc, where it must be populated by the client in order to download or resend all media.

A file reference may expire, in which case it cannot be used in outgoing constructors: it must be refreshed by refetching the message, story, etc where the media last appeared, as specified below.

File sources

A file source contains information that the client may use to re-fetch a media (and a new file reference for it), by invoking a specific action.

Each source type is associated to a specific refresh action ».

Incoming traversers » and source extraction logic are used together to populate the file source table », which maps a file ID to one or more file sources.

The map file » contains all sources needed to implement the file reference database, including some skipped sources which are not handled by any extractor.

This is the full list of valid sources contained by the current map file: 

fileSourceMessage#7e015bb0 flags:# from_scheduled:flags.0?true quick_reply_shortcut_id:flags.1?int peer:long id:int = FileSource;
fileSourceStarsTransaction#c1bac8c7 flags:# peer:long id:string refund:flags.0?true ton:flags.1?true = FileSource;
fileSourceStory#c820e3eb id:int peer:long = FileSource;
fileSourceWebPage#9e5b749c url:string = FileSource;
fileSourceBotApp#01cf8b7a id:long access_hash:long = FileSource;
fileSourceUserFull#70fdb7b0 id:long = FileSource;
fileSourceAdminLog#4797f959 channel:long max_id:long = FileSource;
fileSourceStoryAlbum#5e01f223 peer:long = FileSource;
fileSourceBotPreviewMedia#0aa91441 bot:long = FileSource;
fileSourceBotPreviewInfo#f9d2d6fc bot:long lang_code:string = FileSource;
fileSourcePaidMedia#b18d9042 id:int peer:long = FileSource;
fileSourceSavedMusic#dd1a7664 user_id:long id:long access_hash:long = FileSource;
fileSourceChatFull#9de75fde chat_id:long = FileSource;
fileSourceChannelFull#6fe19339 channel:long = FileSource;
fileSourcePremiumPromo#c907a44f = FileSource;
fileSourceAttachMenuBot#c3002694 bot:long = FileSource;
fileSourceTheme#92d05e0c id:long access_hash:long = FileSource;
fileSourceWallPaper#50dbf2f7 id:long access_hash:long = FileSource;
fileSourceStickerSet#34c73709 stickerset:InputStickerSet = FileSource;
fileSourceSavedGifs#13e78e07 = FileSource;
fileSourceSavedRingtones#2b25ef1b = FileSource;
fileSourceAvailableEffects#eb8578f0 = FileSource;
fileSourceAvailableReactions#0e432388 = FileSource;
fileSourceUserProfilePhoto#e39ee274 user_id:long max_id:long = FileSource;
fileSourceDocumentByHash#0f151e0f sha256:bytes size:long mime_type:string = FileSource;

See here » for the refresh action associated to each file source type.

Incoming traversers

See here » for the full list of incoming traversers contained by the current map file.

Schema:

traverseMethodResult#fda9f236 name:string push_sources:Vector<Source> is_needed_parent:Bool = TraverserIncoming;
traverseIncomingConstructor#6e4d00b6 type:string predicate:string params:Vector<TraverseParam> push_sources:Vector<Source> is_needed_parent:Bool = TraverserIncoming;
traverseCommitSourceLocation#94cc59c5 type:string predicate:string push_sources:Vector<Source> stored_constructor:string = TraverserIncoming;

traverseParam#62161b67 flags:# name:string is_flag:flags.0?true is_vector:flags.1?true type:string = TraverseParam;

source#dd27c696 flags:# predicate:string is_constructor:flags.0?true stored_constructor:string stored_params:Vector<FieldExtractor> skipped_flags:Vector<string> needs_parent:flags.3?string parent_is_constructor:flags.4?true = Source;

Incoming traversers traverse constructors received from the API, and use the gathered information to:

There are three traverser types.

All three traverser types have an optional push_sources field, which may contain a list of sources to push to the source stack when the traverser is triggered: these sources will be used to store file sources to the file source table » when a traverseCommitSourceLocation is reached.

The only allowed entry points for traversers are:

  • The results of all methods covered by a traverseMethodResult traverser.

  • Update constructors covered by a traverseCommitSourceLocation traversers.

    Note: All Updates constructors must be converted to a vector of Update constructors, including short variants like updateShortMessage, updateShortSentMessage, updateShortChatMessage, which must be pre-converted to Update constructors by the client using information extracted from the method call (not included in the map file for simplicity).

    The map file does not contain traversers for Updates returned by methods, as those are supposed to be handled by the update system: Update constructors should only be traversed after they are properly extracted, deduplicated and reordered by the update system; traversal can be triggered directly from the update system.

Example implementation

Use the following runtime structures:

  • HashMap<predicate, Vector<object>> parentContext for is_needed_parent/needs_parent checks.
  • Vector<FileSource> sourceStack for the source stack.

parentContext controls eligibility during source extraction » with needs_parent; sourceStack provides values to commit at traverseCommitSourceLocation.

Execution order at runtime:

  1. Start from an allowed entry point (traverseMethodResult or Update).
  2. Apply traversers recursively as follows:
    2.1. Populate and push valid sources from push_sources to sourceStack.
    A source is valid and can be pushed if source extraction » can be populated according to stored_params (all required flag fields are set and all paths can be correctly extracted), and if parentContext contains an entry for source.needs_parent (if set).
    2.2. If is_needed_parent is set, push the current constructor/method to parentContext, using its predicate as key.
    2.3. Recurse into children as specified by params, or into the method result for traverseMethodResult.
    2.4. If is_needed_parent is set, pop the value that was pushed in step 2.2.
    2.5. Pop only the sources that were pushed in step 2.1 (keep all other sources unchanged).
  3. When traverseCommitSourceLocation matches a constructor:
    3.1. Push valid sources from push_sources to sourceStack (some traverseCommitSourceLocation constructors can also contain sources).
    3.2. Commit FileId -> file_reference to the file reference table » and FileId -> FileSource to the file source table » for all currently active sources on the source stack.
    3.3. Pop only the sources that were pushed in step 3.1 (keep all other sources unchanged).
  4. Continue traversal to collect all matches in the same payload.

In other words, incoming traversal is a DFS-like walk where sources are pushed as context and committed only when a media location constructor is reached.

Traverser type list

  • traverseMethodResult - For method calls: attaches sources using method call parameters, then traverses on the method result.

    Only traverse the method result if the returned constructor has a matching traverseIncomingConstructor.

    Params:

    • name - The name of the method call to attach to (for example, messages.getMessages).
    • push_sources - A list of sources to push when the traverser is triggered.
    • is_needed_parent - If set, the traversal must expose this method result as parent context for downstream source extraction, because at least one downstream Source has needs_parent pointing to this method. Push the method call to a separate parent context map, keyed by method name (can be the same context map used for constructors with is_needed_parent=true, for example, HashMap<predicate, Vector<object>>).

    The map file » contains traverseMethodResults for all methods that return file references.

  • traverseIncomingConstructor - Attaches sources on constructors with a specific predicate (name), then traverses on some of the constructor's parameters.

    Params:

    • type - The type of the constructor to traverse (for example, Document, Photo, etc, present to simplify codegen).

    • predicate - The name of the constructor to traverse (for example, document, photo, etc).

    • push_sources - A list of sources to push when the traverser is triggered.

    • is_needed_parent - If set, the traversal must expose this constructor as parent context for downstream source extraction, because at least one downstream Source has needs_parent pointing to this constructor. Push this object to a separate parent context map, keyed by predicate (for example, HashMap<predicate, Vector<object>>).

    • params - A list of parameters to traverse.
      Only traverse the parameter if the contained constructor has a matching traverseIncomingConstructor.

      traverseParam contains the name of the parameter to traverse and (for simplicity, even if this info can be extracted from the schema) the parameter's type, and whether it's a flag, a vector, or a flagged vector.
      The type will always contain the type of the underlying parameter (i.e. without Vector<> or flags.X?).

    The map file » contains traverseIncomingConstructors for all incoming constructors that contain file references (nested within parameters).

  • traverseCommitSourceLocation - Stores the file reference contained in the current constructor and file sources currently on the source stack to the file source table » and file reference table ».

    Params:

    • type - The type of the constructor in predicate (for example, Document, Photo, etc, present to simplify codegen).
    • predicate - The name of the incoming media constructor received from the API (for example, document, photo). This constructor is guaranteed to have an id field and a file_reference field.
    • push_sources - A list of sources to push when the traverser is triggered, before any sources are stored to the file source table.
    • stored_constructor - The type of the file ID that will be used to populate the file reference table » and file source table » (for example, fileIdDocument, fileIdPhoto).

    When a media constructor of predicate traverseCommitSourceLocation.predicate is received, the client must:

    • Extract the file_reference field of the media constructor
    • Extract the id field of the media constructor, generating a new file ID of type traverseCommitSourceLocation.stored_constructor
    • Store both in the file reference table », using the file ID as key and the file reference as value.
    • Store all file sources currently on the source stack into the file source table », using the file ID as key and the file source as value (multiple values can be stored for the same key).

    For example:

    • Assume a document{id: 12345, file_reference: "...", ...} is received from the API; two file sources are currently present on the source stack when the document is traversed, one fileSourceMessage{...} and one fileSourceWebPage{...}.
    • A new fileIdDocument{id: 12345} => "..." entry must be added to the file reference table »
    • Two new fileIdDocument{id: 12345} => fileSourceMessage{...} and fileIdDocument{id: 12345} => fileSourceWebPage{...} entries must be added to the file source table ».

    The map file » contains traverseCommitSourceLocations for all incoming constructors that have a file_reference and an id field.
    The current map file has the following traverseCommitSourceLocations:

Outgoing traversers

See here » for the full list of outgoing traversers contained by the current map file.

Schema:

traverseMethodCall#9244f5bb name:string params:Vector<TraverseParam> = TraverserOutgoing;
traverseOutgoingConstructor#3fc5026d type:string predicate:string params:Vector<TraverseParam> = TraverserOutgoing;
traverseSwapLocation#032bf95d type:string predicate:string stored_constructor:string = TraverserOutgoing;

traverseParam#62161b67 flags:# name:string is_flag:flags.0?true is_vector:flags.1?true type:string = TraverseParam;

Outgoing traversers extract a file ID » from outgoing media objects sent to the API and use it to update and swap the file reference » within that same outgoing media object before it is sent.

There are three traverser types:

  • traverseMethodCall - Traverses the specified method call parameters.

    Params:

    • name - The name of the method whose parameters should be traversed.

    • params - The parameters to traverse.

      traverseParam contains the name of the parameter to traverse and (for simplicity, even if this info can be extracted from the schema) the parameter's type, and whether it's a flag, a vector, or a flagged vector.
      The type will always contain the type of the underlying parameter (i.e. without Vector<> or flags.X?).

  • traverseOutgoingConstructor - Traverses the specified constructor parameters.

    Params:

    • type - The type of the constructor to traverse (for example, InputMedia).

    • predicate - The name of the constructor to traverse (for example, inputMediaDocument, inputMediaPhoto, etc).

    • params - The parameters to traverse.

      traverseParam contains the name of the parameter to traverse and (for simplicity, even if this info can be extracted from the schema) the parameter's type, and whether it's a flag, a vector, or a flagged vector.
      The type will always contain the type of the underlying parameter (i.e. without Vector<> or flags.X?).

  • traverseSwapLocation - Refreshes the file reference contained in the current constructor using the file sources stored in the file source table ».

    Params:

    • type - The type of the outgoing media constructor sent to the API (for example, InputMedia).
    • predicate - The name of the outgoing media constructor sent to the API (for example, inputPhoto, inputDocument). This constructor is guaranteed to have an id field and a file_reference field.
    • stored_constructor - The type of the file ID used to lookup values in the file reference table » and file source table » (for example, fileIdDocument, fileIdPhoto)

    The map file » contains outgoing traversers for all outgoing media constructors that have a file_reference and an id field.
    The current map file has the following outgoing traversers:

(Legacy inputFileLocation constructors are ignored by the current map file).

Outgoing traversers are triggered when a media method call returns a FILE_REFERENCE_EXPIRED or FILE_REFERENCE_INVALID RPC error.

Methods dealing with multiple media files at the same time like messages.sendMultiMedia may also return file reference errors in a slightly different form, as FILE_REFERENCE_%d_EXPIRED errors (or a FILE_REFERENCE_%d_INVALID, which should be treated in the same way): in this case, the media with the file reference to refresh will be located at index %d of the passed multi_media vector.

The same FILE_REFERENCE_%d_INVALID error may also be emitted by messages.sendMedia or other methods requiring a single InputMedia constructor, when an inputMediaPaidMedia is provided with an array of extended_media media files.

When the error is received, clients should:

  • Use the traversers to find all outgoing media constructor(s) matching traverseSwapLocation, contained in the arguments of the failed method call.

    • For FILE_REFERENCE_%d_INVALID errors, select the correct one according to the index, otherwise select the first one.

  • Using the id field of the selected constructor, generate a new file ID » of type traverseSwapLocation.stored_constructor.

  • Use the file ID » to lookup file sources » from the file source table

  • Refresh the file reference using the refresh actions » associated to the file sources:

    Loop once through all stored file sources, executing the associated refresh action », exiting from the loop immediately if:

    • The file reference table » entry associated to the same file ID » is updated with a new value OR
    • There are no remaining sources, in which case the original FILE_REFERENCE_EXPIRED error should be rethrown to the user.

  • Fetch the new file reference from the file reference table » and store it in the file_reference field of the selected constructor from step 1

  • Repeat the original call.

For example, when a messages.sendMedia{media: inputMediaPhoto{id: inputPhoto{id: 12345, file_reference: "...", ...}, ...}, ...} method call sent to the API fails with FILE_REFERENCE_EXPIRED, a new fileIdPhoto{id: 12345} is used to fetch all associated file sources » from the file source table, refresh the file reference, then fetch the new file reference (abc) from the file reference table », and resend the new method call: messages.sendMedia{media: inputMediaPhoto{id: inputPhoto{id: 12345, file_reference: "abc", ...}, ...}, ...}.

It's also possible to pre-emptively replace the file reference of media method calls by running the following simplified flow for all method calls, without waiting for an RPC error (this can be useful in cases where an expired file reference may be cached in some other database or provided externally, such as via an old draftMessage returned by the API):

  • Use the traversers to find all outgoing media constructor(s) matching traverseSwapLocation, contained in the arguments of the method call.
  • For all outgoing media constructors:
    • Using the id field of the selected constructor of type traverseSwapLocation.predicate, generate a new file ID » of type traverseSwapLocation.stored_constructor.
    • If present, fetch the current file reference from the file reference table » and store it in the file_reference field of the selected constructor
  • Proceed with the call

It's also possible to avoid using outgoing traversers altogether if the FileId or FileSources are already passed as external metadata to the client function that invokes the method call (i.e. as a separate additional argument, separate from the ones passed to the API).

Source extraction

A source extractor contains instruction that the client may use to extract a file source ».

A file source » contains instructions on how to extract and store file sources from an incoming method response or update.

See here » for the full list of outgoing source extractors contained by the current map file.

Incoming traversers » and source extractors are used together to populate the file source table », which maps a file ID to one or more file sources.

Schema:

source#dd27c696 flags:# predicate:string is_constructor:flags.0?true stored_constructor:string stored_params:Vector<FieldExtractor> skipped_flags:Vector<string> needs_parent:flags.3?string parent_is_constructor:flags.4?true = Source;

Note: The following sections assume that all Updates constructors have already been converted to a vector of Update constructors, including short variants like updateShortMessage, updateShortSentMessage, updateShortChatMessage, which must be pre-converted to Update constructors by the client using information extracted from the method call.
While this operation could be done within the map file, it would needlessly increase the number of paths: given that most clients already convert short constructors to Update constructors, the map file only considers paths starting from the Update constructors.

Parameters:

  • predicate - Indicates the name of the constructor or of the method where extraction of the fields must start.
  • is_constructor - If set, predicate points to a constructor, otherwise it points to a method.
  • needs_parent - If set, contains the name of a constructor that must be present as a parent in the current traversal branch, or a method whose response is currently being traversed; this parent object is required by one or more path extractors in the source.
  • parent_is_constructor - If set, needs_parent points to a constructor; otherwise it points to a method.
  • stored_constructor - Contains the name of a FileSource constructor from the TL schema specified in fileReferenceMap.db_schema, which should be stored to the database after being populated by stored_params as specified below.
  • stored_params - Contains info on how to populate the FileSource constructor specified in stored_constructor
  • skipped_flags - In some cases, some flag fields of stored_constructor aren't referenced by action: this field contains the full list of flags which are never referenced by the current source and must be unset.

Each unique predicate can have one or more stored_constructors with different params.
Note that method sources cannot make use of needs_parent.

Source field extraction

// Extractor
extractAndStore#72069549 from:Path to:string = FieldExtractor;

extractInputStickerSetFromDocumentAttributesAndStore#369d8d14 from:Path to:string = FieldExtractor;
extractInputStickerSetFromStickerSetAndStore#c167d470 from:Path to:string = FieldExtractor;

extractPeerIdFromPeerAndStore#7d33019c from:Path to:string = FieldExtractor;
extractPeerIdFromInputPeerAndStore#a51acfb4 from:Path to:string = FieldExtractor;

extractChannelIdFromChannelAndStore#5675bc97 from:Path to:string = FieldExtractor;
extractChannelIdFromInputChannelAndStore#b662660e from:Path to:string = FieldExtractor;

extractUserIdFromUserAndStore#4778ec63 from:Path to:string = FieldExtractor;
extractUserIdFromInputUserAndStore#7720aa2e from:Path to:string = FieldExtractor;

// Paths
paramNotFlag#acd9d5cf = ParamFlag;
paramIsFlagAbortIfEmpty#f8fe9fee = ParamFlag;
paramIsFlagFallback#202b77a1 fallback:TypedOp = ParamFlag;
paramIsFlagPassthrough#1dc6e17d = ParamFlag;

pathPart#19a10fbf type:string constructor:string param:string param_type:string flag:ParamFlag = PathPart;

path#c3586a2 parts:Vector<PathPart> = Path;
pathParent#58f13684 parts:Vector<PathPart> = Path;

Field extractors are used to extract a parameter from one or more constructors, i.e. updateStory.story -> storyItem.media -> messageMediaDocument.document -> document.file_reference and store it to the specified field of a FileSource.

Extractors

extractAndStore#72069549 from:Path to:string = FieldExtractor;

extractInputStickerSetFromDocumentAttributesAndStore#369d8d14 from:Path to:string = FieldExtractor;
extractInputStickerSetFromStickerSetAndStore#c167d470 from:Path to:string = FieldExtractor;

extractPeerIdFromPeerAndStore#7d33019c from:Path to:string = FieldExtractor;
extractPeerIdFromInputPeerAndStore#a51acfb4 from:Path to:string = FieldExtractor;

extractChannelIdFromChannelAndStore#5675bc97 from:Path to:string = FieldExtractor;
extractChannelIdFromInputChannelAndStore#b662660e from:Path to:string = FieldExtractor;

extractUserIdFromUserAndStore#4778ec63 from:Path to:string = FieldExtractor;
extractUserIdFromInputUserAndStore#7720aa2e from:Path to:string = FieldExtractor;
extractAndStore

Extracts the value at the specified path and stores it to $FileSource.$to.

extractInputStickerSetFromDocumentAttributesAndStore

Takes the Vector<DocumentAttribute> at from, looks for a documentAttributeSticker, and stores the InputStickerSet contained in documentAttributeSticker.stickerset into $FileSource.$to.

Aborts if there is no attribute of type documentAttributeSticker in the passed vector.

extractInputStickerSetFromStickerSetAndStore

Takes the stickerSet at from, transforms it into an InputStickerSet, and stores it into $FileSource.$to.

extractPeerIdFromPeerAndStore

Takes the Peer at from, transforms it into a bot API peer ID (of type long) », and stores it into $FileSource.$to.

extractPeerIdFromInputPeerAndStore

Takes the InputPeer at from, transforms it into a bot API peer ID (of type long) », and stores it into $FileSource.$to.

If from points to an inputPeerSelf, stores the ID of the current user, instead.

Aborts if from points to an inputPeerEmpty.

extractChannelIdFromChannelAndStore

Takes the channel at from, takes the id field and stores it into $FileSource.$to.

extractChannelIdFromInputChannelAndStore

Takes the InputChannel at from, takes the channel_id field and stores it into $FileSource.$to.

Aborts if from points to an inputChannelEmpty.

extractUserIdFromUserAndStore

Takes the user at from, takes the id field and stores it into $FileSource.$to.

extractUserIdFromInputUserAndStore

Takes the InputUser at from, takes the user_id field and stores it into $FileSource.$to.

If from points to an inputUserSelf, stores the ID of the current user, instead.

Aborts if from points to an inputUserEmpty.

Paths

The first part of the path always points to:

  • The constructor/method of the source (source.predicate), for path
  • The constructor/method of the parent contained in source.needs_parent, for pathParent

A path is composed of multiple pathParts.

Each pathPart contains the following fields, which describe how to extract the field.

  • constructor - Indicates the required constructor/method predicate. If a different constructor type is encountered (i.e. documentEmpty instead of document), abort extraction.
    By definition, if a method name is passed, it will always be equal to the method of the associated source.
  • type - Indicates the type of the required constructor/method.
  • param - Indicates the required parameter; if it's an empty string, it indicates the return value of a method.
  • param_type - Indicates the type of the required parameter; if the parameter is a flag, the underlying type is specified; if param points to the return value of a method, param_type will be equal to type.
  • flag - Contains exactly one of the following constructors:
    • paramNotFlag - The current parameter is not a flag
    • paramIsFlagAbortIfEmpty - The current parameter is a flag, and if it's not set, abort extraction.
    • paramIsFlagFallback - The current parameter is a flag, and if it's not set, use the specified TypedOp as fallback value.
      Note that OPs of type copyOp, getInputChannelByIdOp, getInputUserByIdOp and getInputPeerByIdOp are not allowed in this context.
    • paramIsFlagPassthrough - The current parameter is a flag, and its value should be copied/returned verbatim; can only be used on the last element of a path and within the arguments of a constructorOp/callOp/getMessageOp only if the argument that uses this path is a flag of the same type.

Refresh actions

Refresh actions are used to refresh an expired file reference of a media file according to the file sources » stored in the file source table », associated to the FileId.

See here » for the full list of refresh actions contained by the current map file. 

refreshAction#d4e15689 stored_constructor:string action:ActionOp = RefreshAction;

// Actions
callOp#c2ff3383 method:string args:Vector<TypedOpArg> = ActionOp;
getMessageOp#237849e8 peer:TypedOp id:TypedOp from_scheduled:TypedOp quick_reply_shortcut_id:TypedOp = ActionOp;

// For string => TypedOp dictionaries
typedOpArg#3a2930c2 key:string value:TypedOp = TypedOpArg;

// Typed constructors, the type is specified to simplify codegen,
// but isn't strictly necessary as it can be inferred from the TypedOpOp.
// It is fully pre-validated during the generation of the definition file.
typedOp#705b10ec type:string op:TypedOpOp = TypedOp;

Params:

  • stored_constructor - The name of a file source (an object of type FileSource), specified in the db_schema.
  • action - The action to execute when refreshing a file source of type stored_constructor.

There can only be one action per file source type.

The arguments of the action are composed of a set of typedOp constructors.

typedOp » is a wrapper for a TypedOpOp constructor which also contains the TL type of the associated TypedOpOp; this isn't strictly necessary for evaluation, but it can be useful during automatic code generation from the definition file.

callOp

callOp is a generic action which invokes the method specified in method with the arguments specified in args.

callOp.args will always contain at least all of the required parameters, and possibly some flagged parameters as well.

getMessageOp

getMessageOp is a specialized action which invokes either messages.getMessages or channels.getMessages depending on the type of the peer (which is always a subtype of InputPeer), passing the id as the only element to the vector id parameter.
If the flag to which the from_scheduled points is set, messages.getScheduledMessages should be executed instead.
If the flag to which quick_reply_shortcut_id points is set, messages.getQuickReplyMessages should be executed instead, passing quick_reply_shortcut_id to the shortcut_id parameter and id as the only element of the id vector parameter.

Action parameters

copyOp#f48f418f from:string = TypedOpOp;

getInputChannelByIdOp#3cb47531 from:string = TypedOpOp;
getInputUserByIdOp#c0ee4326 from:string = TypedOpOp;
getInputPeerByIdOp#19813750 from:string = TypedOpOp;

// Literals & constructors (methods not allowed or needed here)
constructorOp#107f8d8a constructor:string args:Vector<TypedOpArg> = TypedOpOp;

vectorOp#f8fb8f72 values:Vector<TypedOp> = TypedOpOp;

intLiteralOp#cbfabe7c value:int = TypedOpOp;
longLiteralOp#d08b8d3a value:long = TypedOpOp;
stringLiteralOp#2b56ea8e value:string = TypedOpOp;
bytesLiteralOp#fdb395a4 value:bytes = TypedOpOp;
boolLiteralOp#37e07911 value:Bool = TypedOpOp;
doubleLiteralOp#3651e3bf value:double = TypedOpOp;
themeFormatLiteralOp#8e4f9208 = TypedOpOp;

Action parameters are represented by TypedOpOp constructors.

copyOp

The most commonly used type, copies the value(s) from the stored $FileSource.$from.

getInputChannelByIdOp

Returns an InputChannel constructor from the client's peer database, based on the channel ID of type long specified in the stored $FileSource.$from.

getInputUserByIdOp

Returns an InputUser constructor from the client's peer database, based on the user ID of type long specified in the stored $FileSource.$from.

getInputPeerByIdOp

Returns an InputPeer constructor from the client's peer database, based on the bot API peer ID » of type long specified in the stored $FileSource.$from.

constructorOp

Constructs the constructor of type (predicate) constructor using the arguments specified in args.

vectorOp

Constructs a vector with the values passed in values.

intLiteralOp

Constructs a literal int with the value passed in value.

longLiteralOp

Constructs a literal long with the value passed in value.

stringLiteralOp

Constructs a literal string with the value passed in value.

bytesLiteralOp

Constructs a literal bytes with the value passed in value.

boolLiteralOp

Constructs a literal Bool with the value passed in value.

doubleLiteralOp

Constructs a literal double with the value passed in value.

themeFormatLiteralOp

Constructs a string, indicating the theming engines supported by the client (used when working with theme-related media, can be an empty string if the client doesn't support themes).

Telegram
Telegram is a cloud-based mobile and desktop messaging app with a focus on security and speed.
About
Mobile Apps
Desktop Apps
Platform
About
Blog
Press
Moderation