o MThC@sfUddlZddlZddlZddlZddlZddlZddlmZddlmZm Z m Z ddl m Z ddl mZmZmZmZmZmZmZmZmZmZmZmZerdddlmZGdd d e ZeeefZejZd d ZGd d d e d dZ!dMddZ"dMddZ#ddZ$ddZ%ddZ&dNddZ'ddej()DZ*dNddZ+d dej()DZ,d!dej()Dhd"BZ-d#d$Z.d%d&Z/d'd(Z0d)d*Z1d+d,Z2ej3fZ4eed-fe5d.<z dd/l6m7Z7e4e7f7Z4Wn e8yYnwd0d1Z9d2d3Z:d4d5Z;d6d7ZZ?d:d;Z@dNdd?ZBGd@dAdAZCdBdCZDdDdEZEdFdGZFe ddHdIdJZGdKdLZHdS)ON)ABCMeta)ModuleexprAST) lru_cache) CallableDictIterableIteratorListOptionalTupleUnioncastAny TYPE_CHECKINGType)NodeNGc@seZdZdZdZdZdS) EnhancedASTNr)__name__ __module__ __qualname__ first_token last_tokenlinenorrQ/home/air/segue/gemini/backup/venv/lib/python3.10/site-packages/asttokens/util.pyr,srcCsdtj|t|dfS)zRReturns a human-friendly representation of a token with the given type and string.z%s:%su)tokentok_namereprlstrip)tok_typestringrrr token_repr7sr%c@seZdZdZddZdS)Tokena TokenInfo is an 8-tuple containing the same 5 fields as the tokens produced by the tokenize module, and 3 additional ones useful for this module: - [0] .type Token type (see token.py) - [1] .string Token (a string) - [2] .start Starting (row, column) indices of the token (a 2-tuple of ints) - [3] .end Ending (row, column) indices of the token (a 2-tuple of ints) - [4] .line Original line (string) - [5] .index Index of the token in the list of tokens that it belongs to. - [6] .startpos Starting character offset into the input text. - [7] .endpos Ending character offset into the input text. cCst|j|jSN)r%typer$selfrrr__str__Lsz Token.__str__N)rrr__doc__r+rrrrr&>s r&z0type string start end line index startpos endposcCs|j|ko |dup |j|kS)zVReturns true if token is of the given type and, if a string is given, has that string.N)r(r$rr#tok_strrrr match_tokenQsr/cCs>t|||stdt||t||jd|jddfdS)z Verifies that the given token is of the expected type. If tok_str is given, the token string is verified too. If the token doesn't match, raises an informative ValueError. z+Expected token %s, got %s on line %s col %srrN)r/ ValueErrorr%strstartr-rrr expect_tokenWs r3cCs|tjtjtjfvS)zS These are considered non-coding tokens, as they don't affect the syntax tree. )rNLCOMMENTENCODING) token_typerrris_non_coding_tokencsr8cCs tttgtft|jS)z; Generates standard library tokens for the given code. )tokenizegenerate_tokensrrr1ioStringIOreadline)textrrrr:ks r:cCst|drtStS)z Returns a function which yields all direct children of a AST node, skipping children that are singleton nodes. The function depends on whether ``node`` is from ``ast`` or from the ``astroid`` module. get_children)hasattriter_children_astroiditer_children_astnoderrriter_children_funcvsrEFcCs|st|rgS|Sr') is_joined_strr?)rDinclude_joined_strrrrrAs rAc Cs<h|]\}}t|trt|tjtjtjtjtjfr|qSr) isinstancer( issubclassast expr_contextboolopoperatorunaryopcmpop.0ncrrr s rTccsr|s t|r dSt|tjr't|j|jD]\}}|dur!|V|VqdSt|D] }|jt vr6|Vq,dSr') rFrHrJrzipkeysvaluesiter_child_nodes __class__ SINGLETONS)rDrGkeyvaluechildrrrrBs   rBcC*h|]\}}t|trt|tjr|qSr)rHr(rIrJstmtrPrrrrT   cCr^r)rHr(rIrJrrPrrrrTr`>ConstDelAttrDelName AssignAttr AssignNamecC |jjtvS)z+Returns whether node is an expression node.)rYrexpr_class_namesrCrrris_expr rhcCrf)z)Returns whether node is a statement node.)rYrstmt_class_namesrCrrris_stmtrirkcC |jjdkS)z&Returns whether node is a module node.rrYrrCrrr is_modulerirncCrl)zFReturns whether node is a JoinedStr node, used to represent f-strings. JoinedStrrmrCrrrrFs rFcCrl)zSReturns whether node is an `Expr` node, which is a statement that is an expression.ExprrmrCrrr is_expr_stmtrirq.CONSTANT_CLASSES)racCs t|tS)z(Returns whether node is a Constant node.)rHrrrCrrr is_constants rscCst|o|jtuS)z)Returns whether node is an Ellipsis node.)rsr\EllipsisrCrrr is_ellipsissrucCrl)z2Returns whether node is a starred expression node.StarredrmrCrrr is_starredrirwcCs0|jjdvp|jjdkotttttj|jS)z?Returns whether node represents a slice, e.g. `1:2` in `x[1:2]`)SliceExtSlicer ) rYranymapis_slicerrJr eltsrCrrrr|s  r|cCsB|jjdko t|tj o |j|juo|juoduSS)Nrx)rYrrHrJrloweruppersteprCrrris_empty_astroid_slices   rcCs|sdd}t|}t}d}|dtfg}|r^|\}}} | turQ||vs(J|||||\} } |||| ft|} ||D] } || | | tfqDn |||tt t | }|s|S)a5 Scans the tree under the node depth-first using an explicit stack. It avoids implicit recursion via the function call stack to avoid hitting 'maximum recursion depth exceeded' error. It calls ``previsit()`` and ``postvisit()`` as follows: * ``previsit(node, par_value)`` - should return ``(par_value, value)`` ``par_value`` is as returned from ``previsit()`` of the parent. * ``postvisit(node, par_value, value)`` - should return ``value`` ``par_value`` is as returned from ``previsit()`` of the parent, and ``value`` is as returned from ``previsit()`` of this node itself. The return ``value`` is ignored except the one for the root node, which is returned from the overall ``visit_tree()`` call. For the initial node, ``par_value`` is None. ``postvisit`` may be None. cSsdSr'r)rDpvaluer\rrrszvisit_tree..N) rEset _PREVISITpopaddappendleninsertrr r&)rDprevisit postvisit iter_childrendoneretstackcurrent par_valuer\r post_valueinsrRrrr visit_trees(    rccsnt|}t}|g}|r5|}||vsJ|||Vt|}|||D]}|||q(|s dSdS)a1 Recursively yield all descendant nodes in the tree starting at ``node`` (including ``node`` itself), using depth-first pre-order traversal (yieling parents before their children). This is similar to ``ast.walk()``, but with a different order, and it works for both ``ast`` and ``astroid`` trees. Also, as ``iter_children()``, it skips singleton nodes generated by ``ast``. By default, ``JoinedStr`` (f-string) nodes and their contents are skipped because they previously couldn't be handled. Set ``include_joined_str`` to True to include them. N)rErrrrr)rDrGrrrrrrSrrrwalk's    rcCsXd}g}t|D]\}}}|||||||}q|||dd|S)ah Replaces multiple slices of text with new values. This is a convenience method for making code modifications of ranges e.g. as identified by ``ASTTokens.get_text_range(node)``. Replacements is an iterable of ``(start, end, new_text)`` tuples. For example, ``replace("this is a test", [(0, 4, "X"), (8, 9, "THE")])`` produces ``"X is THE test"``. rN)sortedrjoin)r> replacementsppartsr2endnew_textrrrreplaceDs   rc@s eZdZdZddZddZdS) NodeMethodsz[ Helper to get `visit_{node_type}` methods given a node's class and cache the results. cCs i|_dSr')_cacher)rrr__init__\s zNodeMethods.__init__cCs:|j|}|sd|j}t|||j}||j|<|S)z Using the lowercase name of the class as node_type, returns `obj.visit_{node_type}`, or `obj.visit_default` if the type-specific method is not found. visit_)rgetrr~getattr visit_default)r*objclsmethodnamerrrr`s  zNodeMethods.getN)rrrr,rrrrrrrXs rccs|g}|D],}|jtjtjtjfvr"|r|dj|jkr"||qt|D]}|Vq&g}|Vqt|D]}|Vq6dS)a Fixes tokens yielded by `tokenize.generate_tokens` to handle more non-ASCII characters in identifiers. Workaround for https://github.com/python/cpython/issues/68382. Should only be used when tokenizing a string that is known to be valid syntax, because it assumes that error tokens are not actually errors. Combines groups of consecutive NAME, NUMBER, and/or ERRORTOKEN tokens into a single NAME token. N) r(r9NAME ERRORTOKENNUMBERrr2rcombine_tokens)original_tokensgrouptokcombined_tokenrrrpatched_generate_tokensns    rcCsftdd|Drtdd|Ddkr|Stjtjddd|D|dj|d j|djd gS) Ncss|] }|jtjkVqdSr')r(r9rrQrrrr sz!combine_tokens..cSsh|]}|jqSr)linerrrrrTsz!combine_tokens..rrcss|]}|jVqdSr')r$)rQtrrrrsrr)r(r$r2rr) rzrr9 TokenInforrr2rr)rrrrrs(rcCs*ddt||D}|rt|dS|S)zq If the given AST node contains multiple statements, return the last one. Otherwise, just return the node. cSs&g|]}t|st|jdvr|qS)) excepthandler ExceptHandler match_case MatchCase TryExcept TryFinally)rkr(r)rQr]rrr szlast_stmt..r)rE last_stmt)rD child_stmtsrrrrs   r)maxsizecsbdt}ddt|D}dd|D}tt|t|k}tfdd|D}|o0|S)a The positions attached to nodes inside f-string FormattedValues have some bugs that were fixed in Python 3.9.7 in https://github.com/python/cpython/pull/27729. This checks for those bugs more concretely without relying on the Python version. Specifically this checks: - Values with a format spec or conversion - Repeated (i.e. identical-looking) expressions - f-strings implicitly concatenated over multiple lines. - Multiline, triple-quoted f-strings. z( f"a {b}{b} c {d!r} e {f:g} h {i:{j}} k {l:{m:n}}" f"a {b}{b} c {d!r} e {f:g} h {i:{j}} k {l:{m:n}}" f"{x + y + z} {x} {y} {z} {z} {z!a} {z:z}" f''' {s} {t} {u} {v} ''' )cSsg|] }t|tjr|qSr)rHrJNamerQrDrrrrsz*fstring_positions_work..cSsg|]}|j|jfqSr)r col_offsetrrrrrsc3s"|] }t||jkVqdSr')rJget_source_segmentidrsourcerrrs  z)fstring_positions_work..)rJparserrrall)tree name_nodesname_positionspositions_are_uniquecorrect_source_segmentsrrrfstring_positions_works  rcCstjdkrdSt|ddD]6}t|tjsq |jD])}t|ddt|tjrBt s8t|j D]}t|ddq/|j rBt|j ddqq dS)zy Add a special attribute `_broken_positions` to nodes inside f-strings if the lineno/col_offset cannot be trusted. ) NT)rG_broken_positions) sys version_inforrHrJrorWsetattrFormattedValuerr\ format_spec)r joinedstrpartr]rrrannotate_fstring_nodess     rr')F)IrJ collectionsr;rrr9abcrrrr functoolsrtypingrrr r r r r rrrrrastroid_compatrrAstNoderr% namedtupler&r/r3r8r:rErA__dict__itemsrZrBrjrgrhrkrnrFrqConstantrr__annotations__astroidra ImportErrorrsrurwr|robjectrrrrrrrrrrrrrrsr   8           +