kh|+ ddlZddlZddlZddlmZddlmZddlmZm Z m Z m Z m Z m Z mZmZddlZddlZddlmZddlmZddlmZgdZd-d ej.d efd Z d.d e d ej.dededej.f dZ d.d ej.dededefdZ d.d ej.dededefdZdZ d/de e eefdee e fdedefdZ d/d ej.dedefdZ d/d ej.dedefdZ! d0de e eej.fde"deee ed ffd!eee eej.ffd"Z#dd#d$e efd%Z$ d1d&e"d'ed(ed$e efd)Z%d*e"d+edefd,Z&y)2N) defaultdict)chain)AnyCallableDictIteratorOptionalTupleTypeUnion)nn)Tensor)use_reentrant_ckpt) model_parameters named_apply named_modulesnamed_modules_with_paramsadapt_input_convgroup_with_matcher group_modulesgroup_parametersflatten_modulescheckpoint_seq checkpointmodel exclude_headcr|r!|jDcgc]}|c}ddS|jScc}w)N) parameters)rrps S/var/www/teggl/fontify/venv/lib/python3.12/site-packages/timm/models/_manipulate.pyrrs: ++-.a.s33!!/s 4fnmodule depth_first include_rootreturnc|s |r ||||jD],\}}|rdj||fn|}t||||d.|r |r ||||S)N)r#name.T)r"r#r(r$r%)named_childrenjoinr)r"r#r(r$r% child_name child_modules r!rrsp < &t$$*$9$9$;m L59SXXtZ01z r,Z[gklm| &t$ Mr(c#K|s|r||f|jD]3\}}|rdj||fn|}t|||dEd{5|r |r||fyyy7wNr)T)r#r(r$r%)r*r+rr#r(r$r%r,r-s r!rr,s <Fl$*$9$9$;^ L59SXXtZ01z  jkX\^ ^ ^^|Fl${ ^sA A" A  A"c#K|jr |s|r||f|jD]3\}}|rdj||fn|}t|||dEd{5|jr |r |r||fyyyy7 wr0) _parametersr*r+rr1s r!rr<s +,Fl$*$9$9$;^ L59SXXtZ01z ,jkX\^ ^ ^^klFl/;k ^sAA;A9!A;)i named_objects group_matcher return_valuesreversecttrg}tjD]i\}\}}| t|tt fr+|D]%}|t j|d|f|dfgz }'M|t j||fdfgz }k|fd} tt } |D]#\} } | | | j|r| n| %tt } d}ttd| jD]2} |dks| dtdk7r|dz }| |j| | 4|r1|rJdi}| jD]\}}|D]}|||< |S| S)Nrc tttfruD]d\}}}|j|}|s||j |f}tt t tjtd|cSt dfS|}t|tjjs|fSt|S)Ninf) isinstancelisttuplematchgroupsmapfloatr from_iterablefilter collectionsabcIterable)r(match_fnprefixsuffixrpartsordr5s r! _get_groupingz)group_with_matcher.._get_groupingds mdE] 3,9 W(&&NN4(#QXXZ8E UE,?,?tU@S,T!UVV  W <= %Cc;??#;#;<t : r.c |duS)N)xs r!z$group_with_matcher..{s Qd]r.z-reverse mapping only sensible for name output)r<dict enumerateitemsr>r=recompilerappendsortedrDkeysMATCH_PREV_GROUPextend)r4r5r6r7compiled group_ordinal group_namemspecsspecrNgroupingkvlayer_id_to_paramlidparam_to_layer_idlmns ` r!rrOs -&2;Mstrr Sequential ModuleList ModuleDictrr*r+)rrprIrqprefix_is_tupler(r#s r!rrs !/O,$ ; &MM2=="--HLMM+L%# f Z 5&%%' "1wt)   'Fl"88VTN3DFl"# sB%C'C(3C use_reentrantrzcx| t}tjjj|g|d|i|S)z~ checkpoint wrapper fn A thin wrapper around torch.utils.checkpoint.checkpoint to default use_reentrant to False rz)rtorchutilsr)functionrzargskwargss r!rrsN*, ;; ! ! , ,   $    r.everyflatten skip_lastc| t}d}t|tjjr|j }|rt j|}t|ttfs t|}t|}|r|dz}d}td||D]K} t| |zdz |dz }tjjj|| ||||}M|r||dzt|dz ||S|S)a,A helper function for checkpointing sequential models. Sequential models execute a list of modules/functions in order (sequentially). Therefore, we can divide such a sequence into segments and checkpoint each segment. All segments except run in :func:`torch.no_grad` manner, i.e., not storing the intermediate activations. The inputs of each checkpointed segment will be saved for re-running the segment in the backward pass. See :func:`~torch.utils.checkpoint.checkpoint` on how checkpointing works. .. warning:: Checkpointing currently only supports :func:`torch.autograd.backward` and only if its `inputs` argument is not passed. :func:`torch.autograd.grad` is not supported. .. warning: At least one of the inputs needs to have :code:`requires_grad=True` if grads are needed for model inputs, otherwise the checkpointed part of the model won't have gradients. Args: functions: A :class:`torch.nn.Sequential` or the list of modules or functions to run sequentially. x: A Tensor that is input to :attr:`functions` every: checkpoint every-n functions (default: 1) flatten: flatten nn.Sequential of nn.Sequentials skip_last: skip checkpointing the last function in the sequence if True use_reentrant: Use re-entrant checkpointing Returns: Output of running :attr:`functions` sequentially on :attr:`*inputs` Example: >>> model = nn.Sequential(...) >>> input_var = checkpoint_seq(model, input_var, every=2) cfd}|S)NcFtdzD] }||}|S)Nr9)range)_xjend functionsstarts r!forwardz5checkpoint_seq..run_function..forwards25#'* &!Yq\"% &Ir.rQ)rrrrs``` r! run_functionz$checkpoint_seq..run_functions r.r9rOrry)rr<r|r ruchildrenrrCr>r=lenrminr}r) rrRrrrrzrnum_checkpointedrrs r!rrsV*, )UXX001&&( '' 2 i% /)$ 9~A Cq*E2 %%-!#%5%9: KK " " - - Y / ' .  C|C!GS^a%7CAFF Hr.in_chans conv_weightc|j}|j}|j\}}}}|dk(r\|dkDrC|jddzdk(sJ|j||dzd||}|j dd}n~|j dd}nj|dk7re|dk7r t dt tj|dz }|jd|ddddd|ddddf}|dt|z z}|j|}|S) Nr9rF)dimkeepdimTz*Weight format not supported by conversion.) dtyperBshapereshapesumNotImplementedErrorintmathceilrepeatto)rr conv_typeOIJKrs r!rr s!!I##%K""JAq!Q1} q5$$Q'!+q0 00%--aaAqAK%//a/?K%//a/>K Q 6%&RS S8a<01F%,,Q1=a(Aq>PQK Ah/ 0K..+K r.)F)TF)FF)r9r sequential)r9FFN)'collections.abcrErrWr itertoolsrtypingrrrrr r r r r|torch.utils.checkpointr r timm.layersr__all__Moduleboolrrrtrrr\rrrrrrrrrQr.r!rs #NNN * y"BII"T"!# "        YY $ "      $ "        $ :c3h0:T8^,:: :@$ ` `` `$ h hh h.0;G #c299n 56##c5c?*+#CtBII!778 #B%)D>2(, I I  I  I  ~ I XsFr.