khddlZddlmZddlmZmZmZmZddlZddl m cm Z ddlm Z m Z ddlmZmZddlmZddlmZmZdd lmZd d lmZmZmZd d lmZd d lmZm Z gdZ!dejdejfdZ"ejFjIddejdejdee%dejfdZ&ejFjIdGdde jNZ(Gdde jNZ) dSde de de d e dee%d!e%d"ee%d#e*d$e*d%ee d&ee d'eejd(e+de fd)Z,ejFjId*Gd+d,e jNZ-Gd-d.e-Z.Gd/d0e jNZ/Gd1d2e/Z0Gd3d4e jNZ1d5ee%d6e%d7ee%d!ee%dee%d8e*d9eed:e+d;ede1fd<Z2d=eiZ3Gd>d?eZ4Gd@dAeZ5GdBdCeZ6GdDdEeZ7GdFdGeZ8GdHdIeZ9ee dJe4jtfKdddLd9ee4d:e+d;ede1fdMZ;ee dJe5jtfKdddLd9ee5d:e+d;ede1fdNZ<ee dJe6jtfKdddLd9ee6d:e+d;ede1fdOZ=ee dJe7jtfKdddLd9ee7d:e+d;ede1fdPZ>ee dJe8jtfKdddLd9ee8d:e+d;ede1fdQZ?ee dJe9jtfKdddLd9ee9d:e+d;ede1fdRZ@y)TN)partial)AnyCallableListOptional)nnTensor)MLPPermute)StochasticDepth)ImageClassificationInterpolationMode)_log_api_usage_once)register_modelWeights WeightsEnum)_IMAGENET_CATEGORIES)_ovewrite_named_paramhandle_legacy_interface) SwinTransformerSwin_T_WeightsSwin_S_WeightsSwin_B_WeightsSwin_V2_T_WeightsSwin_V2_S_WeightsSwin_V2_B_Weightsswin_tswin_sswin_b swin_v2_t swin_v2_s swin_v2_bxreturnc ,|jdd\}}}tj|ddd|dzd|dzf}|dddddddddf}|dddddddddf}|dddddddddf}|dddddddddf}tj||||gd}|S)Nrr .r)shapeFpadtorchcat)r%HW_x0x1x2x3s _/var/www/teggl/fontify/venv/lib/python3.12/site-packages/torchvision/models/swin_transformer.py_patch_merging_padr7#sggbclGAq! a!Q1q5!QU+,A 31addA  B 31addA  B 31addA  B 31addA  B 2r2r"B'A Hr7relative_position_bias_tablerelative_position_index window_sizec|d|dz}||}|j||d}|jdddjjd}|S)Nrrr)r )viewpermute contiguous unsqueeze)r9r:r;Nrelative_position_biass r6_get_relative_position_biasrC1sg AQ'A9:QR388ArB3;;Aq!DOOQ[[\]^ !!r8rCcneZdZdZej fdededejfffd Z de fdZ xZ S) PatchMergingzPatch Merging Layer. Args: dim (int): Number of input channels. norm_layer (nn.Module): Normalization layer. Default: nn.LayerNorm. dim norm_layer.ct|t|||_t j d|zd|zd|_|d|z|_yNr Fbiassuper__init__rrFrLinear reductionnormselfrFrG __class__s r6rOzPatchMerging.__init__EI D!1s7AG%@q3w' r8r%c`t|}|j|}|j|}|Sz Args: x (Tensor): input tensor with expected layout of [..., H, W, C] Returns: Tensor with layout of [..., H/2, W/2, 2*C] )r7rRrQrTr%s r6forwardzPatchMerging.forwardLs. q ! IIaL NN1 r8 __name__ __module__ __qualname____doc__r LayerNormintrModulerOr rZ __classcell__rUs@r6rErE>? IK (C(Xc299n-E(  r8rEcneZdZdZej fdededejfffd Z de fdZ xZ S)PatchMergingV2zPatch Merging Layer for Swin Transformer V2. Args: dim (int): Number of input channels. norm_layer (nn.Module): Normalization layer. Default: nn.LayerNorm. rFrG.ct|t|||_t j d|zd|zd|_|d|z|_yrIrMrSs r6rOzPatchMergingV2.__init__`rVr8r%c`t|}|j|}|j|}|SrX)r7rQrRrYs r6rZzPatchMergingV2.forwardgs. q ! NN1  IIaLr8r[rds@r6rgrgYrer8rgTinput qkv_weight proj_weightrB num_heads shift_sizeattention_dropoutdropoutqkv_bias proj_bias logit_scaletrainingc H |j\} }}}|d||dzz |dz}|d||dzz |dz}tj|ddd|d|f}|j\}}}}|j}|d|k\rd|d<|d|k\rd|d<t |dkDr"t j ||d |d fd}||dz||dzz}|j| ||dz|d||dz|d|}|jddddddj| |z|d|dz|}| ;| 9| j} | jdz}| |d|zjtj||| }|j|jd|jdd|||zjddddd}|d|d|d}}}| }tj|d tj|d j!d d z}t j"| t%j&d j)} || z}n,|||zdzz}|j+|j!d d }||z}t |dkDr|j-||f}d|d f|d |d f|d d ff}d|d f|d |d f|d d ff} d}!|D]%}"| D]}#|!||"d|"d|#d|#df<|!dz }! '|j||dz|d||dz|d}|jddddj||d|dz}|j/d|j/dz }|j1|dk7t3dj1|dk(t3d}|j|jd|z|||jd|jd}||j/dj/dz}|jd ||jd|jd}tj4|d }tj6||| }|j+|j!ddj|jd|jd|}tj||| }tj6||| }|j| ||dz||dz|d|d|}|jddddddj| |||}t |dkDr t j ||d|dfd}|d d d |d |d d fj9}|S)a Window based multi-head self attention (W-MSA) module with relative position bias. It supports both of shifted and non-shifted window. Args: input (Tensor[N, H, W, C]): The input tensor or 4-dimensions. qkv_weight (Tensor[in_dim, out_dim]): The weight tensor of query, key, value. proj_weight (Tensor[out_dim, out_dim]): The weight tensor of projection. relative_position_bias (Tensor): The learned relative position bias added to attention. window_size (List[int]): Window size. num_heads (int): Number of attention heads. shift_size (List[int]): Shift size for shifted window attention. attention_dropout (float): Dropout ratio of attention weight. Default: 0.0. dropout (float): Dropout ratio of output. Default: 0.0. qkv_bias (Tensor[out_dim], optional): The bias tensor of query, key, value. Default: None. proj_bias (Tensor[out_dim], optional): The bias tensor of projection. Default: None. logit_scale (Tensor[out_dim], optional): Logit scale of cosine attention for Swin Transformer V2. Default: None. training (bool, optional): Training flag used by the dropout parameters. Default: True. Returns: Tensor[N, H, W, C]: The output tensor after shifted window attention. rr)rr )shiftsdimsr rJNr))rFgY@)maxggY)prt)r*r+r,copysumr-rollr=r>reshapeclonenumelzero_linearsize normalize transposeclampmathlogexpmatmul new_zerosr@ masked_fillfloatsoftmaxrpr?)$rjrkrlrBr;rmrnrorprqrrrsrtBr/r0Cpad_rpad_br%r1pad_Hpad_W num_windowslengthqkvqkvattn attn_maskh_slicesw_slicescounthws$ r6shifted_window_attentionrts FJAq!Q ^a+a.0 0KN BE ^a+a.0 0KN BE eaAua/0AAueQ"J1~ 1 1~ 1  : JJq:a=.:a=.!A OKN*u A/FGK q%;q>);q>5KPQN;RT_`aTbdefA !Q1a#++A O[^kZ[n=\^_`A8#7>>#!Q&!f*%++- ((1j( +C ++affQiAy!y. I Q QRSUVXY[\^_ `C!fc!fc!f!qA{{1"% A2(>(H(HR(PPkk+488E?CGGI k! iD( (xx B+, ( (D :KK/ Q(KN?Z]N*KzZ[}n^bMcdQ(KN?Z]N*KzZ[}n^bMcd A 6; !A$1+qtad{23   NN5KN#:KNEU`abUcLcepqrest %%aAq199+{ST~XcdeXfGfg ''*Y-@-@-CC )))q.%-HTTU^bcUcejkneop yyk1; 166RS9VWV\V\]^V_`i))!,66q99yyYq 166!9= 99Tr "D 99T. BD A  A&..qvvay!&&)QGA K+A !w2A q%;q>)5KN+BKPQNT_`aTbdefA !Q1a#++AueQ?A : JJq*Q-A!?f M !RaR!Q,""$A Hr8rceZdZdZ ddedeedeededededed effd Zd Z d Z d e jfdZ de d e fdZxZS)ShiftedWindowAttentionz/ See :func:`shifted_window_attention`. rFr;rnrmrqrrrorpc pt |t|dk7st|dk7r td||_||_||_||_||_tj||dz||_ tj||||_ |j|jy)Nr z.window_size and shift_size must be of length 2rxrK)rNrOlen ValueErrorr;rnrmrorprrPrproj#define_relative_position_bias_tabledefine_relative_position_index) rTrFr;rnrmrqrrrorprUs r6rOzShiftedWindowAttention.__init__s  { q C Oq$8MN N&$"!2 99S#'9IIc3Y7  002 ++-r8ctjtjd|jdzdz d|jdzdz z|j |_tjj|j dy)Nr rr{Gz?std) r Parameterr-zerosr;rmr9init trunc_normal_rTs r6rz:ShiftedWindowAttention.define_relative_position_bias_table sw,.LL KKT--a0014T=M=Ma=P9PST9TUW[WeWe f- ) d??TJr8ctj|jd}tj|jd}tjtj||d}tj |d}|dddddf|dddddfz }|j dddj}|dddddfxx|jddz z cc<|dddddfxx|jddz z cc<|dddddfxxd|jdzdz zcc<|jdj }|jd|y)Nrrijindexingr r)r:) r-aranger;stackmeshgridflattenr>r?rregister_buffer)rTcoords_hcoords_wcoordscoords_flattenrelative_coordsr:s r6rz5ShiftedWindowAttention.define_relative_position_indexs?<< 0 0 34<< 0 0 34U^^HhNOvq1(At4~aqj7QQ)11!Q:EEG1a D$4$4Q$7!$;; 1a D$4$4Q$7!$;; 1a A(8(8(;$;a$?? "1"5"5b"9"A"A"C 68OPr8r&cXt|j|j|jSN)rCr9r:r;rs r6get_relative_position_biasz1ShiftedWindowAttention.get_relative_position_biass(*  - -t/K/KTM]M]  r8r%ch|j}t||jj|jj||j |j |j|j|j|jj|jj|j S) Args: x (Tensor): Tensor with layout of [B, H, W, C] Returns: Tensor with same layout as input, i.e. [B, H, W, C] )rnrorprqrrrt) rrrweightrr;rmrnrorprLrtrTr%rBs r6rZzShiftedWindowAttention.forward#s"&!@!@!B' HHOO II   "    NN"44LLXX]]iinn]]  r8TTr|r|)r\r]r^r_rarboolrrOrrr-r rrZrcrds@r6rrs#&. .#Y.I .  .  ..!..4K Q ELL   F r8rceZdZdZ ddedeedeededededed effd Zd Z d e jfd Z de fdZ xZS)ShiftedWindowAttentionV2z2 See :func:`shifted_window_attention_v2`. rFr;rnrmrqrrrorpc 4t |||||||||tjt j dt j |ddfz|_tjtjdddtjdtjd|d |_ |r\|jjjd z} |jj| d| zjj!yy) N)rqrrrorp rr iTrK)inplaceFrx)rNrOrrr-ronesrs SequentialrPReLUcpb_mlprrLrdatar) rTrFr;rnrmrqrrrorprrUs r6rOz!ShiftedWindowAttentionV2.__init__@s     /  << "uzz9aQRBS7T2T(UV}} IIa4 ("''$*?3PY`eAf   XX]]((*a/F HHMM&1v: . 3 3 9 9 ; r8ctj|jddz |jdtj}tj|jddz |jdtj}tjtj ||gd}|j dddjjd}|dddddddfxx|jddz zcc<|dddddddfxx|jddz zcc<|dz}tj|tjtj|dzzd z }|jd |y) Nrr)dtyperrr g?g@relative_coords_table) r-rr;float32rrr>r?r@signlog2absr)rTrelative_coords_hrelative_coords_wrs r6rz+B)CTEUEUVWEX`e`m`mn!LL4+;+;A+>+B)CTEUEUVWEX`e`m`mn % ENNM &ELL&  r8rceZdZdZddddej efdededeedeede d e d e d e d e d ejfde d ejfffd Z de fdZxZS)SwinTransformerBlocka Swin Transformer Block. Args: dim (int): Number of input channels. num_heads (int): Number of attention heads. window_size (List[int]): Window size. shift_size (List[int]): Shift size for shifted window attention. mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. Default: 4.0. dropout (float): Dropout rate. Default: 0.0. attention_dropout (float): Attention dropout rate. Default: 0.0. stochastic_depth_prob: (float): Stochastic depth rate. Default: 0.0. norm_layer (nn.Module): Normalization layer. Default: nn.LayerNorm. attn_layer (nn.Module): Attention layer. Default: ShiftedWindowAttention @r|rFrmr;rn mlp_ratiorprostochastic_depth_probrG. attn_layerc Lt |t|| ||_| |||||||_t |d|_| ||_t|t||z|gtjd||_ |jjD]~} t| tjstj j#| j$| j&Ttj j)| j&dy)N)rorprow)activation_layerrrpgư>r)rNrOrnorm1rr stochastic_depthnorm2r rarGELUmlpmodules isinstancerPrxavier_uniform_rrLnormal_) rTrFrmr;rnrrprorrGrmrUs r6rOzSwinTransformerBlock.__init__s D!_     /   !00Eu M_ sSy137"''[_ipq!!# 6A!RYY'''166%GGOOAFFO5  6r8r%c||j|j|j|z}||j|j|j |z}|Sr)rrrrrrYs r6rZzSwinTransformerBlock.forwardsS %%dii 1 &>? ? %%dhhtzz!}&=> >r8)r\r]r^r_rr`rrarrrrbrOr rZrcrds@r6rrs *#&'*/1||/E!6 !6!6#Y !6 I !6  !6!6!!6 %!6S"))^,!6S"))^,!6Fr8rceZdZdZddddej efdededeedeede d e d e d e d e d ejfde d ejfffd Z de fdZxZS)SwinTransformerBlockV2a Swin Transformer V2 Block. Args: dim (int): Number of input channels. num_heads (int): Number of attention heads. window_size (List[int]): Window size. shift_size (List[int]): Shift size for shifted window attention. mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. Default: 4.0. dropout (float): Dropout rate. Default: 0.0. attention_dropout (float): Attention dropout rate. Default: 0.0. stochastic_depth_prob: (float): Stochastic depth rate. Default: 0.0. norm_layer (nn.Module): Normalization layer. Default: nn.LayerNorm. attn_layer (nn.Module): Attention layer. Default: ShiftedWindowAttentionV2. rr|rFrmr;rnrrprorrG.rc 8t |||||||||| |  y)N)rrprorrGr)rNrO) rTrFrmr;rnrrprorrGrrUs r6rOzSwinTransformerBlockV2.__init__s5     /"7!!  r8r%c||j|j|j|z}||j|j|j |z}|Sr)rrrrrrYs r6rZzSwinTransformerBlockV2.forwardsU %%djj1&>? ? %%djj!&=> >r8)r\r]r^r_rr`rrarrrrbrOr rZrcrds@r6rrs *#&'*/1||/G   #Y  I     !  % S"))^, S"))^, 4r8rceZdZdZdddddddefdeeded eed eed eed ed ededededee de jfdee de jfde de jfffd Z dZ xZS)ra; Implements Swin Transformer from the `"Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" `_ paper. Args: patch_size (List[int]): Patch size. embed_dim (int): Patch embedding dimension. depths (List(int)): Depth of each Swin Transformer layer. num_heads (List(int)): Number of attention heads in different layers. window_size (List[int]): Window size. mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. Default: 4.0. dropout (float): Dropout rate. Default: 0.0. attention_dropout (float): Attention dropout rate. Default: 0.0. stochastic_depth_prob (float): Stochastic depth rate. Default: 0.1. num_classes (int): Number of classes for classification head. Default: 1000. block (nn.Module, optional): SwinTransformer Block. Default: None. norm_layer (nn.Module, optional): Normalization layer. Default: None. downsample_layer (nn.Module): Downsample layer (patch merging). Default: PatchMerging. rr|g?iN patch_size embed_dimdepthsrmr;rrpror num_classesrG.blockdownsample_layercxt|t|| |_| t} | t t jd} g}|jt jt jd||d|df|d|dftgd| |t|}d}tt|D]}g}|d|zz}t||D]Y}| t|z|dz z }|j| |||||Dcgc]}|dzdk(rdn|dzc}||||| |dz }[|jt j||t|dz ks|j| || t j||_|dt|dz zz}| ||_tgd |_t j&d|_t j*d|_t j.|| |_|j3D]~}t5|t j.st j6j9|j:d |j<Vt j6j?|j<ycc}w) Ngh㈵>)epsrxrr) kernel_sizestride)rr rxrr )r;rnrrprorrG)rrxrr rr) rNrOrrrrrr`appendrConv2dr rrangerrfeaturesrRr>AdaptiveAvgPool2davgpoolFlattenrrPheadrrrrrrLzeros_)rTrrrrmr;rrprorrrGrrlayerstotal_stage_blocksstage_block_idi_stagestagerFi_layersd_probr num_featuresrrUs r6rOzSwinTransformer.__init__s D!& =(E   48J"$ MM yz!}jm.LV`abVceopqerUs %9%   ![S[) AG%'Eaj(C 1 $/%2GGK]`aKab !'*$/OZ#[!1)9AqAv$E#["+ '*;.5#-  !#! $" MM"--/ 0#f+/* .sJ?@/ A0 v.  1Vq#99 |, |, ++A. zz!} IIlK8  +A!RYY'%%ahhD%966%GGNN166*  +-$\sJ7c|j|}|j|}|j|}|j|}|j |}|j |}|Sr)r rRr>rrrrYs r6rZzSwinTransformer.forward_sV MM!  IIaL LLO LLO LLO IIaLr8)r\r]r^r_rErrarrrrrbrOrZrcrds@r6rrs4#&'*9=485AM+IM+M+S M+ 9 M+ #Y M+M+M+!M+ %M+M+Xc299n56M+bii01M+#3 >2M+^r8rrrrrweightsprogresskwargsc |#t|dt|jdtd||||||d|} |"| j |j |d| S)Nr categories)rrrrmr;rT)r check_hash)rrmetarload_state_dictget_state_dict) rrrrmr;rrrrmodels r6_swin_transformerr&is{fmSl9S5TU  3    E g44hSW4XY Lr8rc peZdZedeeddejieddddd d d id d ddZ e Z y)rz7https://download.pytorch.org/models/swin_t-704ceda3.pth crop_size resize_size interpolationibr(r(Uhttps://github.com/pytorch/vision/tree/main/references/classification#swintransformer ImageNet-1KguV^T@glW@zacc@1zacc@5gX9@g\( [@YThese weights reproduce closely the results of the paper using a similar training recipe. num_paramsmin_sizerecipe_metrics_ops _file_size_docsurl transformsr"N r\r]r^rrrrBICUBIC _COMMON_META IMAGENET1K_V1DEFAULTr!r8r6rrsg E 3CO`OhOh   ""m##   t M*Gr8rc peZdZedeeddejieddddd d d id d ddZ e Z y)rz7https://download.pytorch.org/models/swin_s-5e29d889.pthr(r*irr.r/r0gCT@gףp= X@r1gZd{!@gx&g@r2r3r;Nr>r!r8r6rrsg E 3CO`OhOh   ""m##  !t M*Gr8rc peZdZedeeddejieddddd d d id d ddZ e Z y)rz7https://download.pytorch.org/models/swin_b-68c6b09e.pthr(r*i<;r.r/r0gh|?T@g)\(X@r1g&1.@gt@r2r3r;Nr>r!r8r6rrsg E 3CO`OhOh   ""m##  !t M*Gr8rc peZdZedeeddejieddddd d d id d ddZ e Z y)rz:https://download.pytorch.org/models/swin_v2_t-b137f0e2.pthr*iRrHrHXhttps://github.com/pytorch/vision/tree/main/references/classification#swintransformer-v2r0gS㥛T@g rX@r1g(\@gMb([@r2r3r;Nr>r!r8r6rrsg H 3CO`OhOh   ""p##  !t M*Gr8rc peZdZedeeddejieddddd d d id d ddZ e Z y)rz:https://download.pytorch.org/models/swin_v2_s-637d8ceb.pthrHrIr*irJrKr0g!rhT@gNbX94X@r1gd;O'@gg@r2r3r;Nr>r!r8r6rrg H 3CO`OhOh   ""p##  !t M*Gr8rc peZdZedeeddejieddddd d d id d ddZ e Z y)rz:https://download.pytorch.org/models/swin_v2_b-781e5279.pthrHir*i=rJrKr0gI +U@gK7X@r1g33333S4@gˡEu@r2r3r;Nr>r!r8r6rr rMr8r pretrained)r)rrc dtj|}tdddgdgdgdddgd||d|S) a Constructs a swin_tiny architecture from `Swin Transformer: Hierarchical Vision Transformer using Shifted Windows `_. Args: weights (:class:`~torchvision.models.Swin_T_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.Swin_T_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.swin_transformer.SwinTransformer`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.Swin_T_Weights :members: rJ`r r r rxrS 皙?rrrrmr;rrrr!)rverifyr&rrrs r6rr"sP.##G,G  q6 F!    r8c dtj|}tdddgdgdgdddgd||d|S) a Constructs a swin_small architecture from `Swin Transformer: Hierarchical Vision Transformer using Shifted Windows `_. Args: weights (:class:`~torchvision.models.Swin_S_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.Swin_S_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.swin_transformer.SwinTransformer`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.Swin_S_Weights :members: rJrQr r r rTrW333333?rYr!)rrZr&r[s r6r r HsP.##G,G  q6 F!    r8c dtj|}tdddgdgdgdddgd||d|S) a Constructs a swin_base architecture from `Swin Transformer: Hierarchical Vision Transformer using Shifted Windows `_. Args: weights (:class:`~torchvision.models.Swin_B_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.Swin_B_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.swin_transformer.SwinTransformer`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.Swin_B_Weights :members: rJr]rJrr rW?rYr!)rrZr&r[s r6r!r!nsP.##G,G  q6 F!    r8c xtj|}tdddgdgdgdddgd||ttd |S) a Constructs a swin_v2_tiny architecture from `Swin Transformer V2: Scaling Up Capacity and Resolution `_. Args: weights (:class:`~torchvision.models.Swin_V2_T_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.Swin_V2_T_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.swin_transformer.SwinTransformer`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.Swin_V2_T_Weights :members: rJrQrRrTrrX rrrrmr;rrrrrr!)rrZr&rrgr[s r6r"r"sV. &&w/G  q6 F!$'    r8c xtj|}tdddgdgdgdddgd||ttd |S) a Constructs a swin_v2_small architecture from `Swin Transformer V2: Scaling Up Capacity and Resolution `_. Args: weights (:class:`~torchvision.models.Swin_V2_S_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.Swin_V2_S_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.swin_transformer.SwinTransformer`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.Swin_V2_S_Weights :members: rJrQr]rTrr_rfr!)rrZr&rrgr[s r6r#r#sV. &&w/G  q6 F!$'    r8c xtj|}tdddgdgdgdddgd||ttd |S) a Constructs a swin_v2_base architecture from `Swin Transformer V2: Scaling Up Capacity and Resolution `_. Args: weights (:class:`~torchvision.models.Swin_V2_B_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.Swin_V2_B_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.swin_transformer.SwinTransformer`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.Swin_V2_B_Weights :members: rJrar]rbrrdrfr!)rrZr&rrgr[s r6r$r$sV. &&w/G  q6 F!$'    r8)r|r|NNNT)Ar functoolsrtypingrrrrr-torch.nn.functionalr functionalr+r ops.miscr r ops.stochastic_depthr transforms._presetsrrutilsr_apirrr_metar_utilsrr__all__r7fxwraprarCrbrErgrrrrrrrrr&r@rrrrrrrArr r!r"r#r$r!r8r6rws 00 #2H'66'B " %,, 5<<  "#""',,"IN"dhildm" \\" +,2996RYYF #!%"&*.p p p p # p c p  p S p p p vp p %,,'p p  p f ()N RYYN bS 5S l62996r/1/djbiijZS  ICy  c  ! k ">& [2[2[2 2 2 2,0L0L!MN26!x/!$!Y\!ap!O!H,0L0L!MN26!x/!$!Y\!ap!O!H,0L0L!MN26!x/!$!Y\!ap!O!H,0A0O0O!PQ8