kh*M  dZddlmZmZmZddlmZmZddlm Z ddl m Z m Z ddl mZmZmZmZgZeeedd d ddd edd d d dd edd dd dd edd dd dd fdddddeeedd d ddd edd gdd d dd edd gddd dd edd dd ddfddddd d!de" eeddd ddd edd#d#d d dd edd$dd dd edddd dd fdddddde%eedd d dd&d edd d d d&d edd dd d&d edd dd d&d fdddddd'ded&() eedddddd* eddd+d dd* edd,d d dd* eddd d dd* fdd-dd.ed/d01eedd d ddd edd d d dd edd dd dd edd dd dd fdddd.ed/d 01eedd d ddd edd gdd d dd edd gddd dd edd dd ddfddddd d!d.ed/d0" eeddd ddd edd#d#d d dd d.ed/dd#23 edd$dd dd edddd dd fddddd.ed/d04eedd d dd&d edd d d d&d edd dd d&d edd dd d&d fddddd'd.ed/d d&56eedd d ddd edd d d dd edd dd dd edd dd dd fdddd7ed891eeddd ddd edd#d#d d dd edd$dd dd edddd dd fddddd7ed894eedd d ddd edd d d dd edd dd dd edd dd dd fdddd7ed:91eedd d;d d&d edd$dd d&dedd d?d d&dfd@ddddd!ed AeddBd.edCd dDEF eeddd ddd edd#d d dd d7edG9Hedd$dd dd d.edIHedddd dd deHfddddJeeddd ddd edd#d d dd d.edIHedd$dd dd d.edIHedddd dd deHfddddJKZddLedMeedNedOefdPZddQedOeeeffdRZe edSdTddUdVWedXdTddUdVdYZeddUdV[ed\dTddUdVWedUdVdU]ed^dTdUdVdU_ed`dTdUdVdUdYaedbdTdUdVdUdYaedcdTdUdVdUdYaedddTdedUdVdYfedgdTdedUdVhedidTddUdVdYZedjdTdkdkdldmdndmdYo edpdTddUdVWedqdTddUdVWdrZe ddNedOefdsZe ddNedOefdtZe ddNedOefduZe ddNedOefdvZe ddNedOefdwZe ddNedOefdxZe ddNedOefdyZ e ddNedOefdzZ!e ddNedOefd{Z"e ddNedOefd|Z#e ddNedOefd}Z$e ddNedOefd~Z%e ddNedOefdZ&e ddNedOefdZ'e ddNedOefdZ(y:)a Bring-Your-Own-Attention Network A flexible network w/ dataclass based config for stacking NN blocks including self-attention (or similar) layers. Currently used to implement experimental variants of: * Bottleneck Transformers * Lambda ResNets * HaloNets Consider all of the models definitions here as experimental WIP and likely to change. Hacked together by / copyright Ross Wightman, 2021. )AnyDictOptionalIMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD)build_model_with_cfg)register_modelgenerate_default_cfgs) ByoBlockCfg ByoModelCfgByobNetinterleave_blocksbottleg?)typedcsgsbri)r self_attni)typesrrrrrri@tieredmaxpoolT bottleneck)blocksstem_chs stem_type stem_poolfixed_input_sizeself_attn_layerself_attn_kwargs)reveryrrrrrigZd;O?)rrrrrsiluise) r r!r"r# act_layer num_features attn_layerr%r&)r r!r"r#r,r$r%r&eca)dim_head) r r!r"r#r$r,r.r%r&g? 7x7halo) block_size halo_size)r r!r"r#r%r&)r9r: num_heads) rr(rrrrrr%r&)r r!r"r#r,r%r&)r9r:r3)r r!r"r#r,r.r%r&lambda )rN0` i  )rd_ratio) bottle_in linear_outgQ?)r9r:qk_ratio) r r!r# downsampler-r,r. attn_kwargs block_kwargsr%r& )rrrrrrr%r&)r:)r r!r"r#r,) botnet26t sebotnet33ts botnet50tseca_botnext26ts halonet_h1 halonet26t sehalonet33ts halonet50tseca_halonext26tslambda_resnet26tlambda_resnet50tslambda_resnet26rpt_256 haloregnetz_blamhalobotnet50tshalo2botnet50tsvariant cfg_variant pretrainedreturnc dtt||f|s t|nt|tdd|S)a<Create a Bring-Your-Own-Attention network model. Args: variant: Model variant name. cfg_variant: Config variant name if different from model variant. pretrained: Load pretrained weights. **kwargs: Additional model arguments. Returns: Instantiated ByobNet model. T)flatten_sequential) model_cfg feature_cfg)r r model_cfgsdict)r\r]r^kwargss O/var/www/teggl/fontify/venv/lib/python3.12/site-packages/timm/models/byoanet.py_create_byoanetrh s@ * -8*W%j>UD1    urlc 4|dddddttddddd |S) zGenerate default model configuration. Args: url: URL for pretrained weights. **kwargs: Override default configuration values. Returns: Model configuration dictionary. ir'rmrGrGgffffff?bicubiczstem.conv1.convzhead.fcF) rj num_classes input_size pool_sizecrop_pct interpolationmeanstd first_conv classifierr$min_input_sizer)rjrfs rg_cfgrzs74}SY9%.B'y!]     rizshttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/botnet26t_c1_256-167a0e9f.pthztimm/)r'rr)r8r8)rj hf_hub_idr$rqrrzxhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/sebotnet33ts_a1h2_256-957e3c3e.pthgGz?)rjr{r$rqrrrs)r$rqrrzxhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/eca_botnext26ts_c_256-95a898f6.pth)rqrrryzuhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/halonet26t_a1h_256-3083328c.pth)rjr{rqrrryzthttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/sehalonet33ts_256-87e053f9.pth)rjr{rqrrryrszwhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/halonet50ts_a1h2_256-f3a3daee.pthzyhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/eca_halonext26ts_c_256-06906299.pthzyhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/lambda_resnet26t_c_256-e5a5c857.pth)r'r4r4)rjr{ryrqrrrsz|https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/lambda_resnet50ts_a1h_256-b87370f7.pth)rjr{ryrqrrz{https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/lambda_resnet26rpt_c_256-ab00292d.pthzxhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/haloregnetz_c_raa_256-c8ad7616.pth)?r|r|z stem.convrlrn) rjr{rurvrwrqrrryrsz}https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/lamhalobotnet50ts_a1h2_256-fe3d9445.pthz{https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/halo2botnet50ts_a1h2_256-fd9c11a3.pth)zbotnet26t_256.c1_in1kzsebotnet33ts_256.a1h_in1kzbotnet50ts_256.untrainedzeca_botnext26ts_256.c1_in1kzhalonet_h1.untrainedzhalonet26t.a1h_in1kzsehalonet33ts.ra2_in1kzhalonet50ts.a1h_in1kzeca_halonext26ts.c1_in1kzlambda_resnet26t.c1_in1kzlambda_resnet50ts.a1h_in1kzlambda_resnet26rpt_256.c1_in1kzhaloregnetz_b.ra3_in1kzlamhalobotnet50ts_256.a1h_in1kzhalo2botnet50ts_256.a1h_in1kc @|jddtdd|i|S)z4 Bottleneck Transformer w/ ResNet26-T backbone. img_sizerr^) botnet26t_256rM setdefaultrhr^rfs rgrrrs( j#&  YJ YRX YYric tdd|i|S)zY Bottleneck Transformer w/ a ResNet33-t backbone, SE attn for non Halo blocks, SiLU, r^)sebotnet33ts_256rNrhrs rgrrzs  _* _X^ __ric @|jddtdd|i|S)z> Bottleneck Transformer w/ ResNet50-T backbone, silu act. r~rr^)botnet50ts_256rOrrs rgrrs( j#&  [j [TZ [[ric @|jddtdd|i|S)z> Bottleneck Transformer w/ ResNet26-T backbone, silu act. r~rr^)eca_botnext26ts_256rPrrs rgrrs) j#&  ePZ e^d eeric tdd|i|S)za HaloNet-H1. Halo attention in all stages as per the paper. NOTE: This runs very slowly! r^)rQrrs rgrQrQs  IJ I& IIric tdd|i|S)zJ HaloNet w/ a ResNet26-t backbone. Halo attention in final two stages r^)rRrrs rgrRrRs  IJ I& IIric tdd|i|S)zc HaloNet w/ a ResNet33-t backbone, SE attn for non Halo blocks, SiLU, 1-2 Halo in stage 2,3,4. r^)rSrrs rgrSrS  Lz LV LLric tdd|i|S)zT HaloNet w/ a ResNet50-t backbone, silu act. Halo attention in final two stages r^)rTrrs rgrTrTs  JZ J6 JJric tdd|i|S)zT HaloNet w/ a ResNet26-t backbone, silu act. Halo attention in final two stages r^)rUrrs rgrUrU  O* O OOric tdd|i|S)zG Lambda-ResNet-26-T. Lambda layers w/ conv pos in last two stages. r^)rVrrs rgrVrVrric tdd|i|S)zR Lambda-ResNet-50-TS. SiLU act. Lambda layers w/ conv pos in last two stages. r^)rWrrs rgrWrWs  P: P PPric @|jddtdd|i|S)zN Lambda-ResNet-26-R-T. Lambda layers w/ rel pos embed in last two stages. r~rr^)rXrrs rgrXrXs' j#&  U Uf UUric tdd|i|S)z Halo + RegNetZ r^)rYrrs rgrYrYrric tdd|i|S)z3 Combo Attention (Lambda + Halo + Bot) Network r^)lamhalobotnet50ts_256rZrrs rgrrs  iT^ ibh iiric tdd|i|S)z1 Combo Attention (Halo + Halo + Bot) Network r^)halo2botnet50ts_256r[rrs rgrrs  ePZ e^d eeri)NF)r))F))__doc__typingrrr timm.datarr_builderr _registryr r byobnetr rrr__all__rerdstrboolrhrz default_cfgsrrrrrQrRrSrTrUrVrWrXrYrrrirgrs  '&A*<II  XcQ1 F XcQ1 F $;qDARSX\ ] [AqT J  $  XcQ1 F $;A3!sVW\]bf g $;A3!tWX]^cg h  qDA! F  $  XcQ1 F $;1STUZ[`d e $;qDARSX\ ] $;qDARSX\ ]  $ XcQ2$ G XcQ2$ G $;qDARTY] ^ [Ard K  $r*" [AqQ3 G [AaC H [B#qS I [AaC H  a8  XcQ1 F XcQ1 F $;qDARSX\ ] [AqT J  a8  XcQ1 F $;A3!sVW\]bf g $;A3!tWX]^cg h  qDA! F  a8  XcQ1 F -Q!saARV &VWcd9e g $;qDARSX\ ] $;qDARSX\ ]  a8 ! XcQ2$ G XcQ2$ G $;qDARTY] ^ [Ard K  a"E ! XcQ1 F XcQ1 F $;qDARSX\ ] [AqT J    " XcQ1 F $;1STUZ[`d e $;qDARSX\ ] $;qDARSX\ ]    ' XcQ1 F XcQ1 F $;qDARSX\ ] [AqT J     XbA" C XbA" C $;1cUV[]bc d  qC1 B  $'DT:a$G!(" XcQ1 F -cQ1 (4": G -daA$ &9J L -daA$ ,tv G  !$ XcQ1 F -cQ1 &9J L -daA$ &9J L -daA$ ,tv G  !wm `Sx}QUls(cT#s(^(%! B-6K"& G-6TX"Z!%-6!K#' G-6$K !MV\ij D F=R# C F=[_a! F F=[_a!% H F=[_!a !% H$&[_!a#' K$&#R'+ J-6TX'Z # G /=F[hsw y '+ L-6'K%) J-6%Ku>&> BZdZZZ``G`` \t\'\\fDfwffJ4JgJJJ4JgJJ MdMMM KDKwKK PPGPP PPGPP Q$QWQQ VtV'VVMdMMM jdjjj fDfwffri