hbddlmZmZmZmZddlZddlmcmZ ddlmZddl m Z ddl m Z mZddlmZddlmZdd lmZdd lmZmZmZdd lmZmZdd lmZmZdd lm Z ddl!m"Z"m#Z#m$Z$ddl%m&Z&ddl'm(Z(e#r ddl)m*Z*ddl+m,Z,e$jZe.Z/GddedZ0GddejbZ2GddejbZ3GddejbZ4GddejbZ5Gd d!ejbZ6d"Z7d@d#Z8d$ejrd%e:d&ejrfd'Z; dAd(ejbd)ejrd*ejrd+ejrd,eejrd-eGd2d3eZ?e"Gd4d5eZ@Gd6d7ejbZAe"Gd8d9e@ZB dBd:eejreCejrdfd;ee:d,eejrd&eejre:ffd<ZDGd=d>e@eZEgd?ZFy)C)CallableOptional TypedDictUnionN)nn)ACT2FN)Cache DynamicCache)GenerationMixin)AttentionMaskConverter)GradientCheckpointingLayer)BaseModelOutputWithPastMoeCausalLMOutputWithPastMoeModelOutputWithPast)ROPE_INIT_FUNCTIONSdynamic_rope_update)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)auto_docstringis_torch_flex_attn_availablelogging)deprecate_kwarg)GraniteMoeSharedConfig) BlockMask)make_flex_block_causal_maskceZdZUdZej ed<ej ed<eed<eed<ejed<y)GraniteFlashAttentionKwargsa Keyword arguments for advanced Flash Attention, causal-conv1d, and mamba_ssm kernel usage. Use cases include padding-free training and fewer `torch.compile` graph breaks. Attributes: cu_seq_lens_q (`torch.LongTensor`) Gets cumulative sequence length for query state. cu_seq_lens_k (`torch.LongTensor`) Gets cumulative sequence length for key state. max_length_q (`int`): Maximum sequence length for query state. max_length_k (`int`): Maximum sequence length for key state. seq_idx (`torch.IntTensor): Index of each packed sequence. cu_seq_lens_q cu_seq_lens_k max_length_q max_length_kseq_idxN) __name__ __module__ __qualname____doc__torch LongTensor__annotations__int IntTensor|/var/www/html/eduruby.in/venv/lib/python3.12/site-packages/transformers/models/granitemoeshared/modeling_granitemoeshared.pyr r 3s7"###### __r0r F)totalc`eZdZdZdeffd ZdejdejfdZxZ S)GraniteMoeSharedMLPz~ MLP layer for shared experts Args: config: Configuration object with model hyperparameters. configc`t||j|_|j|_t |j |_tj|j|jdzd|_ tj|j|jd|_ y)NFbias) super__init__ hidden_size input_sizeshared_intermediate_sizer hidden_act activationrLinear input_linear output_linearselfr5 __class__s r1r;zGraniteMoeSharedMLP.__init__Us  ,,!:: !2!23IIdoot7G7G!7KRWXYYt'7'7uUr0 hidden_statesreturnc|j|}|jdd}|j|d|dz}|j|}|S)Nr7dimrr)rBchunkr@rC)rErGchunked_hidden_statess r1forwardzGraniteMoeSharedMLP.forward^s^))-8 - 3 3A2 3 >(=a(@ADYZ[D\\ **=9 r0) r&r'r(r)rr;r*TensorrO __classcell__rFs@r1r4r4Ls2V5VU\\ellr0r4c,eZdZdfd ZdZdZxZS)GraniteMoeSharedRMSNormct|tjt j ||_||_y)zF GraniteMoeSharedRMSNorm is equivalent to T5LayerNorm N)r:r;r Parameterr*onesweightvariance_epsilon)rEr<epsrFs r1r;z GraniteMoeSharedRMSNorm.__init__gs1 ll5::k#:; #r0c"|j}|jtj}|j dj dd}|tj ||jzz}|j|j|zS)Nr7rJT)keepdim) dtypetor*float32powmeanrsqrtrYrX)rErG input_dtypevariances r1rOzGraniteMoeSharedRMSNorm.forwardosy#)) %((7  $$Q',,R,>% Ht?T?T4T(UU {{]--k:::r0c^t|jjd|jS)Nz, eps=)tuplerXshaperY)rEs r1 extra_reprz"GraniteMoeSharedRMSNorm.extra_reprvs* ))*+6$2G2G1HIIr0)gư>)r&r'r(r;rOrhrQrRs@r1rTrTfs$;Jr0rTc6eZdZdedededdffd ZdZxZS)GraniteMoeSharedParallelExperts num_expertsr= output_sizerHNct|tjt j ||||_||_||_||_ y)a Initialize the GraniteMoeSharedParallelExperts module. The experts weights are stored in [num_experts, output_size, input_size] format. Such that it's compatible with many MoE libraries, such as [Megablock](https://github.com/databricks/megablocks) and [ScatterMoE](https://github.com/shawntan/scattermoe), as well as the [MoE kernel](https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/layers/fused_moe/fused_moe.py) used in vllm. Args: num_experts (int): Number of experts. input_size (int): Size of the input. output_size (int): Size of the output. N) r:r;rrVr*emptyrXrkr=rl)rErkr=rlrFs r1r;z(GraniteMoeSharedParallelExperts.__init__{sD" ll5;;{K#TU &$&r0c|j|d}g}t|jD]7}|jt j |||j |9tj|d}|S)a Forward pass of the GraniteMoeSharedParallelExperts module. Args: inputs (Tensor): Input tensor. expert_size: Expert size information. Returns: Tensor: Output tensor. rrK) splitrangerkappendFlinearrXr*cat)rEinputs expert_size input_list output_listiresultss r1rOz'GraniteMoeSharedParallelExperts.forwardsq\\+1\5  t''( HA   qxx 1 t{{1~F G H))KQ/r0r&r'r(r-r;rOrQrRs@r1rjrjzs)'C'S's't'.r0rjc2eZdZdededeffd ZdZxZS)GraniteMoeSharedTopKGatingr=rktop_kct|||_||_||_t j ||d|_y)a Initialize the top-k gating mechanism. Args: input_size (`int`): Size of the input. num_experts (`int`): Number of experts. top_k (`int`): Number of top experts to select. Fr8N)r:r;rkr=rrrAlayer)rEr=rkrrFs r1r;z#GraniteMoeSharedTopKGating.__init__s: &$ YYz;UC r0c|j|j}|j|jd\}}t j |dj |}t j|jd|jg|j|j}|jd|d}|jjd}|j}|j!} | j#d\} } | j%|jd} |j!}|| } | | | ||fS)NrrKrr]devicetrunc) rounding_mode)rfloattopkrr*softmaxtype_aszerossizerkr]rscatterlongsumtolistflattensortdiv)rErGlogits top_k_logits top_k_indices top_k_gatesrgatesrw top_k_experts_index_sorted_experts batch_index batch_gatess r1rOz"GraniteMoeSharedTopKGating.forwards.M*002&,kk$**!k&D# mmmLa8@@O    a $"2"2 3;;L;LU`UgUg  a2jjl&&q) "((* &--/ "/"4"4Q"7 *..tzz.Q "))+ !"67 #[+{FRRr0r|rRs@r1r~r~s'D3DSDD&Sr0r~c.eZdZdZdeffd ZdZxZS)GraniteMoeSharedMoEz A Sparsely gated mixture of experts layer with 1-layer Feed-Forward networks as experts. Args: config: Configuration object with model hyperparameters. r5ct||j|_|j|_t |j |_t|j|j|jdz|_ t|j|j|j|_ t|j|j|j|_y)Nr7)r=rkr)r:r;r<r=intermediate_sizer r?r@rjnum_local_expertsrBrCr~num_experts_per_tokrouterrDs r1r;zGraniteMoeSharedMoE.__init__s  ,,!33 !2!23;  $ $doot7G7G!7K =  $ $d&6&6 100,,  r0c|j\}}}|jd|}|j|\}}}}} ||} |j| |} | j dd} |j | d| dz} |j | |} | |dddfz} tj||z|jf| j| j}|jd|| }|j|||j}|| fS)a Forward pass of the mixture of experts layer. Args: layer_input (Tensor): Input tensor. Returns: Tensor: Output tensor. Tensor: Router logits. rJr7rKrrNr)rreshaperrBrMr@rCr*rr=r]r index_addview)rE layer_inputbszlengthemb_sizerrrrw router_logits expert_inputsrGrNexpert_outputsr layer_outputs r1rOzGraniteMoeSharedMoE.forwards!, 0 0 2VX!))"h7 BF++kBZ?; [-#K0 ))-E - 3 3A2 3 >(=a(@ADYZ[D\\ ++M;G'+ag*>> S6\4??;>CWCW`n`u`uvq+~F #((fdooF ]**r0)r&r'r(r)rr;rOrQrRs@r1rrs 5 &+r0rc|dd|jddzf}|d|jddzdf}tj| |fdS)z*Rotates half the hidden dims of the input..NrJr7rK)rgr*ru)xx1x2s r1 rotate_halfrsZ 3"!''"+"" " #B 3 q " " #B 99rc2YB ''r0c|j|}|j|}||zt||zz}||zt||zz}||fS)aApplies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`, *optional*): Deprecated and unused. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. ) unsqueezer)qkcossin position_ids unsqueeze_dimq_embedk_embeds r1apply_rotary_pos_embrsY( -- &C -- &C3w;q>C/0G3w;q>C/0G G r0rGn_reprHc|j\}}}}|dk(r|S|dddddddddfj|||||}|j|||z||S)z This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) rN)rgexpandr)rGrbatchnum_key_value_headsslenhead_dims r1 repeat_kvr5so 2?1D1D.E h z!!Qa"23::5BUW\^bdlmM  (;e(CT8 TTr0modulequerykeyvalueattention_maskscalingdropoutc Tt||j}t||j} tj||j dd|z} |#|ddddddd|j df} | | z} t jj| dtjj|j} t jj| ||j} tj| | } | j ddj} | | fS)Nr7rrJ)rLr])ptrainingr)rnum_key_value_groupsr*matmul transposergr functionalrr_r^r]rr contiguous) rrrrrrrkwargs key_states value_states attn_weights causal_mask attn_outputs r1eager_attention_forwardrAs3 ; ;Q5RR S0)S0)r0rceZdZdedeffd Zeddd ddejd e ejd e ejde e d e e d e e d e ejde e de e ejejfdeede ej e e ej ej fffdZxZS)GraniteMoeSharedDecoderLayerr5rct||j|_t|||_|j dkDrt ||_t|j|j|_ t|j|j|_ |j|_ |jdk(rd|_yt||_y)N)r5rrrZ)r:r;r<r self_attnrrblock_sparse_moerT rms_norm_epsinput_layernormpost_attention_layernormresidual_multiplierr>r4 shared_mlprs r1r;z%GraniteMoeSharedDecoderLayer.__init__s !--2&IV  # #a '$7$?D !6v7I7IvObObc(?@R@RX^XkXk(l%#)#=#= "("A"AQ"F$L_`fLgr0rrrrrGrroutput_attentionsrroutput_router_logitsrrrHc l|} |j|}|jd|||||||| d| \}} | ||jzz}|} |j|}|j |\} }|j | }n| |j |z}~ | ||jzz}|f}|r|| fz }|r||fz }|S)aD Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`, *optional*): attention mask of size `(batch_size, sequence_length)` if flash attention is used or `(batch_size, 1, query_sequence_length, key_sequence_length)` if default attention is used. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). past_key_values (`Tuple(torch.FloatTensor)`, *optional*): cached past key and value projection states cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence output_router_logits (`bool`, *optional*): Whether or not to return the logits of all the routers. They are useful for computing the router loss, and should not be returned during inference. position_embeddings (`tuple[torch.FloatTensor, torch.FloatTensor]`, *optional*): Tuple containing the cosine and sine positional embeddings of shape `(batch_size, seq_len, head_dim)`, with `head_dim` being the embedding dimension of each attention head. kwargs (`dict`, *optional*): Arbitrary kwargs. Can be used to provide `GraniteFlashAttentionKwargs` for padding-free training and/or improve torch.compile performance. )rGrrrrrrrr/)r r rrr r)rErGrrrrrrrrrresidualself_attn_weightsmoe_hidden_statesroutputss r1rOz$GraniteMoeSharedDecoderLayer.forwardsN!,,]; ,:4>> , ')%+/) 3 ,  , ( (!=43K3K#KK !55mD +/+@+@+O(= ?? "-M- 0NNM  =43K3K#KK  "  )+ +G   ' 'Gr0)NNNFFNFN)r&r'r(rr-r;rr*rPrr+r rrfrr FloatTensorrOrQrRs@r1rrsH h5 h# h%0A6R2637+/,1$)59/4KOO||O!.Ou//0 O "% O $D> OD>O!!1!12O'tnO&eELL%,,,F&GHO45O u  (51B1BEDUDU1U+V"WW XOSOr0rcJeZdZUeed<dZdZdgZdgZdZ dZ dZ fdZ xZ S)GraniteMoeSharedPreTrainedModelr5modelTrrFct||t|tr<|jj j d|jjyy)Nr)rastd) r: _init_weights isinstancerjrXdatanormal_r5initializer_range)rErrFs r1rz-GraniteMoeSharedPreTrainedModel._init_weights"sG f% f= > MM   & &CT[[5R5R & S ?r0)r&r'r(rr,base_model_prefixsupports_gradient_checkpointing_no_split_modules_skip_keys_device_placement_supports_flash_attn_supports_sdpa_can_compile_fullgraphrrQrRs@r1rrsD ""&*#78#4"5N"TTr0rc~eZdZUejed<ddeffd Zeje dZ xZ S)GraniteMoeSharedRotaryEmbeddinginv_freqr5ct|t|drUt|jt r;|jj d|jj d|_nd|_|j|_ |j|_ ||_ t|j|_ |j|j|\}|_|jd|d|j |_y)N rope_scaling rope_typetypedefaultr,F) persistent)r:r;hasattrrr.dictgetr/max_position_embeddingsmax_seq_len_cachedoriginal_max_seq_lenr5r rope_init_fnattention_scalingregister_bufferr,original_inv_freq)rEr5rr,rFs r1r;z(GraniteMoeSharedRotaryEmbedding.__init__+s  6> *z&:M:Mt/T#0044[&BUBUBYBYZ`BabDN&DN"("@"@$*$B$B! /?+/+<+>@GG HZHZ[\H]_acdehhijiqiqr ,QaZ 8 > > @'1!((--'E!((--[`J`ahhmmfk ^^ U C 5&,,.1F1L1L1NNYYZ[]^_E))UEN3C'')d444C'')d444C  5 vvAGGv$cff177f&;;;  5 5s BF%%F.r) r&r'r(r*rPr,rr;no_gradrrOrQrRs@r1r+r+(s>ll/5/"U]]_ < ,>FDWDWX &+#$*$?$?!!--33((DNN: '-'E'E$ ++'-'E'E$EIEaEaekEk9&Aqu ! osF input_idsrrr inputs_embedsrroutput_hidden_statesr return_dictrrHc v||n|jj}||n|jj}||n|jj}| | n|jj} |du|duz r t d|j r%|jr|rtjdd}||j|}||jz}t|tdtfs t d|r|t|j}| F||j!nd} t#j$| | |j&dz|j(} || j+d}|j-||| ||}|}d}|j.|j/||}|rd nd}|rd nd}| rd nd}|j0D]7}|r||fz }|||||||| | | }|d}|r ||dfz }| s/||d fz }9|j3|}|r||fz }| st5d ||||fDSt7||||| S)Nz:You must specify exactly one of input_ids or inputs_embedszX`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.FzBThe `past_key_values` should be either a `Cache` object or `None`.)r5rrrr/)rrrrrrrrrJc3&K|] }|| ywrr/).0vs r1 z0GraniteMoeSharedModel.forward..sbcbos)last_hidden_staterrG attentionsr)r5rr^ruse_return_dictrrVrrrrQrWrr0r r get_seq_lengthr*arangergrr_update_causal_maskrZrTrUrfr)rEr\rrrr]rrr^rr_rrpast_seen_tokensrrGrall_hidden_statesall_self_attnsall_router_logits decoder_layer layer_outputss r1rOzGraniteMoeSharedModel.forwardgs 2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK?ab b 0*$++>O  !CRC^==?de "\\ "2]5H5H5K"KTaThThN  )33A6L.. M>?L] & " ?? &"&//-"N #7BD0d"6BD![[ :M#!m%55!)*) /"3#-%9$7 M*!,M =#3"55#!mB&7%99!- :0 -0  -!1 1 )?T]n%>>*'7 MM  ""&,,Q/ !+??AMnell;$$R(%7!; PP +')#))!, Q  KK , , 6*%%**.DD%  E*..I0CCKQZ[Kr0ryrzr]r{c ||jdk(r|}|Stj|j}tj||f|||j }|dk7rtj |d}|tj||j |jddkDz}|ddddddfj|ddd}||j}|jd} |ddddddd| f|ddddddfj|j z} | dk(} |ddddddd| fj| ||ddddddd| f<|S) aM Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. N) fill_valuer]rr)diagonalrarJr)rLr*rrfullrtriurjrrclonergr^ masked_fill) rryrzr]rr{rrr mask_length padding_masks r1rzKGraniteMoeSharedModel._prepare_4d_causal_attention_mask_with_cache_positions<  %.*<*<*>!*C(K*' E*..I** -0Ye\j\q\qK!##jjqA 5<< n>S>STWeWmWmnprsWtt tK%dD!Q&67>>z1bRTUK))//1 ,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c )6 Aq!\k\12r0) NNNNNNNNNNN)F)r&r'r(rr;rrr*r+rPrr listrrrfrrOrk staticmethodr-r]rrQrRs@r1rJrJLs52151537KO59$(,0/3/3&*59l E,,-l !.l u//0 l "%tE4E4E/F(F"GH l   1 12 l D>l $D>l 'tnl 'tnl d^l !!1!12l  u-- .l l h#( BellK78BllB B  B  BH4 444{{ 4  4  44r0rJ gate_logitsrkc|t|tsyt|trC|dj}tj|Dcgc]}|j |c}d}tj jjd}tj||d\}} tj jj| |} |>tj| jd} tj|d} n1|j\} }|jd| |zz}|dddddddfj|| |||fjd||j }tj | j|zdtj |dz } |ddddddfj|| ||jdfjd|jdj |}tj ||zdtj |dz } |jj"|jj"nd}|jdt%|z}tj | dd|||jdzf| j'dz}||zScc}w)a Computes auxiliary load balancing loss as in Switch Transformer - implemented in Pytorch. See Switch Transformer (https://huggingface.co/papers/2101.03961) for more details. This function implements the loss function presented in equations (4) - (6) of the paper. It aims at penalizing cases where the routing between experts is too unbalanced. Args: gate_logits: Logits from the `gate`, should be a tuple of model.config.num_hidden_layers tensors of shape [batch_size X sequence_length, num_experts]. num_experts: Number of experts top_k: The number of experts to route per-token, can be also interpreted as the `top-k` routing parameter. attention_mask (`torch.Tensor`, *optional*): The attention_mask used in forward function shape [batch_size X sequence_length] if not None. Returns: The auxiliary loss. NrrKrJr)rrfrr*rur^rrrrone_hotrarrgrrrindexr-r)rrkrrcompute_device layer_gateconcatenated_gate_logitsrouting_weightsrselected_experts expert_masktokens_per_expertrouter_prob_per_expertr{ryrSexpert_attention_mask router_per_expert_attention_mask device_indexrank overall_losss r1load_balancing_loss_funcrRs:*[%"@+u%$Q..#(99^i-jPZjmmN.K-jpq#r hh))112JPR1SO**_eDA((%%--.> LK!JJ{'8'8':B"'O!C&4&:&:# O4::1=*B^_ 4AtT1 2 V& OUKX Y WR , R  "IIk&7&7&9>> from transformers import AutoTokenizer, GraniteMoeSharedForCausalLM >>> model = GraniteMoeSharedForCausalLM.from_pretrained("ibm/PowerMoE-3b") >>> tokenizer = AutoTokenizer.from_pretrained("ibm/PowerMoE-3b") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you." ```N) r\rrrr]rrr^rr_rrrOrJr)lossaux_lossrrrGrgrr/)r5rrr^rhrrr-slicerlogits_scalingr loss_functionrOrrrkrrr^rrrrGrg)rEr\rrrr]rrrr^rr_rrrrrG slice_indicesrrroutputs r1rOz#GraniteMoeSharedForCausalLM.forwards*P2C1N-TXT_T_TqTq$8$D $++JjJj %9$D $++JjJj &1%SV8W~ot4]k mA}a,?@A$++444  \\^F%4%% ;;11 D /)4%%'"+  (( H !11HKK 4LLLY,F#"v-'+'7D7V# CV C(#33!//))!//  r0) NNNNNNNNNNNNr)r&r'r(_tied_weights_keysrr;rrr*r+rPrr rrrr-rfrrOrQrRs@r1rrsw*+ 5 151537KO59-1$(,0/3/3&*5934k E,,-k !.k u//0 k "%tE4E4E/F(F"GH k   1 12 k ))*k D>k $D>k 'tnk 'tnk d^k !!1!12k c5<</0k u// 0!k k r0r)rrJr)Nr)r)Nr7N)Gtypingrrrrr*torch.nn.functionalrrrs activationsr cache_utilsr r generationr modeling_attn_mask_utilsr modeling_layersrmodeling_outputsrrrmodeling_rope_utilsrrmodeling_utilsrrprocessing_utilsrutilsrrrutils.deprecationrconfiguration_granitemoesharedr!torch.nn.attention.flex_attentionrintegrations.flex_attentionr get_loggerr&rr Moduler4rTrjr~rrrrPr-rrrrrrr+rJrfrr__all__r/r0r1rsY,87 !.)>9jjKF&JJ0B !;J   H %)52"))4JbiiJ(*bii*Z-S-S`9+"))9+x(6 UU\\ U# U%,, U&% II% <<% % << % U\\* %  %%:T) T)n^#=^BToTT"!?S&#S&U\\* S&  5<<  S&l| "A?| ~ fr0