hYdZddlmZddlmZmZddlZddlm Z ddl m Z ddl m Z dd lmZmZmZdd lmZej(eZerddlZed Gd de ZdgZy)z Processor class for SAM2. )deepcopy)OptionalUnionN) ImageInput)ProcessorMixin) BatchEncoding) TensorTypeis_torch_availablelogging)requires)torch)backendsceZdZdZdgZdZd deedeffd Z d!de de d ee e e e e e e jfd ee e e e ee jfd ee e e e e e jfd ee e e e e jfd ee eefdefdZ d"deddddfdZd#dZd$dZd%dZdZdZ d$de e jej4e fdedededeede f dZd&dZ d'dZxZS)( Sam2Processora Constructs a SAM2 processor which wraps a SAM2 image processor and an 2D points & Bounding boxes processor into a single processor. [`Sam2Processor`] offers all the functionalities of [`Sam2ImageProcessorFast`] and [`Sam2VideoProcessor`]. See the docstring of [`~Sam2ImageProcessorFast.__call__`] and [`~Sam2VideoProcessor.__call__`] for more information. Args: image_processor (`Sam2ImageProcessorFast`): An instance of [`Sam2ImageProcessorFast`]. target_size (`int`, *optional*): The target size (target_size, target_size) to which the image will be resized. point_pad_value (`int`, *optional*, defaults to -10): The value used for padding input points. image_processorSam2ImageProcessorFast target_sizepoint_pad_valuec t||fi|||_|||_y|jjd|_y)Nheight)super__init__rrsizer)selfrrrkwargs __class__s f/var/www/html/eduruby.in/venv/lib/python3.12/site-packages/transformers/models/sam2/processing_sam2.pyrzSam2Processor.__init__:sC 3F3.*5*A;tG[G[G`G`aiGjimagessegmentation_maps input_points input_labels input_boxesoriginal_sizesreturn_tensorsreturnc ||j|f||d|} nU|Ht|tjr|j j }t d|i|} n td| d}|0t|dk7r"t|t|k7r td||||j|dd d d } |j|d dd} |j|d ddd } | |j| dd } | |j| dd }| |j| dd | |  k7r td| -t| d k\rtfd| Dr td| b|j|  d gz}tj|tj}|j||d| j!d|i| J|j| }tj|tj"}| j!d|i| Ltj| tj}|j||d| j!d|i| S)a This method uses [`Sam2ImageProcessorFast.__call__`] method to prepare image(s) for the model. It also prepares 2D points and bounding boxes for the model if they are provided. Args: images (`ImageInput`, *optional*): The image(s) to process. segmentation_maps (`ImageInput`, *optional*): The segmentation maps to process. input_points (`list[list[list[list[float]]]]`, `torch.Tensor`, *optional*): The points to add to the frame. input_labels (`list[list[list[int]]]`, `torch.Tensor`, *optional*): The labels for the points. input_boxes (`list[list[list[float]]]`, `torch.Tensor`, *optional*): The bounding boxes to add to the frame. original_sizes (`list[list[float]]`, `torch.Tensor`, *optional*): The original sizes of the images. return_tensors (`str` or `TensorType`, *optional*): The type of tensors to return. **kwargs: Additional keyword arguments to pass to the image processor. Returns: A [`BatchEncoding`] with the following fields: - `pixel_values` (`torch.Tensor`): The processed image(s). - `original_sizes` (`list[list[float]]`): The original sizes of the images. - `reshaped_input_sizes` (`torch.Tensor`): The reshaped input sizes of the images. - `labels` (`torch.Tensor`): The processed segmentation maps (if provided). - `input_points` (`torch.Tensor`): The processed points. - `input_labels` (`torch.Tensor`): The processed labels. - `input_boxes` (`torch.Tensor`): The processed bounding boxes. N)r!r&r%) tensor_typez0Either images or original_sizes must be providedz{original_sizes must be of length 1 or len(images). If you are passing a single image, you must pass a single original_size.pointsz;[image level, object level, point level, point coordinates])expected_depth input_nameexpected_formatexpected_coord_sizerlabelsz([image level, object level, point level])r.r/r0boxesz)[image level, box level, box coordinates]zbInput points and labels have inconsistent dimensions. Please ensure they have the same dimensions.c3@K|]}t|dkyw)r*N)len).0 img_boxesboxes_max_dimss r z)Sam2Processor.__call__..s[is9~q(99[szInput boxes have inconsistent dimensions that would require padding, but boxes cannot be padded due to model limitations. Please ensure all images have the same number of boxes.)dtypeT)preserve_paddingr"r#is_bounding_boxr$)r isinstancerTensorcputolistr ValueErrorr5_validate_single_input_get_nested_dimensionsany_pad_nested_listtensorfloat32_normalize_tensor_coordinatesupdateint64)rr r!r"r#r$r%r&rencoding_image_processorprocessed_pointsprocessed_labelsprocessed_boxespoints_max_dimslabels_max_dims padded_points final_points padded_labels final_labels final_boxesr8s @r__call__zSam2Processor.__call__?sV  ';t';';("3-( ( $  '.%,,7!/!3!3!5!'46F5Wes't $OP P22BC  #n"5":s>?RVYZ`Va?aN   #|'?;CZ#:: # ]$% ;   $:: # J ;   #99 " K$% :O +"&"="=>N"OPRQR"S+"&"="=>N"OPRQR"S*!%!rr?r5shapenumpyrA_convert_to_nested_listnpndarraylistintr[rBtype)rdatar. current_depthitems rrfz%Sam2Processor._convert_to_nested_listsO < dELL ) 22c$**o6Jzz|**,,jnobf44T>=[\K\]oo bjj ) 22c$**o6J{{}$jnobf44T>=[\K\]oo d #. koobf44T>=[\K\]oo sEl +K6tDzlCD D!p pps EE E"c|g}t|ts|St|dk(r|jt|nt |dt||d<t|dkDrw|D]r}t|ts|j |}t |D]@\}}|dzt|k\r|j|)t ||dz|||dz<Bt|S)a` Get the maximum dimensions at each level of nesting. Args: nested_list (`list`): Nested list structure. max_dims (`list`, *optional*): Current maximum dimensions (for recursion). Returns: `list`: A list of maximum dimensions for each nesting level. rr*)r>rir5appendmaxrD enumerate)r nested_listmax_dimsrnsub_dimsidims rrDz$Sam2Processor._get_nested_dimensionss  H+t,O x=A  OOC , -hqk3{+;riextend_create_empty_nested_structurerangerrF) rrs target_dims current_level pad_value current_sizer template_dimstemplate_rvs rrFzSam2Processor._pad_nested_list,s"  ,,I C , , +t,&-K;' !-0  C ,q0 0    {kL.HI Ja 3{#3a#77$/ 0A0C$DM#BB=R[\H!*{[9J-KKH""kT`F`@a#b1HX$6#bc!,MA,=,? @ >>}iX""k@R#S1HX$6#ST 3{+a/ /3{+, vk!nd3%)%:%:;q>;XehiXikt%uKN v$c $Ts FF ct|dk(r |g|dzSt|dDcgc]}|j|dd|c}Scc}w)a Create an empty nested structure with given dimensions filled with pad_value. Args: dims (`list`): The dimensions of the nested structure. pad_value (`int`): The value to fill the structure with. r*rN)r5r{rz)rdimsr~rs rrzz,Sam2Processor._create_empty_nested_structurejsT t9>;a( (V[\`ab\cVdeQRD77QR)Le eesAct|tr&t|dk(ryd|j|dzSt|tj t jfrt|jSy)z Get the nesting level of a list structure. Args: input_list (`list`): The list to get the nesting level of. rr*) r>rir5_get_nesting_levelrgrhrr?rd)r input_lists rrz Sam2Processor._get_nesting_levelysb j$ ':!#t..z!}== =  RZZ$> ?z''( (rrlr.r/r0r1c |yt|tjtjfry|j |k7r"t d|d|d|d|j d |4|jd|k7r"t d|d|d|jdd |j||St|tr@|j|}||k7rt d|d |d |d|d |j||Sy) a Validate a single input by ensuring proper nesting and raising an error if the input is not valid. Args: data (`torch.Tensor`, `np.ndarray`, or `list`): Input data to process. expected_depth (`int`): Expected nesting depth. input_name (`str`): Name of the input for error messages. expected_format (`str`): The expected format of the input. expected_coord_size (`int`, *optional*): Expected coordinate size (2 for points, 4 for boxes, None for labels). . NzInput z must be a tensor/array with z, dimensions. The expected nesting format is z. Got z dimensions.rZz as the last dimension, got .z must be a nested list with z( levels. The expected nesting format is z levels.) r>rr?rgrhndimrBrdrfrir)rrlr.r/r0r1rms rrCz$Sam2Processor._validate_single_inputs0 < dU\\2::6 7yyN* ZL(EnEUVBCRBSSYZ^ZcZcYddpq%0::b>%88$  ,IJ]I^^z{|F|FGI|J{KKLM//nE E dD ! 33D9M. ZL(D^DTT|~M}NNTUbTcckl//nE E "rc|r"||jk7}|jdd}tt|D]}||jdks|t|kr||n|d}|j |j ||||} |r5|} tj| j||| ||||<| ||<y)a Helper method to normalize coordinates in a tensor across multiple images. Args: tensor (`torch.Tensor`): Input tensor with coordinates. original_sizes (`list`): Original image sizes. is_bounding_box (`bool`, *optional*, defaults to `False`): Whether coordinates are bounding boxes. preserve_padding (`bool`, *optional*, defaults to `False`): Whether to preserve padding values (for points). rZT)rwkeepdimrr<N) rallr{r5rdrbrrwhere expand_as) rrGr%r=r;mask coord_maskimg_idxr]normalized_coordsimg_masks rrIz+Sam2Processor._normalize_tensor_coordinatess T111Db$7JS01 8Ga(;BSEX;Xw 7^lmn^o $($?$?$$fWo}Ve%@%!$)'2H&+kk **6'?;=NPVW^P_'F7O'8F7O 8rc H|jj|||||||fi|S)a- Remove padding and upscale masks to the original image size. Args: masks (`Union[List[torch.Tensor], List[np.ndarray]]`): Batched masks from the mask_decoder in (batch_size, num_channels, height, width) format. original_sizes (`Union[torch.Tensor, List[Tuple[int,int]]]`): The original sizes of each image before it was resized to the model's expected input shape, in (height, width) format. mask_threshold (`float`, *optional*, defaults to 0.0): Threshold for binarization and post-processing operations. binarize (`bool`, *optional*, defaults to `True`): Whether to binarize the masks. max_hole_area (`float`, *optional*, defaults to 0.0): The maximum area of a hole to fill. max_sprinkle_area (`float`, *optional*, defaults to 0.0): The maximum area of a sprinkle to fill. apply_non_overlapping_constraints (`bool`, *optional*, defaults to `False`): Whether to apply non-overlapping constraints to the masks. Returns: (`torch.Tensor`): Batched masks in batch_size, num_channels, height, width) format, where (height, width) is given by original_size. )rpost_process_masks) rmasksr%mask_thresholdbinarize max_hole_areamax_sprinkle_area!apply_non_overlapping_constraintsrs rrz Sam2Processor.post_process_maskss?F7t##66       -    r)Ni)NNNNNNN)F)r)N)rN)FF)TrrF)__name__ __module__ __qualname____doc__ attributesimage_processor_classrrjrrrrir[rr?strr r rWrbrfrDrFrzrrgrhrCrIr __classcell__)rs@rrr%s $$J4kXc]k\_k"(,UYMQNRKO;?B(B(&B(uT$tDK/@*A%BELL%PQR B( uT$tCy/%:ELL%HIJ B( eDd5k):$;U\\$IJK B(!tDK'8%,,'F!GHB(!sJ!78B( B(JX](6 <$EL#J<| f..2 0FELL"**d230F0F 0F  0F &c] 0F 0Fd!8N*/, rr)rcopyrtypingrrrerg image_utilsrprocessing_utilsrtokenization_utils_baser utilsr r r utils.import_utilsr get_loggerrloggerrr__all__rrrsr"%.4<<*   H % :e Ne e P  r