Upload folder using huggingface_hub

.gitattributes

@@ -48,3 +48,6 @@ src/groundingdino/.asset/hero_figure.png filter=lfs diff=lfs merge=lfs -text
 tool/GroundingDINO/.asset/GD_GLIGEN.png filter=lfs diff=lfs merge=lfs -text
 tool/GroundingDINO/.asset/GD_SD.png filter=lfs diff=lfs merge=lfs -text
 tool/GroundingDINO/.asset/hero_figure.png filter=lfs diff=lfs merge=lfs -text
+tutorial/img/click_segment.jpg filter=lfs diff=lfs merge=lfs -text
+tutorial/img/input_video.jpg filter=lfs diff=lfs merge=lfs -text
+tutorial/img/start_tracking.jpg filter=lfs diff=lfs merge=lfs -text

tutorial/tutorial for Image-Sequence input.md

@@ -0,0 +1,32 @@
+# Tutorial for Image-Sequence input
+
+## Zip the Image-Sequence as input for the WebUI.
+**The structure of test-data-seq.zip must be like this. Please confirm that the image names are in ascending order.**
+```
+- test-data-seq
+    - 000000.png
+    - 000001.png
+    - 000002.png
+    - 000003.png
+    ....
+    - 0000xx.png
+```
+**Note: Please ensure that the image naming method is in ascending alphabetical order.**
+
+## Use WebUI get test Image-Sequence data
+### 1. Switch to the `Image-Seq type input` tab.
+
+ <p align="center"><img src="./img/switch2ImgSeq.jpg" width = "600" height = "300" alt="switch2ImgSeq"/> </p>
+
+### 2. Upload the test dataset or use the provided examples directly.
+- Once the test dataset has finished uploading, the WebUI will automatically extract the first frame and display it in the `Segment result of first frame` component.
+- If you use the provided examples, you may need to manually extract the results by clicking the `extract` button.
+- Below are examples of how to upload an Image-sequence data.
+
+<p align="center"><img src="./img/upload_Image_seq.jpg" width = "600" height = "300"> <img src="./img/use_exa4ImgSeq.jpg" width = "600"></p>
+
+### 3. Select fps for the output video
+
+<p align="center"><img src="./img/select_fps.jpg" width = "600" height = "300"> </p>
+
+### 4. You can follow the [tutorial for WebUI-1.0-Version](./tutorial%20for%20WebUI-1.0-Version.md) to obtain your result.

tutorial/tutorial for WebUI-1.0-Version.md

@@ -0,0 +1,68 @@
+# Tutorial for WebUI 1.0 Version
+
+## Note: 
+- We recommend reinitializing SegTracker by clicking the `Reset button` after processing each video to avoid encountering bugs.
+- If the `SegTracker-Args` are changed, the SegTracker needs to be reinitialized by clicking the Reset button.
+- If the `Drawing board` does not display the image properly, you can refresh the Drawing board by clicking on the `refresh icon` located in the upper right corner of the Drawing board.
+- A video tutorial will be released in the next few days.
+## 1. About Components
+- `input video`: where the uploaded video is displayed for the user to view.
+- `Segment result of first frame`: where the segmentation result of the first frame is displayed for the user to view. Under the `Everything-Tab` and `Click-Tab`, users can interactively add a mask by clicking on the displayed result.
+- `Drawing board`: where users can circle the object they want to track. This component is only visible under the `Stroke-Tab`.
+- `SegTracker-Args`: used to adjust the parameters for initializing SegTracker.
+- `Undo`: used to undo a previously added point prompt or segment-everything operation.
+- `Reset`: used to reset all components and reinitialize SegTracker.
+- `Start Tracking`: used to begin tracking the objects selected by automatic/interactive methods in the video using SegTracker.
+- `Output video`: where the tracking results of the video are displayed for the user to view.
+- `Predicted masks`: show the predicted masks for each frame of the video.
+
+## 2. Upload your video
+- To upload a video, click on the `input video` component. Once uploaded, the `segment result of first frame` component will display the first frame of the video automatically. 
+- The examples for uploading a video are shown below.
+
+ <p align="center"><img src="./img/click_input_video.jpg" width = "600" height = "400" alt="click_input_video"/> <img src="./img/input_video.jpg" width = "300" height = "400" alt="input_video" /></p>
+
+## 3. Adjust the SegTracker-Args to suit your needs
+ - **aot_model**: used to select which version of DeAOT/AOT to use for tracking and propagation.
+ - **sam_gap**: used to control how often SAM is used to add newly appearing objects at specified frame intervals. Increase to decrease the frequency of discovering new targets, but significantly improve speed of inference.
+ - **points_per_side**: used to control the number of points per side used for generating masks by sampling a grid over the image. Increasing the size enhances the ability to detect small objects, but larger targets may be segmented into finer granularity.
+ - **max_obj_num**: used to limit the maximum number of objects that SegTracker can detect and track. A larger number of objects necessitates a greater utilization of memory, with approximately 16GB of memory capable of processing a maximum of 255 objects.
+
+## 4. Interactively modify single-object mask for first frame of video
+### 4.1 Interactively add single-object based on segment-everything(`Everything-Tab`)
+- `Segment everything for first frame`: By clicking the button, SegTracker will be initialized based on the `SegTracker-Args`, and `Segment-everything` will be performed on the first frame of the video. 
+- The example of the `segment-everything` approach are shown below.
+
+  <p align="center"><img src="./img/click_segment_everything.jpg" width = "300" height = "300" alt="click_segment_everything"/> <img src="./img/segment_everything_blackswan.jpg" width = "300" height = "300" alt="segment_everything_blackswan"/></p>
+
+- `Point Prompt`: After applying the Segment-everything function, you can click on the image to add objects that were ignored by segment-everything or assign a separate ID to an object by doing this. 
+- Two examples are provided below: one involves adding water which was previously ignored by the `segment-everything` approach, and the other involves assigning a separate ID to the face of a man.
+
+ <p align="center"><img src="./img/add_positive_base_on_everything.jpg" width = "300" height = "300" alt="add_positive_base_on_everything"/>
+ <img src="./img/add_positive_base_on_everything_cxk.jpg" width = "300" height = "300" alt="add_positive_base_on_everything_cxk"/></p>
+
+- `Note`: The current version only supports adding a mask of the single-object(The added objects are assigned the same ID) on top of the segment everything. We will update the operation of adding multi-objects-mask(The added objects are assigned different IDs) in the feature.
+
+### 4.2 Interactively add object by click(`Click-Tab`)
+- `Point Prompt`: you can select objects to track by clicking on the image with positive and negative points.
+- SegTracker will segment objects according to the specified prompt-points, as demonstrated in the example below.
+
+  <p align="center"><img src="./img/add_positive_points.jpg" width = "300" height = "300" alt="add_positive_points"> <img src="./img/add_positive_points_2.jpg" width = "300" height = "300" alt="add_positive_points_2"></p>
+
+### 4.3 Interactively add object by stroke(`Stroke-Tab`)
+- `Drawing board`: You can circle the object you want to track on it.
+    - `Undo`: To undo a stroke on the `Drawing board`, click the `Undo button` located in the upper right corner of the `Drawing board`.
+    - `Reset`: Click on the `Reset button` in the upper right corner of the `Drawing board` to reset the `Drawing board`.
+- `Segment`: SegTracker will receive the mask you draw and display the segmentation results.
+- Below is an example demonstrating how to circle and segment an object using strokes.
+ <p align="center"><img src="./img/click_segment.jpg" width = "300" height = "400" alt="Drawing_board"> <img src="./img/Drawing_board.jpg" width = "300" height = "400" alt="Drawing_board"></p>
+
+- `Note`: 
+    - The current version only supports adding a mask for a single-object(The added objects are assigned the same ID).
+    - We do not recommend adding a mask by clicking on `Segment result of first frame` under the `Stroke-Tab`, as this may result in bugs.
+    
+## 5. Segment and Track in Video
+- Once the object to be tracked in the video is identified, you can begin tracking by clicking on the `Start Tracking` button. 
+- The results are displayed on the `output video` and `predicted masks`.You can download them.
+
+ <p align="center"><img src="./img/start_tracking.jpg" width = "600" height = "550" alt="Drawing_board"></p>

tutorial/tutorial for WebUI-1.5-Version.md

@@ -0,0 +1,50 @@
+# Tutorial for WebUI 1.5 Version
+## We have added two new features
+- We have added text prompts to allow for interactive selection of objects that will be tracked in the video.
+- We can now interactively add multiple objects for tracking in the video.
+
+
+## Text-Prompts
+### 1. Clone Grounding-DINO to `./src`
+```
+pip install -e git+https://github.com/IDEA-Research/GroundingDINO.git@main#egg=GroundingDINO
+```
+
+### 2. Switch to Text-Tab by clicking `Text` Tab
+
+<p align="center">
+<img src="./img/switch2textT.jpg" height="400">
+</p>
+
+### 3. Upload video or use example dicectly
+
+### 4. Enter text to select the objects you are interested in
+- The `.` is used to split text, just like in the original Grounding-Dino setting.
+
+<p align="center">
+<img src="./img/enter_text.jpg" height="400", width="400">
+</p>
+
+### 5. Get mask of selected object by clicking `Detect` button
+- SAMTrack initialization may take some time.
+
+<p align="center">
+<img src="./img/detect_result.jpg" height="400", width="400">
+</p>
+
+### 6. Track in video
+
+## Multi-Objects select
+### 1. Once we interactively add an object mask, we can click the `Add new object button` to prepare to add a new object.
+
+<p align="center">
+<img src="./img/new_object.jpg" height="400", width="400">
+</p>
+
+### 2. Add a new object by clicking object
+
+<p align="center">
+<img src="./img/second_object.jpg" height="400", width="400">
+</p>
+
+### 3. You can add as many objects as you want by clicking `Add new object` button.