monai
medical
katielink commited on
Commit
50252f2
·
1 Parent(s): 715e62e

enhance data preprocess script and readme file

Browse files
README.md CHANGED
@@ -18,13 +18,13 @@ The datasets used in this work were provided by [Activ Surgical](https://www.act
18
  We've provided a [link](https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/inbody_outbody_samples.zip) of 20 samples (10 in-body and 10 out-body) to show what this dataset looks like.
19
 
20
  ### Preprocessing
21
- After downloading this dataset, python script in `scripts` folder naming `data_process` can be used to get label json files by running the command below and replacing datapath and outpath parameters.
22
 
23
  ```
24
- python scripts/data_process.py --datapath /path/to/data/root --outpath /path/to/label/folder
25
  ```
26
 
27
- After generating label files, please modify the `dataset_dir` parameter specified in `configs/train.json` and `configs/inference.json` to reflect where label files are.
28
 
29
  The input label json should be a list made up by dicts which includes `image` and `label` keys. An example format is shown below.
30
 
@@ -116,6 +116,7 @@ python -m monai.bundle run evaluating \
116
  --config_file configs/inference.json \
117
  --logging_file configs/logging.conf
118
  ```
 
119
 
120
  #### Export checkpoint to TorchScript file:
121
 
 
18
  We've provided a [link](https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/inbody_outbody_samples.zip) of 20 samples (10 in-body and 10 out-body) to show what this dataset looks like.
19
 
20
  ### Preprocessing
21
+ After downloading this dataset, python script in `scripts` folder named `data_process` can be used to generate label json files by running the command below and modifying `datapath` to path of unziped downloaded data. Generated label json files will be stored in `label` folder under the bundle path.
22
 
23
  ```
24
+ python scripts/data_process.py --datapath /path/to/data/root
25
  ```
26
 
27
+ By default, label path parameter in `train.json` and `inference.json` of this bundle is point to the generated `label` folder under bundle path. If you move these generated label files to another place, please modify the `train_json`, `val_json` and `test_json` parameters specified in `configs/train.json` and `configs/inference.json` to where these label files are.
28
 
29
  The input label json should be a list made up by dicts which includes `image` and `label` keys. An example format is shown below.
30
 
 
116
  --config_file configs/inference.json \
117
  --logging_file configs/logging.conf
118
  ```
119
+ The classification result of every images in `test.json` will be printed to the screen.
120
 
121
  #### Export checkpoint to TorchScript file:
122
 
configs/inference.json CHANGED
@@ -5,9 +5,8 @@
5
  "$import torch"
6
  ],
7
  "bundle_root": "/workspace/bundle/endoscopic_inbody_classification",
8
- "output_dir": "$@bundle_root + '/eval'",
9
  "dataset_dir": "/workspace/data/endoscopic_inbody_classification",
10
- "test_json": "$@dataset_dir+'/test.json'",
11
  "test_fp": "$open(@test_json,'r', encoding='utf8')",
12
  "test_dict": "$json.load(@test_fp)",
13
  "test_close": "$@test_fp.close()",
 
5
  "$import torch"
6
  ],
7
  "bundle_root": "/workspace/bundle/endoscopic_inbody_classification",
 
8
  "dataset_dir": "/workspace/data/endoscopic_inbody_classification",
9
+ "test_json": "$@bundle_root+'/label/test_samples.json'",
10
  "test_fp": "$open(@test_json,'r', encoding='utf8')",
11
  "test_dict": "$json.load(@test_fp)",
12
  "test_close": "$@test_fp.close()",
configs/metadata.json CHANGED
@@ -1,7 +1,8 @@
1
  {
2
  "schema": "https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/meta_schema_20220324.json",
3
- "version": "0.3.2",
4
  "changelog": {
 
5
  "0.3.2": "restructure readme to match updated template",
6
  "0.3.1": "add workflow, train loss and validation accuracy figures",
7
  "0.3.0": "update dataset processing",
 
1
  {
2
  "schema": "https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/meta_schema_20220324.json",
3
+ "version": "0.3.3",
4
  "changelog": {
5
+ "0.3.3": "enhance data preprocess script and readme file",
6
  "0.3.2": "restructure readme to match updated template",
7
  "0.3.1": "add workflow, train loss and validation accuracy figures",
8
  "0.3.0": "update dataset processing",
configs/train.json CHANGED
@@ -8,8 +8,8 @@
8
  "ckpt_dir": "$@bundle_root + '/models'",
9
  "output_dir": "$@bundle_root + '/eval'",
10
  "dataset_dir": "/workspace/data/endoscopic_inbody_classification",
11
- "train_json": "$@dataset_dir+'/train.json'",
12
- "val_json": "$@dataset_dir+'/val.json'",
13
  "train_fp": "$open(@train_json,'r', encoding='utf8')",
14
  "train_dict": "$json.load(@train_fp)",
15
  "train_close": "$@train_fp.close()",
 
8
  "ckpt_dir": "$@bundle_root + '/models'",
9
  "output_dir": "$@bundle_root + '/eval'",
10
  "dataset_dir": "/workspace/data/endoscopic_inbody_classification",
11
+ "train_json": "$@bundle_root+'/label/train_samples.json'",
12
+ "val_json": "$@bundle_root+'/label/val_samples.json'",
13
  "train_fp": "$open(@train_json,'r', encoding='utf8')",
14
  "train_dict": "$json.load(@train_fp)",
15
  "train_close": "$@train_fp.close()",
docs/README.md CHANGED
@@ -11,13 +11,13 @@ The datasets used in this work were provided by [Activ Surgical](https://www.act
11
  We've provided a [link](https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/inbody_outbody_samples.zip) of 20 samples (10 in-body and 10 out-body) to show what this dataset looks like.
12
 
13
  ### Preprocessing
14
- After downloading this dataset, python script in `scripts` folder naming `data_process` can be used to get label json files by running the command below and replacing datapath and outpath parameters.
15
 
16
  ```
17
- python scripts/data_process.py --datapath /path/to/data/root --outpath /path/to/label/folder
18
  ```
19
 
20
- After generating label files, please modify the `dataset_dir` parameter specified in `configs/train.json` and `configs/inference.json` to reflect where label files are.
21
 
22
  The input label json should be a list made up by dicts which includes `image` and `label` keys. An example format is shown below.
23
 
@@ -109,6 +109,7 @@ python -m monai.bundle run evaluating \
109
  --config_file configs/inference.json \
110
  --logging_file configs/logging.conf
111
  ```
 
112
 
113
  #### Export checkpoint to TorchScript file:
114
 
 
11
  We've provided a [link](https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/inbody_outbody_samples.zip) of 20 samples (10 in-body and 10 out-body) to show what this dataset looks like.
12
 
13
  ### Preprocessing
14
+ After downloading this dataset, python script in `scripts` folder named `data_process` can be used to generate label json files by running the command below and modifying `datapath` to path of unziped downloaded data. Generated label json files will be stored in `label` folder under the bundle path.
15
 
16
  ```
17
+ python scripts/data_process.py --datapath /path/to/data/root
18
  ```
19
 
20
+ By default, label path parameter in `train.json` and `inference.json` of this bundle is point to the generated `label` folder under bundle path. If you move these generated label files to another place, please modify the `train_json`, `val_json` and `test_json` parameters specified in `configs/train.json` and `configs/inference.json` to where these label files are.
21
 
22
  The input label json should be a list made up by dicts which includes `image` and `label` keys. An example format is shown below.
23
 
 
109
  --config_file configs/inference.json \
110
  --logging_file configs/logging.conf
111
  ```
112
+ The classification result of every images in `test.json` will be printed to the screen.
113
 
114
  #### Export checkpoint to TorchScript file:
115
 
scripts/data_process.py CHANGED
@@ -42,9 +42,9 @@ def generate_labels(data_path, output_path):
42
  train_list = inbody_train_list + outbody_train_list
43
  val_list = inbody_val_list + outbody_val_list
44
  test_list = inbody_test_list + outbody_test_list
45
- save_json(train_list, out_path, "train.json")
46
- save_json(val_list, out_path, "val.json")
47
- save_json(test_list, out_path, "test.json")
48
 
49
 
50
  if __name__ == "__main__":
@@ -58,12 +58,7 @@ if __name__ == "__main__":
58
  )
59
 
60
  # path to save label json.
61
- parser.add_argument(
62
- "--outpath",
63
- type=str,
64
- default=r"/workspace/data/endoscopic_inbody_classification",
65
- help="A path to save the onnx model.",
66
- )
67
 
68
  args = parser.parse_args()
69
  data_path = args.datapath
 
42
  train_list = inbody_train_list + outbody_train_list
43
  val_list = inbody_val_list + outbody_val_list
44
  test_list = inbody_test_list + outbody_test_list
45
+ save_json(train_list, out_path, "train_samples.json")
46
+ save_json(val_list, out_path, "val_samples.json")
47
+ save_json(test_list, out_path, "test_samples.json")
48
 
49
 
50
  if __name__ == "__main__":
 
58
  )
59
 
60
  # path to save label json.
61
+ parser.add_argument("--outpath", type=str, default=r"./label", help="A path to save the onnx model.")
 
 
 
 
 
62
 
63
  args = parser.parse_args()
64
  data_path = args.datapath