add log files

log_no_skip_frame/log_300/rtf.txt

@@ -0,0 +1,9 @@
+RTF: 0.0024
+total_duration: 32590.000 seconds
+(9.05 hours)
+processing time: 79.499 seconds (0.02 hours)
+latency_variance: 9.78
+latency_50_percentile: 366.42
+latency_90_percentile: 449.89
+latency_99_percentile: 711.00
+average_latency_ms: 380.25

log_no_skip_frame/log_300/stats_summary.txt

@@ -0,0 +1,57 @@
+model name is encoder 
+queue 12630.01 s, infer 5729.25 s, input 364.37 s, output 275.41 s 
+Batch_size 1 , 15182 times, infer 216682.19 ms, avg 14.27 ms, 14.27 ms input 697.06    ms, avg 0.05 ms, output 4188.86   ms, avg 0.28 ms 
+Batch_size 2 , 19450 times, infer 285723.26 ms, avg 14.69 ms, 7.35 ms input 12295.46  ms, avg 0.63 ms, output 6776.55   ms, avg 0.35 ms 
+Batch_size 3 , 15271 times, infer 224695.43 ms, avg 14.71 ms, 4.90 ms input 11230.25  ms, avg 0.74 ms, output 6587.21   ms, avg 0.43 ms 
+Batch_size 4 , 9393  times, infer 140112.24 ms, avg 14.92 ms, 3.73 ms input 7815.59   ms, avg 0.83 ms, output 4857.30   ms, avg 0.52 ms 
+Batch_size 5 , 6399  times, infer 97722.85  ms, avg 15.27 ms, 3.05 ms input 5982.20   ms, avg 0.93 ms, output 3904.97   ms, avg 0.61 ms 
+Batch_size 6 , 4697  times, infer 73084.56  ms, avg 15.56 ms, 2.59 ms input 4663.02   ms, avg 0.99 ms, output 3242.98   ms, avg 0.69 ms 
+Batch_size 7 , 3754  times, infer 58443.11  ms, avg 15.57 ms, 2.22 ms input 3908.37   ms, avg 1.04 ms, output 2898.95   ms, avg 0.77 ms 
+Batch_size 8 , 2970  times, infer 47877.06  ms, avg 16.12 ms, 2.02 ms input 3274.59   ms, avg 1.10 ms, output 2579.45   ms, avg 0.87 ms 
+Batch_size 9 , 2078  times, infer 33665.90  ms, avg 16.20 ms, 1.80 ms input 2437.06   ms, avg 1.17 ms, output 1902.57   ms, avg 0.92 ms 
+Batch_size 10, 1378  times, infer 23739.73  ms, avg 17.23 ms, 1.72 ms input 1744.01   ms, avg 1.27 ms, output 1364.55   ms, avg 0.99 ms 
+Batch_size 11, 894   times, infer 16301.42  ms, avg 18.23 ms, 1.66 ms input 1195.50   ms, avg 1.34 ms, output 953.64    ms, avg 1.07 ms 
+Batch_size 12, 467   times, infer 9696.54   ms, avg 20.76 ms, 1.73 ms input 631.21    ms, avg 1.35 ms, output 510.07    ms, avg 1.09 ms 
+Batch_size 13, 245   times, infer 6331.08   ms, avg 25.84 ms, 1.99 ms input 350.32    ms, avg 1.43 ms, output 291.56    ms, avg 1.19 ms 
+Batch_size 14, 140   times, infer 4720.95   ms, avg 33.72 ms, 2.41 ms input 207.27    ms, avg 1.48 ms, output 180.64    ms, avg 1.29 ms 
+Batch_size 15, 55    times, infer 3376.95   ms, avg 61.40 ms, 4.09 ms input 80.11     ms, avg 1.46 ms, output 75.71     ms, avg 1.38 ms 
+Batch_size 16, 3438  times, infer 59514.20  ms, avg 17.31 ms, 1.08 ms input 5870.21   ms, avg 1.71 ms, output 5233.21   ms, avg 1.52 ms 
+model name is feature_extractor 
+queue 2278.98 s, infer 2850.33 s, input 73.25 s, output 96.70 s 
+Batch_size 1 , 109754 times, infer 238234.16 ms, avg 2.17 ms, 2.17 ms input 7683.31   ms, avg 0.07 ms, output 18231.14  ms, avg 0.17 ms 
+Batch_size 2 , 29171 times, infer 105699.98 ms, avg 3.62 ms, 1.81 ms input 3449.10   ms, avg 0.12 ms, output 5779.74   ms, avg 0.20 ms 
+Batch_size 3 , 11758 times, infer 62936.10  ms, avg 5.35 ms, 1.78 ms input 1844.04   ms, avg 0.16 ms, output 2448.10   ms, avg 0.21 ms 
+Batch_size 4 , 6944  times, infer 49034.51  ms, avg 7.06 ms, 1.77 ms input 1372.14   ms, avg 0.20 ms, output 1570.81   ms, avg 0.23 ms 
+Batch_size 5 , 4717  times, infer 41723.72  ms, avg 8.85 ms, 1.77 ms input 1119.25   ms, avg 0.24 ms, output 1223.98   ms, avg 0.26 ms 
+Batch_size 6 , 3431  times, infer 36688.34  ms, avg 10.69 ms, 1.78 ms input 952.87    ms, avg 0.28 ms, output 998.36    ms, avg 0.29 ms 
+Batch_size 7 , 2506  times, infer 37309.09  ms, avg 14.89 ms, 2.13 ms input 798.34    ms, avg 0.32 ms, output 865.39    ms, avg 0.35 ms 
+Batch_size 8 , 1807  times, infer 30005.67  ms, avg 16.61 ms, 2.08 ms input 654.74    ms, avg 0.36 ms, output 689.00    ms, avg 0.38 ms 
+Batch_size 9 , 1331  times, infer 24642.31  ms, avg 18.51 ms, 2.06 ms input 545.80    ms, avg 0.41 ms, output 553.96    ms, avg 0.42 ms 
+Batch_size 10, 948   times, infer 18613.31  ms, avg 19.63 ms, 1.96 ms input 430.54    ms, avg 0.45 ms, output 457.61    ms, avg 0.48 ms 
+Batch_size 11, 662   times, infer 13697.76  ms, avg 20.69 ms, 1.88 ms input 328.40    ms, avg 0.50 ms, output 329.25    ms, avg 0.50 ms 
+Batch_size 12, 440   times, infer 9794.82   ms, avg 22.26 ms, 1.86 ms input 235.28    ms, avg 0.53 ms, output 239.25    ms, avg 0.54 ms 
+Batch_size 13, 299   times, infer 6992.62   ms, avg 23.39 ms, 1.80 ms input 174.26    ms, avg 0.58 ms, output 179.61    ms, avg 0.60 ms 
+Batch_size 14, 193   times, infer 4900.74   ms, avg 25.39 ms, 1.81 ms input 120.43    ms, avg 0.62 ms, output 120.33    ms, avg 0.62 ms 
+Batch_size 15, 129   times, infer 3453.76   ms, avg 26.77 ms, 1.78 ms input 92.38     ms, avg 0.72 ms, output 99.41     ms, avg 0.77 ms 
+Batch_size 16, 420   times, infer 12396.40  ms, avg 29.52 ms, 1.84 ms input 282.04    ms, avg 0.67 ms, output 502.42    ms, avg 1.20 ms 
+model name is scoring 
+queue 49012.70 s, infer 11895.52 s, input 196.23 s, output 343.24 s 
+Batch_size 1 , 5882  times, infer 132622.06 ms, avg 22.55 ms, 22.55 ms input 521.62    ms, avg 0.09 ms, output 1140.11   ms, avg 0.19 ms 
+Batch_size 2 , 6474  times, infer 153402.89 ms, avg 23.70 ms, 11.85 ms input 935.22    ms, avg 0.14 ms, output 1933.58   ms, avg 0.30 ms 
+Batch_size 3 , 5996  times, infer 147382.63 ms, avg 24.58 ms, 8.19 ms input 1184.46   ms, avg 0.20 ms, output 2224.86   ms, avg 0.37 ms 
+Batch_size 4 , 5033  times, infer 127482.63 ms, avg 25.33 ms, 6.33 ms input 1262.77   ms, avg 0.25 ms, output 2223.56   ms, avg 0.44 ms 
+Batch_size 5 , 3743  times, infer 98981.94  ms, avg 26.44 ms, 5.29 ms input 1125.63   ms, avg 0.30 ms, output 1899.00   ms, avg 0.51 ms 
+Batch_size 6 , 2566  times, infer 71483.73  ms, avg 27.86 ms, 4.64 ms input 904.85    ms, avg 0.35 ms, output 1488.02   ms, avg 0.58 ms 
+Batch_size 7 , 1717  times, infer 50576.88  ms, avg 29.46 ms, 4.21 ms input 685.89    ms, avg 0.40 ms, output 1109.79   ms, avg 0.65 ms 
+Batch_size 8 , 1253  times, infer 38934.19  ms, avg 31.07 ms, 3.88 ms input 577.96    ms, avg 0.46 ms, output 904.36    ms, avg 0.72 ms 
+Batch_size 9 , 971   times, infer 30925.81  ms, avg 31.85 ms, 3.54 ms input 466.38    ms, avg 0.48 ms, output 758.35    ms, avg 0.78 ms 
+Batch_size 10, 717   times, infer 23585.22  ms, avg 32.89 ms, 3.29 ms input 371.85    ms, avg 0.52 ms, output 612.48    ms, avg 0.85 ms 
+Batch_size 11, 607   times, infer 20501.16  ms, avg 33.77 ms, 3.07 ms input 329.09    ms, avg 0.54 ms, output 544.74    ms, avg 0.90 ms 
+Batch_size 12, 593   times, infer 20538.60  ms, avg 34.64 ms, 2.89 ms input 338.82    ms, avg 0.57 ms, output 568.57    ms, avg 0.96 ms 
+Batch_size 13, 546   times, infer 19221.10  ms, avg 35.20 ms, 2.71 ms input 339.16    ms, avg 0.62 ms, output 585.51    ms, avg 1.07 ms 
+Batch_size 14, 542   times, infer 19282.08  ms, avg 35.58 ms, 2.54 ms input 355.12    ms, avg 0.66 ms, output 619.56    ms, avg 1.14 ms 
+Batch_size 15, 448   times, infer 16227.70  ms, avg 36.22 ms, 2.41 ms input 315.75    ms, avg 0.70 ms, output 517.83    ms, avg 1.16 ms 
+Batch_size 16, 12023 times, infer 447830.68 ms, avg 37.25 ms, 2.33 ms input 8439.39   ms, avg 0.70 ms, output 14963.91  ms, avg 1.24 ms 
+model name is streaming_wenet 
+queue 0.47  s, infer 20475.10 s, input 633.84 s, output 715.35 s 
+Batch_size 1 , 356580 times, infer 20475095.92 ms, avg 57.42 ms, 57.42 ms input 633837.97 ms, avg 1.78 ms, output 715354.36 ms, avg 2.01 ms 

log_no_skip_frame/log_32/rtf.txt

@@ -0,0 +1,9 @@
+RTF: 0.0052
+total_duration: 32590.000 seconds
+(9.05 hours)
+processing time: 168.621 seconds (0.05 hours)
+latency_variance: 0.25
+latency_50_percentile: 90.22
+latency_90_percentile: 105.84
+latency_99_percentile: 130.93
+average_latency_ms: 88.93

log_no_skip_frame/log_32/stats_summary.txt

@@ -0,0 +1,57 @@
+model name is encoder 
+queue 10090.56 s, infer 3936.97 s, input 234.05 s, output 173.26 s 
+Batch_size 1 , 13884 times, infer 198859.59 ms, avg 14.32 ms, 14.32 ms input 631.94    ms, avg 0.05 ms, output 3761.11   ms, avg 0.27 ms 
+Batch_size 2 , 17435 times, infer 256114.99 ms, avg 14.69 ms, 7.34 ms input 11039.27  ms, avg 0.63 ms, output 5956.60   ms, avg 0.34 ms 
+Batch_size 3 , 12848 times, infer 189206.90 ms, avg 14.73 ms, 4.91 ms input 9435.32   ms, avg 0.73 ms, output 5409.65   ms, avg 0.42 ms 
+Batch_size 4 , 6995  times, infer 104918.56 ms, avg 15.00 ms, 3.75 ms input 5852.76   ms, avg 0.84 ms, output 3513.58   ms, avg 0.50 ms 
+Batch_size 5 , 3890  times, infer 60237.57  ms, avg 15.49 ms, 3.10 ms input 3678.37   ms, avg 0.95 ms, output 2345.19   ms, avg 0.60 ms 
+Batch_size 6 , 2486  times, infer 39847.61  ms, avg 16.03 ms, 2.67 ms input 2472.94   ms, avg 0.99 ms, output 1712.41   ms, avg 0.69 ms 
+Batch_size 7 , 1865  times, infer 30347.26  ms, avg 16.27 ms, 2.32 ms input 1977.82   ms, avg 1.06 ms, output 1446.21   ms, avg 0.78 ms 
+Batch_size 8 , 1473  times, infer 25185.92  ms, avg 17.10 ms, 2.14 ms input 1640.13   ms, avg 1.11 ms, output 1289.14   ms, avg 0.88 ms 
+Batch_size 9 , 1005  times, infer 17624.28  ms, avg 17.54 ms, 1.95 ms input 1166.20   ms, avg 1.16 ms, output 946.57    ms, avg 0.94 ms 
+Batch_size 10, 680   times, infer 13096.26  ms, avg 19.26 ms, 1.93 ms input 877.81    ms, avg 1.29 ms, output 682.44    ms, avg 1.00 ms 
+Batch_size 11, 454   times, infer 9642.28   ms, avg 21.24 ms, 1.93 ms input 618.42    ms, avg 1.36 ms, output 483.13    ms, avg 1.06 ms 
+Batch_size 12, 250   times, infer 6342.47   ms, avg 25.37 ms, 2.11 ms input 350.56    ms, avg 1.40 ms, output 276.32    ms, avg 1.11 ms 
+Batch_size 13, 131   times, infer 4517.73   ms, avg 34.49 ms, 2.65 ms input 196.26    ms, avg 1.50 ms, output 159.76    ms, avg 1.22 ms 
+Batch_size 14, 93    times, infer 3992.58   ms, avg 42.93 ms, 3.07 ms input 136.32    ms, avg 1.47 ms, output 120.78    ms, avg 1.30 ms 
+Batch_size 15, 39    times, infer 3103.50   ms, avg 79.58 ms, 5.31 ms input 49.93     ms, avg 1.28 ms, output 54.89     ms, avg 1.41 ms 
+Batch_size 16, 2238  times, infer 40715.87  ms, avg 18.19 ms, 1.14 ms input 3996.12   ms, avg 1.79 ms, output 3518.54   ms, avg 1.57 ms 
+model name is feature_extractor 
+queue 1491.26 s, infer 1733.09 s, input 45.75 s, output 61.76 s 
+Batch_size 1 , 83369 times, infer 185735.50 ms, avg 2.23 ms, 2.23 ms input 6060.20   ms, avg 0.07 ms, output 13794.40  ms, avg 0.17 ms 
+Batch_size 2 , 21840 times, infer 80621.38  ms, avg 3.69 ms, 1.85 ms input 2689.08   ms, avg 0.12 ms, output 4237.08   ms, avg 0.19 ms 
+Batch_size 3 , 7997  times, infer 43475.36  ms, avg 5.44 ms, 1.81 ms input 1310.00   ms, avg 0.16 ms, output 1697.36   ms, avg 0.21 ms 
+Batch_size 4 , 4281  times, infer 30366.01  ms, avg 7.09 ms, 1.77 ms input 875.93    ms, avg 0.20 ms, output 991.58    ms, avg 0.23 ms 
+Batch_size 5 , 2661  times, infer 23489.33  ms, avg 8.83 ms, 1.77 ms input 644.18    ms, avg 0.24 ms, output 706.58    ms, avg 0.27 ms 
+Batch_size 6 , 1874  times, infer 19877.21  ms, avg 10.61 ms, 1.77 ms input 528.99    ms, avg 0.28 ms, output 554.73    ms, avg 0.30 ms 
+Batch_size 7 , 1318  times, infer 19724.27  ms, avg 14.97 ms, 2.14 ms input 422.81    ms, avg 0.32 ms, output 455.29    ms, avg 0.35 ms 
+Batch_size 8 , 951   times, infer 15989.29  ms, avg 16.81 ms, 2.10 ms input 340.55    ms, avg 0.36 ms, output 363.74    ms, avg 0.38 ms 
+Batch_size 9 , 689   times, infer 12652.47  ms, avg 18.36 ms, 2.04 ms input 277.63    ms, avg 0.40 ms, output 280.57    ms, avg 0.41 ms 
+Batch_size 10, 518   times, infer 10246.33  ms, avg 19.78 ms, 1.98 ms input 235.39    ms, avg 0.45 ms, output 265.99    ms, avg 0.51 ms 
+Batch_size 11, 382   times, infer 8004.96   ms, avg 20.96 ms, 1.91 ms input 189.45    ms, avg 0.50 ms, output 193.29    ms, avg 0.51 ms 
+Batch_size 12, 230   times, infer 5127.53   ms, avg 22.29 ms, 1.86 ms input 121.73    ms, avg 0.53 ms, output 127.22    ms, avg 0.55 ms 
+Batch_size 13, 184   times, infer 4332.25   ms, avg 23.54 ms, 1.81 ms input 106.17    ms, avg 0.58 ms, output 105.74    ms, avg 0.57 ms 
+Batch_size 14, 125   times, infer 3201.34   ms, avg 25.61 ms, 1.83 ms input 77.90     ms, avg 0.62 ms, output 78.08     ms, avg 0.62 ms 
+Batch_size 15, 88    times, infer 2423.79   ms, avg 27.54 ms, 1.84 ms input 65.32     ms, avg 0.74 ms, output 72.55     ms, avg 0.82 ms 
+Batch_size 16, 273   times, infer 8007.30   ms, avg 29.33 ms, 1.83 ms input 184.24    ms, avg 0.67 ms, output 317.33    ms, avg 1.16 ms 
+model name is scoring 
+queue 31641.18 s, infer 7556.64 s, input 118.81 s, output 208.07 s 
+Batch_size 1 , 5759  times, infer 130372.20 ms, avg 22.64 ms, 22.64 ms input 510.55    ms, avg 0.09 ms, output 1115.80   ms, avg 0.19 ms 
+Batch_size 2 , 6405  times, infer 151889.06 ms, avg 23.71 ms, 11.86 ms input 925.51    ms, avg 0.14 ms, output 1913.37   ms, avg 0.30 ms 
+Batch_size 3 , 5937  times, infer 145781.79 ms, avg 24.55 ms, 8.18 ms input 1174.37   ms, avg 0.20 ms, output 2204.70   ms, avg 0.37 ms 
+Batch_size 4 , 4963  times, infer 125459.15 ms, avg 25.28 ms, 6.32 ms input 1247.64   ms, avg 0.25 ms, output 2192.43   ms, avg 0.44 ms 
+Batch_size 5 , 3663  times, infer 96516.73  ms, avg 26.35 ms, 5.27 ms input 1105.07   ms, avg 0.30 ms, output 1860.80   ms, avg 0.51 ms 
+Batch_size 6 , 2458  times, infer 68067.22  ms, avg 27.69 ms, 4.62 ms input 872.58    ms, avg 0.35 ms, output 1431.31   ms, avg 0.58 ms 
+Batch_size 7 , 1549  times, infer 45023.31  ms, avg 29.07 ms, 4.15 ms input 628.24    ms, avg 0.41 ms, output 1004.48   ms, avg 0.65 ms 
+Batch_size 8 , 1033  times, infer 31275.29  ms, avg 30.28 ms, 3.78 ms input 479.93    ms, avg 0.46 ms, output 748.76    ms, avg 0.72 ms 
+Batch_size 9 , 643   times, infer 19479.99  ms, avg 30.30 ms, 3.37 ms input 329.00    ms, avg 0.51 ms, output 520.18    ms, avg 0.81 ms 
+Batch_size 10, 341   times, infer 10508.57  ms, avg 30.82 ms, 3.08 ms input 193.50    ms, avg 0.57 ms, output 301.16    ms, avg 0.88 ms 
+Batch_size 11, 187   times, infer 5730.19   ms, avg 30.64 ms, 2.79 ms input 115.53    ms, avg 0.62 ms, output 177.17    ms, avg 0.95 ms 
+Batch_size 12, 113   times, infer 3604.30   ms, avg 31.90 ms, 2.66 ms input 77.39     ms, avg 0.68 ms, output 117.22    ms, avg 1.04 ms 
+Batch_size 13, 60    times, infer 1991.43   ms, avg 33.19 ms, 2.55 ms input 43.94     ms, avg 0.73 ms, output 69.94     ms, avg 1.17 ms 
+Batch_size 14, 36    times, infer 1203.67   ms, avg 33.44 ms, 2.39 ms input 31.36     ms, avg 0.87 ms, output 43.88     ms, avg 1.22 ms 
+Batch_size 15, 16    times, infer 581.49    ms, avg 36.34 ms, 2.42 ms input 17.38     ms, avg 1.09 ms, output 18.10     ms, avg 1.13 ms 
+Batch_size 16, 7199  times, infer 268050.27 ms, avg 37.23 ms, 2.33 ms input 5035.62   ms, avg 0.70 ms, output 8998.92   ms, avg 1.25 ms 
+model name is streaming_wenet 
+queue 0.33  s, infer 13226.70 s, input 398.61 s, output 443.08 s 
+Batch_size 1 , 237720 times, infer 13226700.24 ms, avg 55.64 ms, 55.64 ms input 398608.57 ms, avg 1.68 ms, output 443078.94 ms, avg 1.86 ms 

log_no_skip_frame/log_80/rtf.txt

@@ -0,0 +1,9 @@
+RTF: 0.0029
+total_duration: 32590.000 seconds
+(9.05 hours)
+processing time: 92.904 seconds (0.03 hours)
+latency_variance: 0.49
+latency_50_percentile: 116.35
+latency_90_percentile: 145.93
+latency_99_percentile: 199.48
+average_latency_ms: 119.78

log_no_skip_frame/log_80/stats_summary.txt

@@ -0,0 +1,57 @@
+model name is encoder 
+queue 11348.11 s, infer 4823.20 s, input 297.35 s, output 219.62 s 
+Batch_size 1 , 14707 times, infer 210197.44 ms, avg 14.29 ms, 14.29 ms input 672.73    ms, avg 0.05 ms, output 4015.62   ms, avg 0.27 ms 
+Batch_size 2 , 18678 times, infer 274333.38 ms, avg 14.69 ms, 7.34 ms input 11806.15  ms, avg 0.63 ms, output 6444.64   ms, avg 0.35 ms 
+Batch_size 3 , 14289 times, infer 210162.26 ms, avg 14.71 ms, 4.90 ms input 10521.22  ms, avg 0.74 ms, output 6092.55   ms, avg 0.43 ms 
+Batch_size 4 , 8421  times, infer 125706.93 ms, avg 14.93 ms, 3.73 ms input 7056.31   ms, avg 0.84 ms, output 4290.25   ms, avg 0.51 ms 
+Batch_size 5 , 5370  times, infer 82200.14  ms, avg 15.31 ms, 3.06 ms input 5052.52   ms, avg 0.94 ms, output 3254.11   ms, avg 0.61 ms 
+Batch_size 6 , 3717  times, infer 58228.24  ms, avg 15.67 ms, 2.61 ms input 3728.77   ms, avg 1.00 ms, output 2548.46   ms, avg 0.69 ms 
+Batch_size 7 , 2878  times, infer 45233.41  ms, avg 15.72 ms, 2.25 ms input 3034.04   ms, avg 1.05 ms, output 2212.70   ms, avg 0.77 ms 
+Batch_size 8 , 2216  times, infer 36387.36  ms, avg 16.42 ms, 2.05 ms input 2490.26   ms, avg 1.12 ms, output 1913.53   ms, avg 0.86 ms 
+Batch_size 9 , 1502  times, infer 24966.39  ms, avg 16.62 ms, 1.85 ms input 1762.75   ms, avg 1.17 ms, output 1381.30   ms, avg 0.92 ms 
+Batch_size 10, 1015  times, infer 18214.54  ms, avg 17.95 ms, 1.79 ms input 1305.94   ms, avg 1.29 ms, output 1003.44   ms, avg 0.99 ms 
+Batch_size 11, 659   times, infer 12731.38  ms, avg 19.32 ms, 1.76 ms input 903.35    ms, avg 1.37 ms, output 697.71    ms, avg 1.06 ms 
+Batch_size 12, 336   times, infer 7645.63   ms, avg 22.75 ms, 1.90 ms input 467.77    ms, avg 1.39 ms, output 366.52    ms, avg 1.09 ms 
+Batch_size 13, 178   times, infer 5246.76   ms, avg 29.48 ms, 2.27 ms input 258.88    ms, avg 1.45 ms, output 214.75    ms, avg 1.21 ms 
+Batch_size 14, 111   times, infer 4269.51   ms, avg 38.46 ms, 2.75 ms input 161.71    ms, avg 1.46 ms, output 143.87    ms, avg 1.30 ms 
+Batch_size 15, 43    times, infer 3170.13   ms, avg 73.72 ms, 4.91 ms input 57.40     ms, avg 1.33 ms, output 60.12     ms, avg 1.40 ms 
+Batch_size 16, 2628  times, infer 46796.59  ms, avg 17.81 ms, 1.11 ms input 4594.61   ms, avg 1.75 ms, output 4042.05   ms, avg 1.54 ms 
+model name is feature_extractor 
+queue 1749.18 s, infer 2220.33 s, input 58.18 s, output 76.26 s 
+Batch_size 1 , 98370 times, infer 215631.66 ms, avg 2.19 ms, 2.19 ms input 7003.57   ms, avg 0.07 ms, output 15871.64  ms, avg 0.16 ms 
+Batch_size 2 , 25956 times, infer 94970.46  ms, avg 3.66 ms, 1.83 ms input 3128.18   ms, avg 0.12 ms, output 4952.20   ms, avg 0.19 ms 
+Batch_size 3 , 10049 times, infer 54377.18  ms, avg 5.41 ms, 1.80 ms input 1609.94   ms, avg 0.16 ms, output 2067.71   ms, avg 0.21 ms 
+Batch_size 4 , 5739  times, infer 40802.86  ms, avg 7.11 ms, 1.78 ms input 1153.57   ms, avg 0.20 ms, output 1294.77   ms, avg 0.23 ms 
+Batch_size 5 , 3717  times, infer 33081.27  ms, avg 8.90 ms, 1.78 ms input 895.17    ms, avg 0.24 ms, output 963.63    ms, avg 0.26 ms 
+Batch_size 6 , 2706  times, infer 29066.33  ms, avg 10.74 ms, 1.79 ms input 760.43    ms, avg 0.28 ms, output 788.62    ms, avg 0.29 ms 
+Batch_size 7 , 1883  times, infer 27953.77  ms, avg 14.85 ms, 2.12 ms input 606.89    ms, avg 0.32 ms, output 650.53    ms, avg 0.35 ms 
+Batch_size 8 , 1350  times, infer 22532.33  ms, avg 16.69 ms, 2.09 ms input 490.93    ms, avg 0.36 ms, output 521.74    ms, avg 0.39 ms 
+Batch_size 9 , 952   times, infer 17572.62  ms, avg 18.46 ms, 2.05 ms input 392.27    ms, avg 0.41 ms, output 388.63    ms, avg 0.41 ms 
+Batch_size 10, 689   times, infer 13670.00  ms, avg 19.84 ms, 1.98 ms input 316.59    ms, avg 0.46 ms, output 342.47    ms, avg 0.50 ms 
+Batch_size 11, 471   times, infer 9909.10   ms, avg 21.04 ms, 1.91 ms input 234.99    ms, avg 0.50 ms, output 234.81    ms, avg 0.50 ms 
+Batch_size 12, 303   times, infer 6805.46   ms, avg 22.46 ms, 1.87 ms input 163.83    ms, avg 0.54 ms, output 165.84    ms, avg 0.55 ms 
+Batch_size 13, 210   times, infer 4941.74   ms, avg 23.53 ms, 1.81 ms input 122.98    ms, avg 0.59 ms, output 121.58    ms, avg 0.58 ms 
+Batch_size 14, 141   times, infer 3614.32   ms, avg 25.63 ms, 1.83 ms input 88.59     ms, avg 0.63 ms, output 87.33     ms, avg 0.62 ms 
+Batch_size 15, 96    times, infer 2627.21   ms, avg 27.37 ms, 1.82 ms input 70.87     ms, avg 0.74 ms, output 77.21     ms, avg 0.80 ms 
+Batch_size 16, 284   times, infer 8306.64   ms, avg 29.25 ms, 1.83 ms input 194.52    ms, avg 0.68 ms, output 335.68    ms, avg 1.18 ms 
+model name is scoring 
+queue 32965.40 s, infer 9660.66 s, input 155.57 s, output 271.33 s 
+Batch_size 1 , 5838  times, infer 131866.60 ms, avg 22.59 ms, 22.59 ms input 517.68    ms, avg 0.09 ms, output 1131.43   ms, avg 0.19 ms 
+Batch_size 2 , 6460  times, infer 153053.49 ms, avg 23.69 ms, 11.85 ms input 933.07    ms, avg 0.14 ms, output 1929.43   ms, avg 0.30 ms 
+Batch_size 3 , 5983  times, infer 147077.29 ms, avg 24.58 ms, 8.19 ms input 1181.85   ms, avg 0.20 ms, output 2220.04   ms, avg 0.37 ms 
+Batch_size 4 , 5022  times, infer 127213.19 ms, avg 25.33 ms, 6.33 ms input 1259.99   ms, avg 0.25 ms, output 2217.91   ms, avg 0.44 ms 
+Batch_size 5 , 3736  times, infer 98780.43  ms, avg 26.44 ms, 5.29 ms input 1123.45   ms, avg 0.30 ms, output 1895.34   ms, avg 0.51 ms 
+Batch_size 6 , 2558  times, infer 71226.98  ms, avg 27.84 ms, 4.64 ms input 901.98    ms, avg 0.35 ms, output 1482.98   ms, avg 0.58 ms 
+Batch_size 7 , 1708  times, infer 50277.21  ms, avg 29.44 ms, 4.21 ms input 682.19    ms, avg 0.40 ms, output 1103.47   ms, avg 0.65 ms 
+Batch_size 8 , 1239  times, infer 38463.39  ms, avg 31.04 ms, 3.88 ms input 571.76    ms, avg 0.46 ms, output 894.18    ms, avg 0.72 ms 
+Batch_size 9 , 965   times, infer 30731.71  ms, avg 31.85 ms, 3.54 ms input 463.38    ms, avg 0.48 ms, output 753.85    ms, avg 0.78 ms 
+Batch_size 10, 710   times, infer 23349.33  ms, avg 32.89 ms, 3.29 ms input 368.42    ms, avg 0.52 ms, output 606.45    ms, avg 0.85 ms 
+Batch_size 11, 605   times, infer 20433.47  ms, avg 33.77 ms, 3.07 ms input 327.87    ms, avg 0.54 ms, output 542.67    ms, avg 0.90 ms 
+Batch_size 12, 591   times, infer 20459.39  ms, avg 34.62 ms, 2.88 ms input 337.39    ms, avg 0.57 ms, output 566.74    ms, avg 0.96 ms 
+Batch_size 13, 540   times, infer 19014.53  ms, avg 35.21 ms, 2.71 ms input 335.72    ms, avg 0.62 ms, output 578.45    ms, avg 1.07 ms 
+Batch_size 14, 537   times, infer 19111.39  ms, avg 35.59 ms, 2.54 ms input 352.17    ms, avg 0.66 ms, output 613.66    ms, avg 1.14 ms 
+Batch_size 15, 443   times, infer 16060.98  ms, avg 36.26 ms, 2.42 ms input 312.60    ms, avg 0.71 ms, output 511.68    ms, avg 1.16 ms 
+Batch_size 16, 8359  times, infer 309729.11 ms, avg 37.05 ms, 2.32 ms input 5920.44   ms, avg 0.71 ms, output 10509.21  ms, avg 1.26 ms 
+model name is streaming_wenet 
+queue 0.40  s, infer 16704.19 s, input 511.10 s, output 567.21 s 
+Batch_size 1 , 297150 times, infer 16704185.55 ms, avg 56.21 ms, 56.21 ms input 511096.19 ms, avg 1.72 ms, output 567212.27 ms, avg 1.91 ms