happyme531

/

MiniCPM-V-2_6-rkllm

like 8

The code errors with "Segmentation fault".

OS: Joshua-Riek Ubuntu 24.04 or
OS: rock-5c_bookworm_kde_b1.output.img.x
OpenCV 4-11.
Numpy 2-2 (installed by opencv-python)
rknpu-toolkit2-lite v2.3.0

Output:

rock@5c:$ workon llm
(llm) rock@5c:$ cd MiniCPM-V-2_6-rkllm
(llm) rock@5c:~/MiniCPM-V-2_6-rkllm$ python multiprocess_inference.py
Start loading language model (size: 7810.02 MB)
I rkllm: rkllm-runtime version: 1.1.2, rknpu driver version: 0.9.7, platform: RK3588

Start loading vision encoder model (size: 942.29 MB)
Vision encoder loaded in 21.00 seconds
I RKNN: [17:37:04.301] RKNN Runtime Information, librknnrt version: 2.3.0 (c949ad889d@2024-11-07T11:35:33)
I RKNN: [17:37:04.301] RKNN Driver Information, version: 0.9.7
I RKNN: [17:37:04.302] RKNN Model Information, version: 6, toolkit version: 2.2.0(compiler version: 2.2.0 (c195366594@2024-09-14T12:24:14)), target: RKNPU v2, target platform: rk3588, framework name: ONNX, framework layout: NCHW, model inference type: dynamic_shape
Received ready signal: vision_ready
Language model loaded in 107.37 seconds
Received ready signal: llm_ready
All models loaded, starting interactive mode...

Enter your input (3 empty lines to start inference, Ctrl+C to exit, for example:
详细描述一下{{./test.jpg}}这张图片
What is the weather in {{./test.jpg}}?
How many people are in {{./test.jpg}}?
):

Describe the image: {{./test.jpg}} in every detail.

Start vision inference...
Vision encoder inference time: 3.60 seconds
Fatal Python error: Segmentation fault

Thread 0x0000ffff7526f180 (most recent call first):
File "/usr/lib/python3.12/threading.py", line 355 in wait
File "/usr/lib/python3.12/multiprocessing/queues.py", line 251 in _feed
File "/usr/lib/python3.12/threading.py", line 1010 in run
File "/usr/lib/python3.12/threading.py", line 1073 in _bootstrap_inner
File "/usr/lib/python3.12/threading.py", line 1030 in _bootstrap

Current thread 0x0000ffff8aaee720 (most recent call first):
File "/home/rock/MiniCPM-V-2_6-rkllm/rkllm_binding.py", line 184 in run
File "/home/rock/MiniCPM-V-2_6-rkllm/multiprocess_inference.py", line 135 in llm_process
File "/usr/lib/python3.12/multiprocessing/process.py", line 108 in run
File "/usr/lib/python3.12/multiprocessing/process.py", line 314 in _bootstrap
File "/usr/lib/python3.12/multiprocessing/popen_fork.py", line 71 in _launch
File "/usr/lib/python3.12/multiprocessing/popen_fork.py", line 19 in init
File "/usr/lib/python3.12/multiprocessing/context.py", line 282 in _Popen
File "/usr/lib/python3.12/multiprocessing/context.py", line 224 in _Popen
File "/usr/lib/python3.12/multiprocessing/process.py", line 121 in start
File "/home/rock/MiniCPM-V-2_6-rkllm/multiprocess_inference.py", line 154 in main
File "/home/rock/MiniCPM-V-2_6-rkllm/multiprocess_inference.py", line 221 in

Extension modules: numpy._core._multiarray_umath, numpy.linalg._umath_linalg, rknnlite.api.rknn_utils, rknnlite.api.rknn_log, rknnlite.api.rknn_model, rknnlite.utils.yaml_parser, rknnlite.api.npu_config.cpu_npu_mapper, rknnlite.api.rknn_platform_utils, psutil._psutil_linux, psutil._psutil_posix, rknnlite.api.rknn_runtime (total: 11)