eos_token_id is a list but might have to be a token for MLC-LLM
Hey,
I'm using the quick start example with this checkpoint which throws an error
Traceback (most recent call last): File "D:\Programme\Anaconda\envs\FastSum\Lib\site-packages\mlc_llm\serve\engine_base.py", line 129, in _convert_model_info mlc_chat_config = MLCChatConfig.model_validate_json(file.read()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Programme\Anaconda\envs\FastSum\Lib\site-packages\pydantic\main.py", line 532, in model_validate_json return cls.__pydantic_validator__.validate_json(json_data, strict=strict, context=context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ pydantic_core._pydantic_core.ValidationError: 1 validation error for MLCChatConfig eos_token_id Input should be a valid integer [type=int_type, input_value=[32000, 32001, 32007], input_type=list] For further information visit https://errors.pydantic.dev/2.5/v/int_type
in the original checkpoint the eos_token_id is 32000.
Should the mlc-chat-config.json be changed here?
@wardenga Should be fixed via https://github.com/mlc-ai/mlc-llm/pull/2446, only one line in MLC-LLM is needed: https://github.com/mlc-ai/mlc-llm/pull/2446/files#diff-fbe04ea4e39c787f1532650de6675d9aebfd4d502789f2984ec1f2c59e444b7f
Try updating to the latest mlc_llm package, or pull the latest change if you are building mlc_llm from source