No genetrate tokens in oobabooga
17:02:14-578319 INFO The AutoGPTQ params are: {'model_basename': 'model',
'device': 'cuda:0', 'use_triton': False,
'inject_fused_attention': True, 'inject_fused_mlp':
True, 'use_safetensors': True, 'trust_remote_code':
False, 'max_memory': {0: '14900MiB', 1: '12000MiB',
'cpu': '99GiB'}, 'quantize_config': None,
'use_cuda_fp16': True, 'disable_exllama': False,
'disable_exllamav2': False}
17:03:27-901969 INFO LOADER: AutoGPTQ
17:03:27-903669 INFO TRUNCATION LENGTH: 4096
17:03:27-905051 INFO INSTRUCTION TEMPLATE: Custom (obtained from model
metadata)
17:03:27-906589 INFO Loaded the model in 73.33 seconds.
17:05:38-516164 INFO Deleted logs/instruct/20240117-17-00-17.json.
Output generated in 0.28 seconds (0.00 tokens/s, 0 tokens, context 538, seed 679366167)