Xenova

/

t5-small

like 3

Hey,

I am trying to run a quantized version of this model in Javascript, on the browser, using @xenova /transformers.

Here is my folder structure:

onnx_model_quantized_final:
onnx:
- decoder_model_merged_quantized.onnx
- decoder_model_quantized.onnx
- decoder_with_past_model_quantized.onnx
- encoder_model_quantized.onnx
config.json
generation_config.json
ort_config.json
special_tokens_map.json
spiece.model
tokenizer.json
tokenizer_config.json

import { pipeline, env } from '@xenova/transformers';
....
env.allowLocalModels = true;
env.allowRemoteModels = false;
....
const generator = await pipeline('translation', 'onnx_model_quantized_final', { quantized: true });

The code above successfully downloads all the necessary files from the locally stored model

However, WebAssembly fails to load the model and gives a generic error **Error**: Can't create a session:

wasm-core-impl.ts:55 Uncaught (in promise) Error: Can't create a session
at e.createSessionFinalize (wasm-core-impl.ts:55:17)
at e.createSession (wasm-core-impl.ts:99:14)
at e.createSession (proxy-wrapper.ts:187:17)
at e.OnnxruntimeWebAssemblySessionHandler.loadModel (session-handler.ts:65:67)
at async Object.createSessionHandler (backend-wasm.ts:49:5)
at async _InferenceSession.create (inference-session-impl.ts:189:21)
at async constructSession (models.js:126:16)
at async Promise.all (:5173/index 2)
at async T5ForConditionalGeneration.from_pretrained (models.js:768:20)
at async AutoModelForSeq2SeqLM.from_pretrained (models.js:5490:20)

Do you know how I could go about solving this? Can this model be run in a browser environment?

Thanks!