what is the context limited of https://rwkv-next-web.ai-creator.net/#/chat?
#1
by
ChiTester11
- opened
Thanks for setting up https://rwkv-next-web.ai-creator.net/#/chat for testing. I tried to enter a long document, about 25K words, the website was not able to handle it, i.e. no output. When I reduce the text to about 2K, it worked. What is the current context limit?
@ChiTester11 about 65k, you can test it with rwkv runner,use this one state model
be aware ,not trained enough with long text datasets instruction, just textbook reference in datasets is long enough.
BTW, plan to train a pdf reader model for very long context input, haven't found enough datasets yet.