No memory within model?
Hi Tom,
First off, thanks for all the great work you do. Many of us appreciate it. I only have an RTX 2070 to work with so I appreciate the quantized models as many others I'm sure.
Secondly, I was wondering if there was a way you could fine tune the SOLAR-10.7B-Instruct model so that it has a memory of previous inputs?
For example, if I prompt: what is the fastest car in the world. It gives an answer about the fastest car. However if I then ask something like: 'Tell me more about this car' It's completely oblivious to what it already told me. It has no memory at all which quite frankly makes the model pretty useless.
Thanks again.
@jdc4429 you have to understand llms are stateless so they don’t have stored memory.
Rather you have to input the context like this.
Let’s say the first question is
User:
what is the fastest car
Assistant:
Then you would get the output of solar model
So
User:
What is the fastest car
Assistant:
The fastest car is thrust ssc
And then you would do
User:
What is the fastest car
Assistant:
The fastest car is thrust ssc
User:
Tell me more
Assistant:
And you would go on and on like that until it reaches the maximum context
In other models the context from previous questions stays in memory until you reset the chat.. this one does not. Is it possible to change this?
@jdc4429 hmm no, thats usually just a feature like in text generation web ui. 99% of the models including big llm that uses llama, mixtral, mistral, falcon, mpt have any memory storage thing, you can only pass previous context like in solar 10b. Even chatgpt or gpt4 doesnt have that
@jlzhou there is blenderbot model but its a very early model from meta and its horrible compared to even tinyllama 1.1b . Blenderbot only has a very weak chatting ability and basically 0 knowledge abt anything else. It does have the ability to store memory without passing previous context tho.
