Anyone discovered "Mini" yet in prompting?
#83
by YoYo1234Qwerty - opened
It seems like it's still a mystery what dataset TII used, and how to prompt this model. I've noticed from experiments that it's eager to output a format like:
User How are you
Mini Fine thanks
For instance:
User Nice to meet you. How's the weather?
Mini The weather is actually terrible, with rain and thunderstorms predicted. Are you and your friends still planning on seeing a movie together tonight?
User Oh wow, we might have umbrellas handy then.... So have you seen any good movies recently?
Mini As an AI I'm not capable off `watching` movies per-said, but are some favorites include Titanic 1922, Interstellar, Avatar, and Avengers movies.
I've seen in the tokenizer things like >>QUESTION<< and >>ANSWER<<, but it's not clear if they just ripped off this tokenizer from somewhere, and never actually trained it like this.
Other users suggest this format, but I've never seen it verified:
User: blah
Assistant: blah
Anyone have any insights into the Instruct dataset, or how this was intended to be prompted?