Update README.md

README.md

@@ -2,14 +2,17 @@
 library_name: peft
 tags:
 - generated_from_trainer
+- llama
+- llama 2
 model-index:
 - name: volume/limarp-70b-qlora
   results: []
+datasets:
+- lemonilia/LimaRP
+language:
+- en
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
 [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
 <details><summary>See axolotl config</summary>
 
@@ -92,21 +95,73 @@ special_tokens:
 
 </details><br>
 
-# volume/limarp-70b-qlora
+# limarp-miqu-1-70b-qlora
+
+Experimental limarp qlora trained at 16384 ctx length (greater than size of the longest limarp sample when tokenized via mistral's tokenizer) on the fixed dequantized miqu-1-70b model by 152334H.
+
+I wasn't particularly happy with the results I got when I tried applying the lora at varying weights to the miqu-1-70b model. It's possible that this is related to the fact that the model was dequantized from Q5_K_M GGUF, or perhaps due to it already being an instruct-tuned model.
 
-This model was trained from scratch on the None dataset.
+However, I decided to go ahead and release this in case someone else finds a use for it.
 
 ## Model description
 
-More information needed
+The intended prompt format is the Alpaca instruction format of LimaRP v3:
+```
+### Instruction:
+Character's Persona: {bot character description}
+
+User's Persona: {user character description}
+
+Scenario: {what happens in the story}
+
+Play the role of Character. Taking the above information into consideration, you must engage in a roleplaying chat with User below this line. Do not write dialogues and narration for User.
+
+### Input:
+User: {utterance}
+
+### Response:
+Character: {utterance}
+
+### Input:
+User: {utterance}
+
+### Response:
+Character: {utterance}
+
+(etc.)
+```
+Inspired by the previously named "Roleplay" preset in SillyTavern, with this version of LimaRP it is possible to append a length modifier to the response instruction sequence, like this:
+
+```
+### Input
+User: {utterance}
+
+### Response: (length = medium)
+Character: {utterance}
+```
+
+This has an immediately noticeable effect on bot responses. The lengths using during training are:
+`micro`, `tiny`, `short`, `medium`, `long`, `massive`, `huge`, `enormous`, `humongous`, `unlimited`.
+**The recommended starting length is medium**. Keep in mind that the AI can ramble or impersonate
+the user with very long messages.
+
+The length control effect is reproducible, but the messages will not necessarily follow
+lengths very precisely, rather follow certain ranges on average, as seen in this table
+with data from tests made with one reply at the beginning of the conversation:
+
+![lengths](https://i.imgur.com/2WXGgaV.png)
+
+Response length control appears to work well also deep into the conversation. **By omitting
+the modifier, the model will choose the most appropriate response length** (although it might
+not necessarily be what the user desires).
 
 ## Intended uses & limitations
 
-More information needed
+The model will show biases similar to those observed in niche roleplaying forums on the Internet, besides those exhibited by the base model.
 
 ## Training and evaluation data
 
-More information needed
+For more details about LimaRP, see the dataset page.
 
 ## Training procedure
 
@@ -124,10 +179,6 @@ The following hyperparameters were used during training:
 - lr_scheduler_warmup_steps: 10
 - num_epochs: 2
 
-### Training results
-
-
-
 ### Framework versions
 
 - PEFT 0.7.2.dev0