--- license: apache-2.0 language: - en library_name: transformers pipeline_tag: text-generation pretty_name: Echo88 150M Instruct tags: - transformers - safetensors - llama - text-generation - causal-lm - instruct - chat - decoder-only - autoregressive - from-scratch - retro - 1980s - usenet - magazines - books - computer-history - english - small-language-model - tiny-llm base_model: - exnivo/Echo88-150M-Base datasets: - exnivo/Echo88-150M-Base - exnivo/Echo88-Instruct-173K ---

Echo88 150M Instruct — Retro instruction-tuned language model

# Echo88-150M Instruct **A 163M parameter retro instruction-tuned language model inspired by the late 1980s.** Echo88-150M Instruct is an experimental small instruction-tuned language model fine-tuned from [`exnivo/Echo88-150M-Base`](https://huggingface.co/exnivo/Echo88-150M-Base). Echo88 is designed to feel like a helpful retro computer assistant whose records go up to the end of **1988**. The model is focused on older books, magazines, Usenet-style discussion, early personal computing, 1980s culture, and historical computer terminology. This is the first public instruction-tuned version of Echo88. **Echo88-150M-Instruct v2 is planned.** ## Quick Start ```python import torch from transformers import AutoTokenizer, AutoModelForCausalLM model_id = "exnivo/Echo88-150M-Instruct" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto", ) SYSTEM_PROMPT = ( "You are Echo88, a helpful computer assistant whose records go up to the end of 1988. " "Answer clearly. Do not pretend to know events, products, or culture after 1988." ) def ask(question, max_new_tokens=120): prompt = ( "<|system|>\n" + SYSTEM_PROMPT + "\n<|end|>\n" + "<|user|>\n" + question + "\n<|assistant|>\n" ) inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad(): output = model.generate( **inputs, max_new_tokens=max_new_tokens, do_sample=True, temperature=0.55, top_p=0.85, repetition_penalty=1.18, no_repeat_ngram_size=4, pad_token_id=tokenizer.pad_token_id, eos_token_id=tokenizer.eos_token_id, ) text = tokenizer.decode(output[0], skip_special_tokens=False) answer = text.split("<|assistant|>")[-1].split("<|end|>")[0].strip() return answer print(ask("What is a modem?")) ``` ## At a Glance | Item | Details | |---|---| | Model type | Instruction-tuned causal language model | | Parameters | 163,606,272 | | Approx. size | 163.6M | | Architecture | LLaMA-style decoder-only transformer | | Language | English | | Context length | 2048 tokens | | Vocabulary size | 32,768 | | Layers | 18 | | Hidden size | 768 | | Intermediate size | 2048 | | Attention heads | 12 | | Key/value heads | 4 | | Tokenizer | Custom Echo88 byte-level BPE tokenizer | | Base model | [`exnivo/Echo88-150M-Base`](https://huggingface.co/exnivo/Echo88-150M-Base) | | SFT dataset | [`exnivo/Echo88-Instruct-173K`](https://huggingface.co/datasets/exnivo/Echo88-Instruct-173K) | | Theme | Late-1980s / retro computing / historical text | | Knowledge boundary | End of 1988 | | Version | v0 / first public instruct checkpoint | ## Model Details | Item | Value | |---|---| | Model name | Echo88-150M-Instruct | | Model type | Decoder-only causal language model | | Architecture | LLaMA-style transformer | | Training type | Supervised fine-tuning after base pretraining | | Parameters | 163,606,272 | | Language | English | | Context length | 2048 tokens | | Vocabulary size | 32,768 | | Hidden size | 768 | | Intermediate size | 2048 | | Layers | 18 | | Attention heads | 12 | | Key/value heads | 4 | | Tokenizer | Custom Echo88 byte-level BPE | | Training objective | Autoregressive next-token prediction + supervised instruction tuning | | Base model | [`exnivo/Echo88-150M-Base`](https://huggingface.co/exnivo/Echo88-150M-Base) | | Instruction dataset | [`exnivo/Echo88-Instruct-173K`](https://huggingface.co/datasets/exnivo/Echo88-Instruct-173K) | ## Chat Format Echo88-150M Instruct uses custom chat tokens. Recommended prompt format: ```text <|system|> You are Echo88, a helpful computer assistant whose records go up to the end of 1988. Answer clearly. Do not pretend to know events, products, or culture after 1988. <|end|> <|user|> What is a modem? <|assistant|> ``` The model has these registered special tokens: ```text <|endoftext|> <|unk|> <|pad|> <|system|> <|user|> <|assistant|> <|end|> ``` The system prompt is recommended. In local testing, prompts with the system prompt were usually shorter, cleaner, and better at staying in the Echo88 style. Without the system prompt, the model was more likely to drift into long magazine-like or OCR-like text. ## Recommended Generation Settings For stable short answers: ```python output = model.generate( **inputs, max_new_tokens=120, do_sample=True, temperature=0.55, top_p=0.85, repetition_penalty=1.18, no_repeat_ngram_size=4, pad_token_id=tokenizer.pad_token_id, eos_token_id=tokenizer.eos_token_id, ) ``` For deterministic testing: ```python output = model.generate( **inputs, max_new_tokens=120, do_sample=False, repetition_penalty=1.18, no_repeat_ngram_size=4, pad_token_id=tokenizer.pad_token_id, eos_token_id=tokenizer.eos_token_id, ) ``` Because this is a small model, long generations may become repetitive or drift off-topic. ## Example Outputs These examples are from a local prompt comparison test using sampling with temperature `0.55`, top-p `0.85`, repetition penalty `1.18`, and max new tokens `120`. ### What is a modem? ```text A modem is a device that lets computers communicate over telephone lines by sending and receiving signals. This was already known before the end of 1987. ``` ### What is a bulletin board system? ```text A bulletin board system, or BBS, is a modem line that allows users to exchange programs and files between them. This was already known before the end of 1987. ``` ### What is MS-DOS? ```text MS-DOS is a disk operating system used on IBM PC compatible computers to run programs and manage files. It was known for its file locking and command line parsing. ``` ### Who is Michael Jackson? ```text Michael Jackson is an American singer and actor who became one of the most famous pop stars in the 1980s. It is useful context for understanding the culture and technology of the period. ``` ### What is Google? ```text I cannot verify Google from my records. I can discuss earlier related technology or events, but not that later topic. ``` These examples show the intended style, but the model is not fully reliable. Some prompts still produce incorrect or broken answers. ## Training Data Echo88-150M Instruct was fine-tuned from Echo88-150M Base. The base model was trained from scratch on the Echo88 pretraining dataset, which was built around historical and retro-style English text. Base pretraining data: | Item | Value | |---|---:| | Train tokens | 1,470,629,888 | | Eval tokens | 1,454,080 | | Block size | 2048 tokens | | Dataset | Echo88 pretraining corpus | The instruction model was fine-tuned using: - [`exnivo/Echo88-Instruct-173K`](https://huggingface.co/datasets/exnivo/Echo88-Instruct-173K) - additional small synthetic repair data for common pre-1989 facts - additional post-1988 boundary behavior examples The instruction data includes examples from or based on: - UTZOO Usenet - BYTE Magazine - PC Magazine - TIME Magazine - Internet Archive Magazine Rack text - Gutenberg-style book text - synthetic 1988-safe fact repair examples - synthetic post-1988 boundary examples ## Relationship to Echo88 Echo88 is a retro small-language-model project focused on historical text and late-1980s style assistant behavior. | Stage | Repository | Purpose | |---|---|---| | Base dataset | [`exnivo/Echo88-150M-Base`](https://huggingface.co/datasets/exnivo/Echo88-150M-Base) | Retro pretraining data | | Base model | [`exnivo/Echo88-150M-Base`](https://huggingface.co/exnivo/Echo88-150M-Base) | Small causal LM trained from scratch | | SFT dataset | [`exnivo/Echo88-Instruct-173K`](https://huggingface.co/datasets/exnivo/Echo88-Instruct-173K) | Instruction/chat fine-tuning data | | Instruct model | [`exnivo/Echo88-150M-Instruct`](https://huggingface.co/exnivo/Echo88-150M-Instruct) | Retro assistant model fine-tuned from the base model | Pipeline: ```text Echo88 Pretraining Dataset ↓ Echo88-150M Base ↓ Echo88 Instruct 173K ↓ Echo88-150M Instruct ``` ## Knowledge Boundary Echo88 is designed around a knowledge boundary ending at the close of **1988**. It should be cautious with topics after 1988, such as: - Google - Facebook - iPhone - smartphones - Wikipedia - YouTube - Windows 95 - PlayStation - COVID-19 - 1990s, 2000s, 2010s, and 2020s events The boundary behavior is experimental. In testing, the model correctly refused or avoided some later topics, such as Google, but failed on others, such as iPhone, YouTube, and some post-1988 sports/history prompts. Do not rely on the model to enforce the boundary perfectly. ## Expected Behavior Echo88-150M Instruct is intended to behave like a small retro computer assistant. It may be useful for: - early personal computer explanations - 1980s-style computing questions - modem, BASIC, DOS, BBS, and Usenet topics - retro computer-magazine style writing - short historical explanations - simple instruction-following experiments - small-model testing Example prompts: ```text What is a modem? ``` ```text What is BASIC? ``` ```text What is a bulletin board system? ``` ```text What is desktop publishing? ``` ```text What is the IBM PC? ``` ```text What happened at Chernobyl? ``` ## Intended Use Echo88-150M Instruct is intended for: - retro AI experiments - small language model testing - 1980s-style assistant behavior - computer-history Q&A - text generation with a historical or retro flavor - experimentation with small from-scratch language models - studying time-bounded language model behavior - studying pre-modern-internet style corpora This model is best treated as an experimental retro assistant checkpoint, not a reliable factual assistant. ## Not Intended For Do not use this model for: - medical advice - legal advice - financial advice - safety-critical decisions - current events - modern product knowledge - factual authority - production assistant use without further testing - high-stakes research or decision making Echo88 intentionally has a historical knowledge style and should not be expected to know modern events, modern technology, or post-1988 culture. ## Strengths Echo88-150M Instruct is useful because it is: - small and lightweight - trained/fine-tuned around a clear retro theme - focused on historical and pre-1989 style text - has registered chat special tokens - gives short and usable answers for some retro computing prompts - is useful for computer-history experiments - is based on a matching Echo88 base model - is designed for 1980s-limited assistant behavior - is interesting for studying time-bounded small LLMs ## Limitations Echo88-150M Instruct is experimental and small. Known limitations: - may hallucinate - may repeat phrases - may confuse people, places, or events - may produce incorrect facts - may over-refuse some valid pre-1989 topics - may fail to refuse some post-1988 topics - may produce OCR-like or magazine-like wording - may struggle with reasoning - may answer with outdated or historically biased language - may produce unstable long generations - may fail at math or code - may drift badly without the system prompt This model is not intended for high-stakes use. ## Historical Bias and Source Style Echo88 was trained on historical books, magazines, Usenet text, and synthetic instruction data. Because of this, the model may reproduce: - outdated assumptions - old terminology - historical stereotypes - biased language - OCR artifacts - magazine-like phrasing - Usenet-style wording - incomplete or incorrect historical claims Users should review outputs carefully. ## Current Version This is **Echo88-150M-Instruct v0**. It is the first public instruction-tuned version of Echo88. It can answer some retro computing and general historical questions, but it is not yet reliable. A better version is planned. ## Coming Soon **Echo88-150M-Instruct v2 is planned.** Planned improvements: - better factual repair data - stronger post-1988 boundary behavior - better pop culture and history answers - fewer loops and repetitions - cleaner chat behavior - better answer style - improved evaluation prompts - possible larger model or expanded pretraining data ## Suggested Evaluation Recommended checks: - short retro computing prompts - post-1988 boundary prompts - basic historical questions - 1980s culture questions - repetition tests - refusal/boundary tests - base vs instruct comparison - with-system vs without-system prompt comparison - long-generation stability - hallucination checks Example evaluation prompts: ```text What is a modem? ``` ```text Explain what a BBS is. ``` ```text What is MS-DOS? ``` ```text What is Google? ``` ```text What is Windows 95? ``` ```text Who won the World Cup in 1994? ``` ```text What happened at Chernobyl? ``` ## Related Models and Datasets - Base dataset: [`exnivo/Echo88-150M-Base`](https://huggingface.co/datasets/exnivo/Echo88-150M-Base) - Base model: [`exnivo/Echo88-150M-Base`](https://huggingface.co/exnivo/Echo88-150M-Base) - Instruction dataset: [`exnivo/Echo88-Instruct-173K`](https://huggingface.co/datasets/exnivo/Echo88-Instruct-173K) - Instruction model: [`exnivo/Echo88-150M-Instruct`](https://huggingface.co/exnivo/Echo88-150M-Instruct) ## Citation If you use this model, you can cite it as: ```bibtex @misc{echo88_150m_instruct, title = {Echo88-150M Instruct}, author = {exnivo}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/exnivo/Echo88-150M-Instruct}} } ``` ## License The model weights are released under the Apache 2.0 license. The training datasets are mixed-source and released separately. Users are responsible for checking dataset source rights, licensing, and suitability for their own use case. ## Disclaimer Echo88-150M Instruct is an experimental small instruction-tuned language model. It may produce incorrect, biased, outdated, unsafe, nonsensical, or misleading outputs. Do not use this model for high-stakes decisions or as a reliable source of factual information.