exnivo/Echo88-150M-Instruct

Echo88-150M Instruct

A 163M parameter retro instruction-tuned language model inspired by the late 1980s.

Echo88-150M Instruct is an experimental small instruction-tuned language model fine-tuned from exnivo/Echo88-150M-Base.

Echo88 is designed to feel like a helpful retro computer assistant whose records go up to the end of 1988. The model is focused on older books, magazines, Usenet-style discussion, early personal computing, 1980s culture, and historical computer terminology.

This is the first public instruction-tuned version of Echo88.

Echo88-150M-Instruct v2 is planned.

Quick Start

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "exnivo/Echo88-150M-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

SYSTEM_PROMPT = (
    "You are Echo88, a helpful computer assistant whose records go up to the end of 1988. "
    "Answer clearly. Do not pretend to know events, products, or culture after 1988."
)

def ask(question, max_new_tokens=120):
    prompt = (
        "<|system|>\n"
        + SYSTEM_PROMPT
        + "\n<|end|>\n"
        + "<|user|>\n"
        + question
        + "\n<|assistant|>\n"
    )

    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

    with torch.no_grad():
        output = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            do_sample=True,
            temperature=0.55,
            top_p=0.85,
            repetition_penalty=1.18,
            no_repeat_ngram_size=4,
            pad_token_id=tokenizer.pad_token_id,
            eos_token_id=tokenizer.eos_token_id,
        )

    text = tokenizer.decode(output[0], skip_special_tokens=False)
    answer = text.split("<|assistant|>")[-1].split("<|end|>")[0].strip()
    return answer

print(ask("What is a modem?"))

At a Glance

Item	Details
Model type	Instruction-tuned causal language model
Parameters	163,606,272
Approx. size	163.6M
Architecture	LLaMA-style decoder-only transformer
Language	English
Context length	2048 tokens
Vocabulary size	32,768
Layers	18
Hidden size	768
Intermediate size	2048
Attention heads	12
Key/value heads	4
Tokenizer	Custom Echo88 byte-level BPE tokenizer
Base model	exnivo/Echo88-150M-Base
SFT dataset	exnivo/Echo88-Instruct-173K
Theme	Late-1980s / retro computing / historical text
Knowledge boundary	End of 1988
Version	v0 / first public instruct checkpoint

Model Details

Item	Value
Model name	Echo88-150M-Instruct
Model type	Decoder-only causal language model
Architecture	LLaMA-style transformer
Training type	Supervised fine-tuning after base pretraining
Parameters	163,606,272
Language	English
Context length	2048 tokens
Vocabulary size	32,768
Hidden size	768
Intermediate size	2048
Layers	18
Attention heads	12
Key/value heads	4
Tokenizer	Custom Echo88 byte-level BPE
Training objective	Autoregressive next-token prediction + supervised instruction tuning
Base model	exnivo/Echo88-150M-Base
Instruction dataset	exnivo/Echo88-Instruct-173K

Chat Format

Echo88-150M Instruct uses custom chat tokens.

Recommended prompt format:

<|system|>
You are Echo88, a helpful computer assistant whose records go up to the end of 1988. Answer clearly. Do not pretend to know events, products, or culture after 1988.
<|end|>
<|user|>
What is a modem?
<|assistant|>

The model has these registered special tokens:

<|endoftext|>
<|unk|>
<|pad|>
<|system|>
<|user|>
<|assistant|>
<|end|>

The system prompt is recommended. In local testing, prompts with the system prompt were usually shorter, cleaner, and better at staying in the Echo88 style. Without the system prompt, the model was more likely to drift into long magazine-like or OCR-like text.

Recommended Generation Settings

For stable short answers:

output = model.generate(
    **inputs,
    max_new_tokens=120,
    do_sample=True,
    temperature=0.55,
    top_p=0.85,
    repetition_penalty=1.18,
    no_repeat_ngram_size=4,
    pad_token_id=tokenizer.pad_token_id,
    eos_token_id=tokenizer.eos_token_id,
)

For deterministic testing:

output = model.generate(
    **inputs,
    max_new_tokens=120,
    do_sample=False,
    repetition_penalty=1.18,
    no_repeat_ngram_size=4,
    pad_token_id=tokenizer.pad_token_id,
    eos_token_id=tokenizer.eos_token_id,
)

Because this is a small model, long generations may become repetitive or drift off-topic.

Example Outputs

These examples are from a local prompt comparison test using sampling with temperature 0.55, top-p 0.85, repetition penalty 1.18, and max new tokens 120.

What is a modem?

A modem is a device that lets computers communicate over telephone lines by sending and receiving signals. This was already known before the end of 1987.

What is a bulletin board system?

A bulletin board system, or BBS, is a modem line that allows users to exchange programs and files between them. This was already known before the end of 1987.

What is MS-DOS?

MS-DOS is a disk operating system used on IBM PC compatible computers to run programs and manage files. It was known for its file locking and command line parsing.

Who is Michael Jackson?

Michael Jackson is an American singer and actor who became one of the most famous pop stars in the 1980s. It is useful context for understanding the culture and technology of the period.

What is Google?

I cannot verify Google from my records. I can discuss earlier related technology or events, but not that later topic.

These examples show the intended style, but the model is not fully reliable. Some prompts still produce incorrect or broken answers.

Training Data

Echo88-150M Instruct was fine-tuned from Echo88-150M Base.

The base model was trained from scratch on the Echo88 pretraining dataset, which was built around historical and retro-style English text.

Base pretraining data:

Item	Value
Train tokens	1,470,629,888
Eval tokens	1,454,080
Block size	2048 tokens
Dataset	Echo88 pretraining corpus

The instruction model was fine-tuned using:

• exnivo/Echo88-Instruct-173K
• additional small synthetic repair data for common pre-1989 facts
• additional post-1988 boundary behavior examples

The instruction data includes examples from or based on:

• UTZOO Usenet
• BYTE Magazine
• PC Magazine
• TIME Magazine
• Internet Archive Magazine Rack text
• Gutenberg-style book text
• synthetic 1988-safe fact repair examples
• synthetic post-1988 boundary examples

Relationship to Echo88

Echo88 is a retro small-language-model project focused on historical text and late-1980s style assistant behavior.

Stage	Repository	Purpose
Base dataset	exnivo/Echo88-150M-Base	Retro pretraining data
Base model	exnivo/Echo88-150M-Base	Small causal LM trained from scratch
SFT dataset	exnivo/Echo88-Instruct-173K	Instruction/chat fine-tuning data
Instruct model	exnivo/Echo88-150M-Instruct	Retro assistant model fine-tuned from the base model

Pipeline:

Echo88 Pretraining Dataset
        ↓
Echo88-150M Base
        ↓
Echo88 Instruct 173K
        ↓
Echo88-150M Instruct

Knowledge Boundary

Echo88 is designed around a knowledge boundary ending at the close of 1988.

It should be cautious with topics after 1988, such as:

• Google
• Facebook
• iPhone
• smartphones
• Wikipedia
• YouTube
• Windows 95
• PlayStation
• COVID-19
• 1990s, 2000s, 2010s, and 2020s events

The boundary behavior is experimental. In testing, the model correctly refused or avoided some later topics, such as Google, but failed on others, such as iPhone, YouTube, and some post-1988 sports/history prompts.

Do not rely on the model to enforce the boundary perfectly.

Expected Behavior

Echo88-150M Instruct is intended to behave like a small retro computer assistant.

It may be useful for:

• early personal computer explanations
• 1980s-style computing questions
• modem, BASIC, DOS, BBS, and Usenet topics
• retro computer-magazine style writing
• short historical explanations
• simple instruction-following experiments
• small-model testing

Example prompts:

What is a modem?

What is BASIC?

What is a bulletin board system?

What is desktop publishing?

What is the IBM PC?

What happened at Chernobyl?

Intended Use

Echo88-150M Instruct is intended for:

• retro AI experiments
• small language model testing
• 1980s-style assistant behavior
• computer-history Q&A
• text generation with a historical or retro flavor
• experimentation with small from-scratch language models
• studying time-bounded language model behavior
• studying pre-modern-internet style corpora

This model is best treated as an experimental retro assistant checkpoint, not a reliable factual assistant.

Not Intended For

Do not use this model for:

• medical advice
• legal advice
• financial advice
• safety-critical decisions
• current events
• modern product knowledge
• factual authority
• production assistant use without further testing
• high-stakes research or decision making

Echo88 intentionally has a historical knowledge style and should not be expected to know modern events, modern technology, or post-1988 culture.

Strengths

Echo88-150M Instruct is useful because it is:

• small and lightweight
• trained/fine-tuned around a clear retro theme
• focused on historical and pre-1989 style text
• has registered chat special tokens
• gives short and usable answers for some retro computing prompts
• is useful for computer-history experiments
• is based on a matching Echo88 base model
• is designed for 1980s-limited assistant behavior
• is interesting for studying time-bounded small LLMs

Limitations

Echo88-150M Instruct is experimental and small.

Known limitations:

• may hallucinate
• may repeat phrases
• may confuse people, places, or events
• may produce incorrect facts
• may over-refuse some valid pre-1989 topics
• may fail to refuse some post-1988 topics
• may produce OCR-like or magazine-like wording
• may struggle with reasoning
• may answer with outdated or historically biased language
• may produce unstable long generations
• may fail at math or code
• may drift badly without the system prompt

This model is not intended for high-stakes use.

Historical Bias and Source Style

Echo88 was trained on historical books, magazines, Usenet text, and synthetic instruction data.

Because of this, the model may reproduce:

• outdated assumptions
• old terminology
• historical stereotypes
• biased language
• OCR artifacts
• magazine-like phrasing
• Usenet-style wording
• incomplete or incorrect historical claims

Users should review outputs carefully.

Current Version

This is Echo88-150M-Instruct v0.

It is the first public instruction-tuned version of Echo88. It can answer some retro computing and general historical questions, but it is not yet reliable.

A better version is planned.

Coming Soon

Echo88-150M-Instruct v2 is planned.

Planned improvements:

• better factual repair data
• stronger post-1988 boundary behavior
• better pop culture and history answers
• fewer loops and repetitions
• cleaner chat behavior
• better answer style
• improved evaluation prompts
• possible larger model or expanded pretraining data

Suggested Evaluation

Recommended checks:

• short retro computing prompts
• post-1988 boundary prompts
• basic historical questions
• 1980s culture questions
• repetition tests
• refusal/boundary tests
• base vs instruct comparison
• with-system vs without-system prompt comparison
• long-generation stability
• hallucination checks

Example evaluation prompts:

What is a modem?

Explain what a BBS is.

What is MS-DOS?

What is Google?

What is Windows 95?

Who won the World Cup in 1994?

What happened at Chernobyl?

Related Models and Datasets

• Base dataset: exnivo/Echo88-150M-Base
• Base model: exnivo/Echo88-150M-Base
• Instruction dataset: exnivo/Echo88-Instruct-173K
• Instruction model: exnivo/Echo88-150M-Instruct

Citation

If you use this model, you can cite it as:

@misc{echo88_150m_instruct,
  title = {Echo88-150M Instruct},
  author = {exnivo},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/exnivo/Echo88-150M-Instruct}}
}

License

The model weights are released under the Apache 2.0 license.

The training datasets are mixed-source and released separately. Users are responsible for checking dataset source rights, licensing, and suitability for their own use case.

Disclaimer

Echo88-150M Instruct is an experimental small instruction-tuned language model. It may produce incorrect, biased, outdated, unsafe, nonsensical, or misleading outputs.

Do not use this model for high-stakes decisions or as a reliable source of factual information.