gabriellarson/ICONN-1-GGUF

ICONN 1

We proudly introduce ICONN-1, the most advanced and human-like open-source artificial intelligence model under 100B parameters of its time. Designed to push the boundaries of natural language understanding and generation, ICONN-1 is built on a Mixture-of-Experts (MoE) architecture that enables dynamic routing through specialized expert pathways, allowing for both computational efficiency and enhanced performance.

Developed entirely from scratch, ICONN-1 is based on a customized Mixtral framework and comprises 88 billion parameters, with 22 billion parameters actively utilized per token. This approach allows ICONN-1 to deliver highly nuanced and contextually accurate responses while maintaining the scalability benefits of sparse activation.

ICONN-1 is released in two distinct forms to serve different application needs:

• ICONN-1 (this version) is optimized for natural, emotionally resonant, and conversational interactions.
• ICONN-e1 is a specialized variant of the model fine-tuned for advanced reasoning, critical analysis, and complex problem-solving.

Together, these models represent a major leap forward in the evolution of AI systems—demonstrating not only deep reasoning but also a commitment to openness, accessibility, and human-aligned intelligence.

These models were each benchmarked on a collection of 500 questions to compare output to a human for emotion and common sense. Benchmark performance may vary due to the stochastic nature of AI models. ICONN 1 retains the highest human-thinking benchmark score through many tests on different temperatures.

Usage

System Requirements

To run ICONN 1 effectively, ensure you have:

• 4× NVIDIA A100 GPUs or a single NVIDIA B100
• At least 120 GB of system RAM
• 120–192 GB of GPU VRAM

If your system does not meet these requirements—which may be the case for many users—you can still experience ICONN through alternative options:

• Use a quantized version of ICONN for lower resource consumption
• Try the lightweight ICONN 1 Mini (7B)

Run the code below to run ICONN 1:

from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
import torch

def run_iconn_chatbot(model_name="ICONNAI/ICONN-1"):

    tokenizer = AutoTokenizer.from_pretrained(model_name)
    model = AutoModelForCausalLM.from_pretrained(model_name)
    
    device = 0 if torch.cuda.is_available() else -1
    
    chat_pipeline = pipeline(
        "text-generation",
        model=model,
        tokenizer=tokenizer,
        device=device,
        max_length=1624,
        do_sample=True,
        top_p=0.9,
        temperature=0.4,
        pad_token_id=tokenizer.eos_token_id
    )
    
    print(f"ICONN chatbot running with model: {model_name}. Type 'exit' to quit.")
    conversation_history = ""
    
    while True:
        user_input = input("You: ")
        if user_input.lower() == "exit":
            print("Goodbye!")
            break
        
        conversation_history += f"User: {user_input}\nBot:"
        
        response = chat_pipeline(conversation_history, max_length=len(tokenizer.encode(conversation_history)) + 100)[0]['generated_text']
        
        bot_reply = response[len(conversation_history):].strip().split("\n")[0]
        
        print(f"Bot: {bot_reply}")
        
        conversation_history += f" {bot_reply}\n"

if __name__ == "__main__":
    run_iconn_chatbot()