Update README.md

6ed6891 verified 1 day ago

8.64 kB

license: apache-2.0
language:
  - en
base_model: google/flan-t5-small
base_model_relation: adapter
library_name: peft
pipeline_tag: text-generation
datasets:
  - syed7741/aegis-industrial-ai-dataset
tags:
  - peft
  - lora
  - flan-t5
  - rag
  - industrial-ai
  - generative-ai
  - semantic-search
  - document-intelligence
  - robotics
  - predictive-maintenance
  - worker-safety
  - workflow-automation
  - fastapi
  - huggingface

AEGIS Industrial RAG Assistant

AEGIS Industrial RAG Assistant is a LoRA/PEFT adapter fine-tuned on the AEGIS Industrial AI Dataset for industrial question answering and Retrieval-Augmented Generation experiments.

The adapter was trained on top of:

google/flan-t5-small

using:

LoRA / PEFT

The project is part of AEGIS AI, an end-to-end industrial artificial intelligence platform combining RAG, document intelligence, semantic search, AI agents, computer vision, predictive maintenance, robotics monitoring, and workflow automation.

Model Status

✅ This repository contains a genuinely trained LoRA adapter.

The trained adapter weights are stored in:

adapter_model.safetensors

The LoRA configuration is stored in:

adapter_config.json

The adapter was trained locally on CPU using the public AEGIS synthetic industrial dataset.

Base Model

google/flan-t5-small

The original FLAN-T5-small parameters remain the base model.

AEGIS fine-tuning was performed using parameter-efficient LoRA adaptation rather than full-model fine-tuning.

Training Dataset

The adapter was trained using:

syed7741/aegis-industrial-ai-dataset

The dataset currently contains:

64 synthetic industrial records

covering:

Worker Safety
Predictive Maintenance
Robot Monitoring
Vision Inspection
AI Alerts
Workflow Automation
Document Assistant
Factory Status

Industries represented include:

Manufacturing
Oil & Gas
Warehousing / Logistics
Robotics

Training Data Preparation

The original 64 industrial records were split before prompt expansion to reduce leakage between the training and evaluation sets.

Training split:

54 records

Evaluation split:

10 records

Each source record was transformed into multiple instruction/question-answer formats.

Final training examples:

Final evaluation examples:

Fine-Tuning Method

The model was trained using LoRA — Low-Rank Adaptation through Hugging Face PEFT.

Parameter	Value
Base model	google/flan-t5-small
Method	LoRA / PEFT
Task	SEQ_2_SEQ_LM
LoRA rank	4
LoRA alpha	16
LoRA dropout	0.05
Target modules	q, v
Learning rate	3e-4
Batch size	1
Gradient accumulation	4
Epochs	2
Device	CPU

Trainable Parameters

The LoRA configuration trained:

172,032 parameters

out of approximately:

77.1 million total parameters

Trainable percentage:

0.223%

This demonstrates parameter-efficient adaptation without retraining the complete FLAN-T5-small model.

Training Results

Epoch 1

Training Loss:   1.6501
Evaluation Loss: 1.1663

Epoch 2

Training Loss:   1.1880
Evaluation Loss: 0.7744

Both training and evaluation loss decreased during the two training epochs.

Fine-Tuned Test Result

Test question:

What should I do before maintaining CONV-02?

Fine-tuned adapter response:

isolate all energy sources, apply lockout/tagout, verify zero-energy state, and record the responsible technician.

This example demonstrates that the trained adapter learned the expected industrial safety response from the AEGIS training examples.

Using the Adapter

Install the required libraries:

pip install transformers peft torch sentencepiece

Load the AEGIS adapter:

from transformers import (
    AutoModelForSeq2SeqLM,
    AutoTokenizer,
)

from peft import (
    PeftConfig,
    PeftModel,
)


ADAPTER_ID = (
    "syed7741/"
    "aegis-industrial-rag-assistant"
)


config = PeftConfig.from_pretrained(
    ADAPTER_ID
)


base_model = (
    AutoModelForSeq2SeqLM
    .from_pretrained(
        config.base_model_name_or_path
    )
)


tokenizer = (
    AutoTokenizer
    .from_pretrained(
        ADAPTER_ID
    )
)


model = PeftModel.from_pretrained(
    base_model,
    ADAPTER_ID,
)


model.eval()

Example Inference

prompt = """
industrial qa:
Context: Before maintenance on CONV-02,
isolate all energy sources, apply lockout/tagout,
verify zero-energy state, and record the responsible
technician.

Question:
What should I do before maintaining CONV-02?
""".strip()


inputs = tokenizer(
    prompt,
    return_tensors="pt",
)


output = model.generate(
    **inputs,
    max_new_tokens=96,
    num_beams=4,
    do_sample=False,
)


answer = tokenizer.decode(
    output[0],
    skip_special_tokens=True,
)


print(answer)

Expected response:

isolate all energy sources, apply lockout/tagout, verify zero-energy state, and record the responsible technician.

AEGIS RAG Architecture

The trained adapter is designed to work as part of the larger AEGIS Retrieval-Augmented Generation system.

User Question
      ↓
React / TypeScript
      ↓
FastAPI
      ↓
AEGIS RAG Service
      ↓
Sentence Transformer
      ↓
Semantic Vector Search
      ↓
Retrieved Industrial Knowledge
      ↓
AEGIS LoRA Adapter
      ↓
Grounded Response
      ↓
Source Attribution

Embedding Model

The AEGIS RAG pipeline currently uses:

sentence-transformers/all-MiniLM-L6-v2

Embedding dimensions:

The embedding model performs semantic retrieval over the industrial knowledge base before relevant context is supplied to the language model.

Technology Stack

AI / Machine Learning

Hugging Face
Transformers
PEFT
LoRA
FLAN-T5
Sentence Transformers
Retrieval-Augmented Generation
Semantic Search
Vector Embeddings

Backend

Python
FastAPI
REST APIs
PostgreSQL

Frontend

React
TypeScript
Material UI

AI Platform Components

RAG
AI Agents
Document Intelligence
Computer Vision
Predictive Maintenance
Robot Monitoring
Worker Safety
Workflow Automation

Current AEGIS Capabilities

Implemented:

✅ Public Hugging Face industrial dataset
✅ Dataset loader
✅ Document construction
✅ Text chunking
✅ Sentence-transformer embeddings
✅ Vector indexing
✅ Semantic retrieval
✅ Local language model
✅ FastAPI RAG endpoint
✅ React / TypeScript integration
✅ Retrieved-source attribution
✅ LoRA/PEFT fine-tuning
✅ Trained adapter checkpoint
✅ Hugging Face model repository
✅ CPU-based training pipeline
✅ No paid LLM API required

Development Roadmap

Planned improvements:

Larger industrial training dataset
Arabic + English training data
Arabic industrial terminology
Multilingual question answering
RAG evaluation suite
Base-model vs fine-tuned-model benchmarking
Hybrid semantic + keyword retrieval
Reranking
AI agents
Conversation memory
Document ingestion
Computer vision integration
Workflow automation
Cloud deployment

Enterprise AI Engineering

AEGIS demonstrates concepts applicable to enterprise AI systems including:

LLM application development
Parameter-efficient fine-tuning
Retrieval-Augmented Generation
Document intelligence
Semantic search
Conversational AI
Multilingual AI
AI backend APIs
Workflow automation
Grounded generation
Source attribution

Safety Notice

The AEGIS training dataset contains synthetic industrial records created for:

AI engineering experimentation
learning
prototyping
research
portfolio demonstration

The model must not be treated as an authoritative source for industrial safety or operational decisions.

Its outputs must not replace:

approved operating procedures
manufacturer documentation
workplace safety requirements
engineering review
regulatory requirements
qualified professional judgment

Related Work

Dataset

syed7741/aegis-industrial-ai-dataset

Model / Adapter

syed7741/aegis-industrial-rag-assistant

GitHub

github.com/syedasim7741/AEGIS-AI

Author

Sayyad Asim

AI Engineering • RAG • AI Agents • LLM Fine-Tuning • Document Intelligence • Computer Vision • Robotics • Industrial AI