ishreyadev commited on
Commit
b4741a1
Β·
verified Β·
1 Parent(s): 384f729

Upload folder using huggingface_hub

Browse files
Files changed (4) hide show
  1. .gitattributes +1 -0
  2. README.md +117 -0
  3. gitattributes +38 -0
  4. pivotai_ft.Q4_K_M.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ pivotai_ft.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,117 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ tags:
5
+ - travel
6
+ - india
7
+ - fine-tuned
8
+ - llama
9
+ - qlora
10
+ - itinerary-optimization
11
+ - price-pivot
12
+ license: apache-2.0
13
+ base_model: unsloth/Meta-Llama-3.1-8B
14
+ datasets:
15
+ - ishreyadev/pivotai-synthetic-v2
16
+ metrics:
17
+ - bertscore
18
+ - rouge
19
+ model-index:
20
+ - name: pivotai-ft
21
+ results:
22
+ - task:
23
+ type: text-generation
24
+ name: Travel Itinerary Optimization
25
+ metrics:
26
+ - type: json_valid
27
+ value: 1.00
28
+ name: JSON Validity Rate
29
+ - type: savings_valid
30
+ value: 1.00
31
+ name: Savings Found Rate
32
+ - type: budget_compliance
33
+ value: 0.987
34
+ name: Budget Compliance
35
+ - type: bertscore_f1
36
+ value: 0.932
37
+ name: BERTScore F1
38
+ - type: grounding_accuracy
39
+ value: 0.895
40
+ name: Grounding Accuracy
41
+ - type: red_team_pass
42
+ value: 0.533
43
+ name: Red-Team Robustness
44
+ ---
45
+
46
+ # pivotai-ft
47
+
48
+ Fine-tuned Llama 3.1 8B for Indian domestic travel optimization. Given a traveler persona, generates an optimized day-by-day itinerary identifying **Price-Pivot Points** β€” transit, accommodation, or activity substitutions that save β‰₯5% without degrading trip quality.
49
+
50
+ Part of the [pivotai](https://github.com/aguru-venkata-saisantosh-patnaik/Agentic-LLM-System_MCP-Orchestration-Fine-Tuning-and-Comparative-Evaluation) project: a multi-agent AI travel optimizer trained via three distinct approaches (SFT, distillation, curriculum). **pivotai-ft** is the best-performing variant, trained via standard supervised fine-tuning on 5,000 synthetic pairs generated by GPT-4o-mini.
51
+
52
+ ## Model Details
53
+
54
+ | Property | Value |
55
+ |----------|-------|
56
+ | Base model | `unsloth/Meta-Llama-3.1-8B` |
57
+ | Training method | QLoRA r=8, Ξ±=16, dropout=0.05 |
58
+ | Training data | 4,749 Alpaca-format pairs (Phase 1 synthetic) |
59
+ | Epochs | 3 |
60
+ | Final train loss | 0.266 |
61
+ | Hardware | Colab T4 (fp16, seq_len=512) |
62
+ | Format | GGUF Q4_K_M (4.6 GB) |
63
+
64
+ ## Evaluation Results (92 test cases)
65
+
66
+ | Metric | Score | Target | βœ“/βœ— |
67
+ |--------|:-----:|:------:|:---:|
68
+ | JSON valid | 100% | 85% | βœ“ |
69
+ | Savings found | 100% | 70% | βœ“ |
70
+ | Budget compliance | 98.7% | 80% | βœ“ |
71
+ | Schema compliance | 83.7% | 80% | βœ“ |
72
+ | BERTScore F1 | 0.932 | 0.70 | βœ“ |
73
+ | ROUGE-L | 0.436 | 0.25 | βœ“ |
74
+ | Reasoning coherence | 0.723 | 0.65 | βœ“ |
75
+ | Grounding accuracy | 0.895 | 0.60 | βœ“ |
76
+ | Intent alignment | 0.322 | 0.55 | βœ— |
77
+ | Red-team pass | 53.3% | 80% | βœ— |
78
+
79
+ **Head-to-head**: beats pivotai-distill 78% of the time, pivotai-curriculum 57%.
80
+
81
+ ## Usage with Ollama
82
+
83
+ ```bash
84
+ # Download GGUF from this repo
85
+ ollama create pivotai-ft -f Modelfile.ft
86
+
87
+ # Run
88
+ ollama run pivotai-ft
89
+ ```
90
+
91
+ Prompt format (Alpaca):
92
+ ```
93
+ ### Instruction:
94
+ Act as pivotai Optimizer. Given a traveler persona for an Indian domestic trip, produce an optimized day-by-day itinerary that minimizes total cost while respecting the budget tier, trip type, and traveler intents. Identify the primary Price-Pivot Point (transit, accommodation, or activity substitution that saves β‰₯5%) and explain it clearly.
95
+
96
+ ### Input:
97
+ {"starting_city": "Mumbai", "destination_city": "Delhi", "type": "Solo", "size": {"adults": 1, "children": 0}, "intents": ["Adventure"], "budget": "Shoestring", "duration_days": 5, "duration_nights": 4}
98
+
99
+ ### Response:
100
+ ```
101
+
102
+ ## Limitations
103
+
104
+ - Trained on Indian domestic travel only (20 cities). Not designed for international travel.
105
+ - Red-team robustness is below target (53.3% vs 80% goal) β€” the model can be prompted to bypass budget constraints.
106
+ - Intent alignment is below target (32.2% vs 55%) β€” cost optimization is prioritized over activity personalization.
107
+ - Inference on CPU takes 30–120 seconds per query (use GPU for production).
108
+
109
+ ## Citation
110
+
111
+ If you use this model, please cite:
112
+
113
+ ```
114
+ Patnaik, A. V. S. (2026). Cost-Matched Data Generation for LLM Fine-Tuning: Comparing
115
+ Supervised Fine-Tuning, Knowledge Distillation, and Curriculum Learning for an Agentic
116
+ Travel-Planning System. Zenodo. https://doi.org/10.5281/zenodo.21198884
117
+ ```
gitattributes ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ pivotai_ft.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ WHITEPAPER.pdf filter=lfs diff=lfs merge=lfs -text
38
+ whitepaper.pdf filter=lfs diff=lfs merge=lfs -text
pivotai_ft.Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e2f26ccf5a00703dffd9ebdd2e004220eb04dd90c8661e91d2b310211c691aff
3
+ size 4920734336