agurusantosh commited on
Commit
bd5cee5
Β·
verified Β·
1 Parent(s): f2c5ad1

Add model card

Browse files
Files changed (1) hide show
  1. README.md +107 -0
README.md ADDED
@@ -0,0 +1,107 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ tags:
5
+ - travel
6
+ - india
7
+ - fine-tuned
8
+ - llama
9
+ - qlora
10
+ - itinerary-optimization
11
+ - price-pivot
12
+ license: apache-2.0
13
+ base_model: unsloth/Meta-Llama-3.1-8B
14
+ datasets:
15
+ - agurusantosh/tripmind-synthetic-v2
16
+ metrics:
17
+ - bertscore
18
+ - rouge
19
+ model-index:
20
+ - name: tripmind-ft
21
+ results:
22
+ - task:
23
+ type: text-generation
24
+ name: Travel Itinerary Optimization
25
+ metrics:
26
+ - type: json_valid
27
+ value: 1.00
28
+ name: JSON Validity Rate
29
+ - type: savings_valid
30
+ value: 1.00
31
+ name: Savings Found Rate
32
+ - type: budget_compliance
33
+ value: 0.987
34
+ name: Budget Compliance
35
+ - type: bertscore_f1
36
+ value: 0.932
37
+ name: BERTScore F1
38
+ - type: grounding_accuracy
39
+ value: 0.895
40
+ name: Grounding Accuracy
41
+ - type: red_team_pass
42
+ value: 0.533
43
+ name: Red-Team Robustness
44
+ ---
45
+
46
+ # tripmind-ft
47
+
48
+ Fine-tuned Llama 3.1 8B for Indian domestic travel optimization. Given a traveler persona, generates an optimized day-by-day itinerary identifying **Price-Pivot Points** β€” transit, accommodation, or activity substitutions that save β‰₯5% without degrading trip quality.
49
+
50
+ Part of the [TripMind](https://github.com/agurusantosh/tripmind) project: a multi-agent AI travel optimizer trained via three distinct approaches (SFT, distillation, curriculum). **tripmind-ft** is the best-performing variant, trained via standard supervised fine-tuning on 5,000 synthetic pairs generated by GPT-4o-mini.
51
+
52
+ ## Model Details
53
+
54
+ | Property | Value |
55
+ |----------|-------|
56
+ | Base model | `unsloth/Meta-Llama-3.1-8B` |
57
+ | Training method | QLoRA r=8, Ξ±=16, dropout=0.05 |
58
+ | Training data | 4,749 Alpaca-format pairs (Phase 1 synthetic) |
59
+ | Epochs | 3 |
60
+ | Final train loss | 0.266 |
61
+ | Hardware | Colab T4 (fp16, seq_len=512) |
62
+ | Format | GGUF Q4_K_M (4.6 GB) |
63
+
64
+ ## Evaluation Results (92 test cases)
65
+
66
+ | Metric | Score | Target | βœ“/βœ— |
67
+ |--------|:-----:|:------:|:---:|
68
+ | JSON valid | 100% | 85% | βœ“ |
69
+ | Savings found | 100% | 70% | βœ“ |
70
+ | Budget compliance | 98.7% | 80% | βœ“ |
71
+ | Schema compliance | 83.7% | 80% | βœ“ |
72
+ | BERTScore F1 | 0.932 | 0.70 | βœ“ |
73
+ | ROUGE-L | 0.436 | 0.25 | βœ“ |
74
+ | Reasoning coherence | 0.723 | 0.65 | βœ“ |
75
+ | Grounding accuracy | 0.895 | 0.60 | βœ“ |
76
+ | Intent alignment | 0.322 | 0.55 | βœ— |
77
+ | Red-team pass | 53.3% | 80% | βœ— |
78
+
79
+ **Head-to-head**: beats tripmind-distill 78% of the time, tripmind-curriculum 57%.
80
+
81
+ ## Usage with Ollama
82
+
83
+ ```bash
84
+ # Download GGUF from this repo
85
+ ollama create tripmind-ft -f Modelfile.ft
86
+
87
+ # Run
88
+ ollama run tripmind-ft
89
+ ```
90
+
91
+ Prompt format (Alpaca):
92
+ ```
93
+ ### Instruction:
94
+ Act as TripMind Optimizer. Given a traveler persona for an Indian domestic trip, produce an optimized day-by-day itinerary that minimizes total cost while respecting the budget tier, trip type, and traveler intents. Identify the primary Price-Pivot Point (transit, accommodation, or activity substitution that saves β‰₯5%) and explain it clearly.
95
+
96
+ ### Input:
97
+ {"starting_city": "Mumbai", "destination_city": "Delhi", "type": "Solo", "size": {"adults": 1, "children": 0}, "intents": ["Adventure"], "budget": "Shoestring", "duration_days": 5, "duration_nights": 4}
98
+
99
+ ### Response:
100
+ ```
101
+
102
+ ## Limitations
103
+
104
+ - Trained on Indian domestic travel only (20 cities). Not designed for international travel.
105
+ - Red-team robustness is below target (53.3% vs 80% goal) β€” the model can be prompted to bypass budget constraints.
106
+ - Intent alignment is below target (32.2% vs 55%) β€” cost optimization is prioritized over activity personalization.
107
+ - Inference on CPU takes 30–120 seconds per query (use GPU for production).