chore: tp eval

zhudotexe · Dec 11, 2024 · edfe2ad · edfe2ad
1 parent 5eec0a4
commit edfe2ad
Show file tree

Hide file tree

Showing 13 changed files with 939 additions and 221 deletions.
diff --git a/README.md b/README.md
@@ -195,8 +195,6 @@ This will output a `score.json` file in the output path with the final scores.
 python bench_travelplanner.py <full|root-fc|baseline|small-leaf|small-all|small-baseline>
 ```
 
-Note: This benchmark does not test the `short-ctx` systems since this benchmark doesn't have a long-context requirement.
-
 **Evaluate**
 
 ```shell

diff --git a/experiments/travelplanner/cohere-hf/full/results_for_tp_eval.jsonl b/experiments/travelplanner/cohere-hf/full/results_for_tp_eval.jsonl
diff --git a/experiments/travelplanner/cohere-hf/short-baseline/results_for_tp_eval.jsonl b/experiments/travelplanner/cohere-hf/short-baseline/results_for_tp_eval.jsonl
diff --git a/experiments/travelplanner/cohere-hf/short-context/results_for_tp_eval.jsonl b/experiments/travelplanner/cohere-hf/short-context/results_for_tp_eval.jsonl
diff --git a/experiments/travelplanner/cohere-hf/small-all/results_for_tp_eval.jsonl b/experiments/travelplanner/cohere-hf/small-all/results_for_tp_eval.jsonl
diff --git a/experiments/travelplanner/openai/baseline/results_for_tp_eval.jsonl b/experiments/travelplanner/openai/baseline/results_for_tp_eval.jsonl
diff --git a/experiments/travelplanner/openai/full/results_for_tp_eval.jsonl b/experiments/travelplanner/openai/full/results_for_tp_eval.jsonl
diff --git a/experiments/travelplanner/openai/root-fc/results_for_tp_eval.jsonl b/experiments/travelplanner/openai/root-fc/results_for_tp_eval.jsonl
diff --git a/experiments/travelplanner/openai/short-baseline/results_for_tp_eval.jsonl b/experiments/travelplanner/openai/short-baseline/results_for_tp_eval.jsonl
diff --git a/experiments/travelplanner/openai/short-context/results_for_tp_eval.jsonl b/experiments/travelplanner/openai/short-context/results_for_tp_eval.jsonl
diff --git a/experiments/travelplanner/openai/small-all/results_for_tp_eval.jsonl b/experiments/travelplanner/openai/small-all/results_for_tp_eval.jsonl
diff --git a/experiments/travelplanner/openai/small-baseline/results_for_tp_eval.jsonl b/experiments/travelplanner/openai/small-baseline/results_for_tp_eval.jsonl
diff --git a/experiments/travelplanner/openai/small-leaf/results_for_tp_eval.jsonl b/experiments/travelplanner/openai/small-leaf/results_for_tp_eval.jsonl