TTool Evaluation #16
TTool Evaluation #16
Conversation
…ted the export of latex and plots
|
Hello! You can find in the folder Here's what each hole corresponding to which tool/number:
|
|
Hello @sushidelivery, first, well done again for the experimental campaign! Now let's try to wrap up everything so that we can put a milestone on the tool. Here are the todos:
If something is not clear, let me know. |
eval/script/visuals.py
Outdated
| def export_latex_table(csv_path: str, out_path: str) -> None: | ||
| """ | ||
| Exports the latex table of the statistics (mean, median, std, min, max, q1, q3) | ||
|
|
||
| Args: | ||
| csv_path (str): The path to the csv files | ||
| Returns: | ||
| None | ||
| """ | ||
| csv_files = glob.glob(os.path.join(csv_path, '*_error.csv')) | ||
| for csv_file in csv_files: | ||
| save_path = os.path.join(out_path, 'latex_{}.tex'.format(csv_file.split('/')[-1].split('.')[0])) | ||
| data = pd.read_csv(csv_file) | ||
| data_dict = data.to_dict(orient='records') | ||
| latex_table = tabulate(data_dict, headers="keys", tablefmt="latex") | ||
| with open(save_path, 'w') as file: | ||
| file.write(latex_table) | ||
| print(f"\033[90m[INFO]: Latex table is exported to {save_path}\033[0m") |
There was a problem hiding this comment.
This part should be changed slightly. After working on the tslam I realized we should include the standard deviations valid and a different formating would work better. Could ou please reformat this function to achieve this latex table please? Note that the names are shorter, we should do the same.
And here's the latex for reference:
\begin{table}[htbp]
\centering
\resizebox{\textwidth}{!}{\begin{tabular}{c|c c c c c c c c c}
\toprule
Tool & Number of & Mean position & Mean orientation & Mean tags & Mean coverage \\
name & operations & error * & error * & detection * & index \\
(-) & (–) & (mm) & (\degree) & (-) & (\%) \\
\midrule
circular & \\
sawblade & 32 & 0.15 ± 0.09 & 0.01 ± 0.01 & 3 ± 2 & 72.30 \\
saber & \\
sawblade & 3 & 1.58 ± 8.54 & 0.01 ± 0.07 & 7 ± 7 & 21.57 \\
drill & \\
hing (\diameter50) & 7 & 0.18 ± 4.32 & 0.01 ± 0.04 & 6 ± 5 & 68.75 \\
drill & \\
auger (\diameter20) & 19 & 0.15 ± 3.77 & 0.07 ± 0.03 & 6 ± 4 & 93.12 \\
drill & \\
auger (\diameter25) & 2 & 0.37 ± 0.61 & 0.04 ± 0.02 & 8 ± 6 & 98.51 \\
drill & \\
oblique (\diameter40) & 2 & 0.22 ± 0.47 & 0.21 ± 0.11 & 8 ± 3 & 99.28 \\
screw & \\
(120) & 8 & 0.16 ± 1.87 & 0.18 ± 0.09 & 6 ± 4 & 95.23 \\
screw & \\
(100) & 8 & 0.29 ± 2.04 & 0.12 ± 0.06 & 7 ± 4 & 97.99 \\
screw & \\
(80) & 8 & 0.01 ± 3.16 & 0.21 ± 0.10 & 8 ± 3 & 97.77 \\
screw & \\
(45) & 8 & 0.01 ± 1.19 & 0.04 ± 0.02 & 6 ± 3 & 97.50 \\
\bottomrule
\multicolumn{6}{l}{\textsuperscript{*}{The errors are represented in mean ± interquartile range (IQR=Q3-Q1)}}
\end{tabular}}
\caption{Table resuming the error metrics for the one fabrication session on a beam with low density and single-stripe layout as tags' distribution.} \label{tab:results:singletools}
\end{table}
eval/script/visuals.py
Outdated
| def draw_boxplot_from_csv(csv_file: str, _name: str) -> plt.figure: | ||
| """ | ||
| Draws the boxplot of the statistics (mean, median, std, min, max, q1, q3) | ||
|
|
||
| Args: | ||
| csv_path (str): The path to the csv files | ||
| _name (str): The name of the plot | ||
| Returns: | ||
| plt.figure: The boxplot | ||
| """ | ||
| data = pd.read_csv(csv_file) | ||
|
|
||
| _name = ' '.join([word.capitalize() for word in _name.split('_')]) | ||
| metrics_info = data[['Max', 'Min', 'Mean', 'Median', 'Std', 'Q1', 'Q3']] | ||
| transposed_data = metrics_info.T | ||
|
|
||
| fig, ax = plt.subplots(figsize=(10., 6.)) | ||
| ax.set_xlabel("Tools' names") | ||
| ax.spines['right'].set_visible(False) | ||
| ax.spines['top'].set_visible(False) | ||
| ax.spines['left'].set_visible(True) | ||
| ax.spines['bottom'].set_visible(True) | ||
|
|
||
| if _name.split(' ')[0] == 'Rotation': | ||
| ax.set_ylabel(f"Angular Error [deg]") | ||
| else: | ||
| ax.set_ylabel(f"Distance Error [mm]") | ||
|
|
||
| boxplot_elements = plt.boxplot(transposed_data.values, labels=data['Name'], notch=False, sym='+', vert=True, | ||
| whis=1.5, positions=None, widths=None, bootstrap=None, usermedians=None, | ||
| conf_intervals=None, showfliers=False) | ||
| data_range = max(metrics_info.max()) - min(metrics_info.min()) | ||
| step = data_range * 0.025 | ||
|
|
||
| for whisker in boxplot_elements['whiskers']: | ||
| whisker.set(color='black', linewidth=1) | ||
| for cap in boxplot_elements['caps']: | ||
| cap.set(color='black', linewidth=2) | ||
| for median in boxplot_elements['medians']: | ||
| median.set(linewidth=0) | ||
| for flier in boxplot_elements['fliers']: | ||
| flier.set(marker='+', color="black", alpha=0.5) | ||
| for i, box in enumerate(boxplot_elements['boxes']): | ||
| x = box.get_xdata()[0] | ||
| top_of_box = box.get_ydata()[2] | ||
|
|
||
| max_val = metrics_info['Max'].iloc[i] | ||
| min_val = metrics_info['Min'].iloc[i] | ||
| mean_val = metrics_info['Mean'].iloc[i] | ||
| q1_val = metrics_info['Q1'].iloc[i] | ||
| q3_val = metrics_info['Q3'].iloc[i] | ||
|
|
||
| positions = np.linspace(start=top_of_box + step, stop=top_of_box + 6 * step, num=6) | ||
|
|
||
| plt.text(x, positions[0], f"Mx={max_val:.2f}", ha='center', va='bottom', fontsize='x-small') | ||
| plt.text(x, positions[1], f"mn={mean_val:.2f}", ha='center', va='bottom', fontsize='x-small') | ||
| plt.text(x, positions[2], f"q3={q3_val:.2f}", ha='center', va='bottom', fontsize='x-small') | ||
| plt.text(x, positions[3], f"q1={q1_val:.2f}", ha='center', va='bottom', fontsize='x-small') | ||
| plt.text(x, positions[4], f"M={min_val:.2f}", ha='center', va='bottom', fontsize='x-small') | ||
|
|
||
| plt.plot([i + 1 - 0.25, i + 1 + 0.25], [mean_val, mean_val], color=CYBERGREEN, linewidth=2) | ||
|
|
||
| mean_legend_entry = Line2D([0], [0], color=CYBERGREEN, linewidth=2, label='Mean') | ||
| plt.legend(handles=[mean_legend_entry], loc='upper right', bbox_to_anchor=(1.1, 1.1), fontsize='x-small') | ||
| plt.xticks(rotation=10) | ||
| plt.tight_layout() | ||
|
|
||
| return plt |
There was a problem hiding this comment.
Here's there are few thing to adjust. Seeing the last version:
Here's the todo:
- (a) the lines should be closer as you did in the previous versions. Here there is way too much space between them
- (b) the M,q1,q2 etc labels should be offset up from the higher black bar (q4) and not the mean as you doing now. This will avoid the text to be cut by the boxplots' axes as in
auger_drill_bit_20_235. - (c) give the the box plot axis X's ticks more in-between space and start the box plots from a further distance from the axis y.
Let me know if something stays unclear we can discuss this. But take as an example the following graph I did for TSlam:
There was a problem hiding this comment.
Hello @9and3! Appreciate your thorough review, as always! Here's the update:
(1) All data (logs, recordings, screenshots) is now stored on both the external hard disk and the server. ✅
(2) Implemented the changes to the graph, and it's looking good so far. However, I'm currently addressing a minor issue that requires further attention. ⏳
(3) Currently in the process of writing. ⏳
(4) Modified the code to enable outputting graphs per session and a final one. ✅
(5) Similarly, done without the self_feeding_bit_40_90. ✅
(6) The User Experience results are registered in the .txt format. ✅
(7) I will think about the progression graph, as discussed :) ⏳
(8) Code and documentation cleanup is currently in progress. ⏳
Your ongoing support is much appreciated!
There was a problem hiding this comment.
Also, the data location:
- in the external hard disk : /JG_Disk04/2023_ttool_eval_media
- on the server: /media/18TB_ibois_stock/2023_ttool_eval_media
the readme for the folders will be provided soon.
|
From our meeting:
|
|
Hello @9and3 ! Also the readme to the output folders stored in the server is added ✅ |
looks good! |
|
The boxplot is fixed too. Please, have a look :) |
This PR includes the initial implementation of the TTool's evaluation. It includes functions that perform evaluations, plot graphs and latex tables for analysis.