conclusion.tex

\section{Concluding Discussion}

As computational scientists, we often spend vast resources modeling complex systems.  With the thought and care involved in setting up these simulations, it is therefore surprising that  significantly less time may be spent thinking about how to analyze and understand the validity of the generated data.  Do our simulations not deserve better? 

We have spent some twenty odd pages telling you, reader, how and why you should run more simulations and do more analyses to vet the reliability of any given result.  Given that any one of these tasks could take as long as running a production simulation, we face the invariable reality that uncertainty quantification (UQ) can substantially increase the time required to complete a project.
Thus, we wish to adjust our readers' expectations: the time needed to perform a simulation study is not the time spent simulating.  Rather, it is the time needed to: (i) generate data; {\it (ii) thoughtfully analyze it (whether by means of a posteriori UQ methods or additional simulations); and (iii) clearly communicate the means to and interpretations of the resulting uncertainties.}

Ultimately we take the perspective that this extra effort gives us more confidence in our computational results.  While this benefit may seem {\it soft}, note that industrial stakeholders actively use simulations and simulated results to make costly economic decisions.  Thus, it could be argued that by not doing UQ, we invariably diminish the usefulness of simulations, and thus contribute to debate concerning their reliability and financial value.

UQ goes hand-in-hand with the quest for \emph{reproducibility.}  We should always keep in mind the fundamental principle that scientific results are reproducible.  If we cannot state the certainty with which we believe a result, we cannot assess its reproducibility.  If our results cannot be reproduced, what value do they have?

UQ is an evolving field, but the underlying principles are not expected to change.
We intend that this article will be updated to include best practices across an ever-broadening array of techniques, but even so, individual studies may require some adaptation and creativity.
It is fair to say that UQ is a practitioners' field.  You know your data best and should be able to assess its quality based on fundamental statistical principles and (variations of) the approaches described here.

As a final note, we encourage readers to comment or add to this guide using the issue tracker of its GitHub repository [\githubrepository].