Why researchers lose track of simulations, and what to do about it

Ask any computational researcher how they know which script produced which result, and you will get one of two answers. Either a confident description of a system that sounds reasonable in theory and breaks in practice, or a sheepish admission that they are not entirely sure.

The problem is not laziness. Researchers are meticulous people. The problem is that the tools available to them were built for software engineers deploying web applications, not scientists running parameter sweeps at 3am before a conference deadline.

What goes wrong

The typical lifecycle of a computational result looks like this. You write a script. You run it with some parameters. Something interesting comes out. You make a figure. You move on to the next run, slightly adjusting the parameters.

Six months later, a reviewer asks you to reproduce figure 3. Or your student wants to extend your work. Or you need to check whether the interesting result holds at a different lattice size. And you discover that you have forty-seven files named run_final.jld2 with no obvious mapping back to the code that produced them.

The workarounds researchers reach for are well-intentioned:

• A JSON wrapper script that logs the git hash and parameters
• Folders named by date
• Long file names encoding the parameter values
• A personal spreadsheet

All of them break. The wrapper script breaks when you forget to run it, or when a collaborator runs the script directly. The date-named folders break when you run two things in one day. The encoded file names break when you have twelve parameters. The spreadsheet breaks when you switch machines.

The real problem: friction

Every one of those systems requires you to remember to do something. That is the failure mode. Researchers are thinking about physics, not provenance. The moment you have to consciously act to capture metadata, you will sometimes forget, and sometimes the system will not be available, and eventually the record will have gaps.

The fix is not a better spreadsheet. It is making capture automatic, so that the record exists whether you thought about it or not.

What Fieldnotes does differently

Fieldnotes captures the metadata at the point where you already have it: just after the run. One fieldnotes log command stores the parameters you used, the output file paths, a snapshot of the script that ran, and a git hash if one is available. Nothing goes inside your script; you drive it from the command line, in any language.

There is no wrapper library to import and no logging code to bury in your script. The record is a quick command you run against the thing you were doing anyway, or a one-line manifest your job drops beside its outputs for bulk logging later.

The result is a queryable database of every run you have ever done, searchable by any parameter combination, linkable to any output file. When the reviewer asks about figure 3, you run fieldnotes show --file fig3.pdf and get back the exact parameters, the exact script version, and the path to the data.

That is what reproducibility looks like in practice: not a policy, not a workflow manager, just a record that exists because the cost of creating it was zero.