Our recent interviews with NetLogo users identified tools for testing code as a development need. And recent literature on good practices in agent-based modeling identifies the lack of code testing, or even a culture of testing, as a major concern–a sign that agent-based modeling is not yet a mature approach to science. Building tools for testing code into NetLogo would make it easier to test code, and would promote a code-testing culture by reminding users that code is not finished until it’s tested.
Here I describe an idea I have for a code testing tool, and I look forward to seeing other ideas.
I often write custom code to test a particular submodel by executing the submodel over wide ranges of values for several different inputs. For example, I might test a fish growth submodel using a test procedure that writes to file the growth rate calculated by the submodel over wide ranges of food intake, swimming speed, temperature, and fish size. Then I analyze that output in Excel (etc.) to search for errors.
I can envision a tool somewhat like BehaviorSpace that would produce this kind of test output. It would let users:
- Create local variables that identify specific turtles and/or patches, for example:
- let a-fish one-of turtles
- let fishs-patch [patch-here] of a-fish
- Create other local variables that are used as input to the code being tested, and specify ranges of values of those local variables (or observer variables). The BehaviorSpace syntax for specifying ranges of values could be used. For example:
- let fish-lengths [5 5 25]
- let patch-foods [0 1 20]
- let temperatures [0 2 30]
- Provide a block of code that updates the submodel for each combination of input variable values:
- ask a-fish [set length fish-lengths]
- ask fishs-patch [set food patch-foods]
- set temperature temperatures ; temperature is an observer variable
- ask a-fish [update-food-intake] ; execute a procedure affecting growth
- Specify the outputs to be reported for each combination of input values, such as:
[length] of a-fish
[food] of fishs-patch
temperature
[growth] of a-fish ; A turtle reporter that calculates growth
Then of course the user could execute the tester so it creates an output file with all the results. The output file could also produce metadata documenting exactly what code was executed how.