Lesson 12 Reporting a model - ODD protocol
In their early days, agent-based models received a lot of interest, because it was a fundamentally new approach with which individual behaviour and interactions could be studied on a system level. However, also considerable critique was formulated by the research community. A big share of this critique was addressed to the large part of subjectivity in the models that would limit scientific credibility. Descriptions of models varied strongly in their level of detail, lengthy verbal explanations with ad hoc structures made it difficult for readers to understand a model – and merely impossible to replicate it. In short, model communication was inefficient for both, the writers and the readers.
The reaction of the agent-based research community to this fair criticism was impressive: not less than 28 researchers came together to publish a joint paper with an agreed standard for the reporting of agent-based models (Grimm et al. 2006). The aim was to provide a framework of presenting a model in a way that ensures that any other modeller could rebuild a published agent-based model and replicate the results. This paper turned out to be influential and the suggested ODD protocol (Overview – Design – Details) was successfully adopted by modellers in the natural- as well as the social sciences.
| Overview | Design concepts | Details |
|---|---|---|
| purpose, entities, state variables and scales, process overview and scheduling | Concepts of complex system design | initialisation, input, submodels |
The elements of the ODD protocol are organised from the ‚big picture‘ in the beginning to the details of the model in the end. It certainly helps a reader to quickly find a way through the model description and to extract the information at the level of detail he is interested. Even more importantly it helps the modeller to structure and specify his / her ideas. So, if you start with the development of a model (or the adaptation of an existing one) it is a good idea to write the ODD protocol right at the beginning! The first time it may be challenging, but with some experience it greatly helps in organising the model.
In this lesson we will talk through the elements of the updated version of the ODD protocol (Grimm et al. 2010). Not least this will also be a structured summary of agent-based modelling. The description of the elements are taken from this publication. For your reference and use, you can download the original ODD template document with detailed descriptions of all report items.
12.1 Overview
12.1.1 Purpose
ODD starts with a concise summary of the overall aims and objectives for which the model was developed, e.g. the problem or research question. Do not describe anything about how the model works here, only what it is to be used for.
12.1.2 Entities, state variables and scales
Agents
are the active entities of agent-based models. An agent may interact with other agents or be affected by external environmental factors. Agents are usually represented as points, but can also be modelled as e.g. grid cells.
The distinct, individual properties of each agent is characterized by its state variables, e.g. sex, age, spatial coordinates, or behavioural strategies. If state variables have units, they should be provided. State variables can change in the course of time (e.g. weight) or remain constant (e.g. sex). State variables should be elementary in the sense that they cannot be calculated from other state variables.
Spatial regions Most often rectangular grid cells, but also e.g. hexagons or irregular polygon patches of a coropleth map. Example state variables include the following: location, a list of agents in the cell, and descriptors of environmental conditions (elevation, vegetation cover, soil type, etc.) represented by the spatial unit. In some ABMs, spatial regions are used to represent agents: the state and behaviour of trees, businesses, etc., that can be modelled as characteristics of a cell. Some overlap of roles can occur. For example, a spatial region may be an entity with its own variables (e.g., soil moisture content, soil nutrient concentration, etc., for a terrestrial cell), but may also function as a location, and hence an attribute, of an organism.
Environment The environment provides the context in which agents act. Its conditions may vary over space. Examples of environmental variables are temperature, rainfall, market price and demand, fishing pressure, and tax regulations.
State variables
All output variables that describe the system and its entities. They define the type of data that results from a simulation.
Scale and extent In describing the spatial and temporal context, it is important to specify the model’s scale, its extent and the resolution. For example: One time step represents one year and simulations were run for 100 years. One grid cell represents 100m and the model landscape comprised 200 x 200 cells, i.e. 400 km2.
12.1.3 Process overview and scheduling
The process overview should answer the following questions:
- Who (i.e., what entity) does what, and in what order?
- When are state variables updated (synchronous or asynchronous update)?
- How is time modelled, as discrete steps or as a continuum over which both continuous processes and discrete events can occur?
Sometimes, pseudo-code helps to describe the schedule of processes in a way that enables re-implementation of the model code. Use self-explanatory process names like ‘update habitat’, ‘move’, or ‘grow’. Here is an exemplary pseudo code of a run-off model:
setup
construct DEM (artificial slope)
colour DEM
check out border patches
simulate-one-year
rainfall
flow
report-patternsThe process name (e.g. “rainfall”) suffices here, do not describe individual processes in more detail. This is done in the third section under ‘submodels’.
12.2 Design concepts
12.2.1 Basic principles
Which general concepts, theories, hypotheses, or modelling approaches are underlying the model’s design? Explain the relationship between these basic principles, the complexity expanded in this model, and the purpose of the study. How were they taken into account? Are they used at the level of submodels (e.g., decisions on land use, or foraging theory), or is their scope the system level (e.g., intermediate disturbance hypotheses)? Will the model provide insights about the basic principles themselves, i.e. their scope, their usefulness in real-world scenarios, validation, or modification use new, or previously developed, theory for agent traits from which system dynamics emerge (e.g., ‘individual-based theory’ as described by Grimm and Railsback (Grimm and Railsback 2013))?
12.2.2 Emergence
What key results or outputs of the model are modelled as emerging from the adaptive traits, or behaviours, of individuals? In other words, what model results are expected to vary in complex and perhaps unpredictable ways when particular characteristics of individuals or their environment change? Are there other results that are more tightly imposed by model rules and hence less dependent on what individuals do, and hence ‘built in’ rather than emergent results?
12.2.3 Adaptation
How do the agents adapt to improve their fitness (directly and indirectly)? What adaptive traits do the individuals have? What rules do they have for making decisions or changing behaviour in response to changes in themselves or their environment? Do these traits explicitly seek to increase some measure of individual success regarding its objectives , e.g., “move to the cell providing fastest growth rate”? Or do they instead simply cause individuals to reproduce observed behaviours (e.g., “go uphill 70% of the time”) that are implicitly assumed to indirectly convey success or fitness?
12.2.4 Objectives
If applicable: what are the objectives of an agent, e.g. economic success or reproductive success. How is the success of meeting this objective measured?
12.2.5 Learning
Many individuals or agents (but also organizations and institutions) change their adaptive traits over time as a consequence of their experience? If so, how?
12.2.6 Prediction
Prediction is fundamental to successful decision-making. If an agent can memorise things and learn from the success of its past behaviour, how do agents predict the future conditions (either environmental or internal) they will experience?
12.2.7 Sensing
What internal and environmental state variables are individuals assumed to sense and consider in their decisions? Is the sensing process itself explicitly modelled?
12.2.8 Interaction
What kinds of interactions among agents are assumed? Are there direct interactions in which individuals encounter and affect others, or are interactions indirect, e.g., via competition for a mediating resource? If the interactions involve communication, how are such communications represented?
12.2.9 Stochasticity
What processes are modelled by assuming they are random or partly random? Justify any stochasticity in the model
12.2.10 Collectives
Groups of agents can have their own behaviours, e.g. animal populations. A collective is usually characterized by the list of its agents, and by specific actions that are only performed by the collective, not by their constitutive entities.
###Observation Which data on state variables at individual and at system level are collected from the model for analysis?
12.3 Details
12.3.1 Initialisation
What is the initial state of the model at time t = 0 of a simulation run? How many entities of what type are there initially, and what are the exact values of their state variables (or how were they set stochastically)? Is initialization always the same, or is it allowed to vary among simulations? Are the initial values chosen arbitrarily or based on data? References to those data should be provided.
12.3.2 Input
Does the model use input from external sources such as data files or other models to represent processes that change over time?
12.3.3 Submodels
The level of detail in the description of submodels depends on the type of publication outlet. In scientific publications (which includes journal paper as well as Master Theses) it suffices to describe the skeleton and principle mechanisms of a process, including specific equations or algorithms. For the sake of length this should be presented in a tabular form or may have to go to an appendix. In a technical report the processes are explained in full detail.