FREIDA: A Framework for developing quantitative agent based models based on qualitative expert knowledge

Abstract

Agent Based Models (ABMs) often deal with systems where there is a lack of quantitative data or where quantitative data alone may be insufficient to fully capture the complexities of real-world systems. Expert knowledge and qualitative insights, such as those obtained through interviews, ethnographic research, historical accounts, or participatory workshops, are critical in constructing realistic behavioral rules, interactions, and decision-making processes within these models. However, there is a lack of systematic approaches that are able to incorporate both qualitative and quantitative data across the entire modeling cycle. To address this, we propose FREIDA (FRamework for Expert-Informed Data-driven Agent-based models), a systematic mixed-methods framework to develop, train, and validate ABMs, particularly in data-sparse contexts. Our main technical innovation is to extract what we call Expected System Behaviors (ESBs) from qualitative data, which are testable statements that can be evaluated on model simulations. Divided into Calibration Statements (CS) for model calibration and Validation Statements (VS) for model validation, they provide a quantitative scoring mechanism on the same footing as quantitative data. In this way, qualitative insights can inform not only model specification but also its parameterization and assessment of fitness for purpose, which is a long standing challenge. We illustrate the application of FREIDA through a case study of criminal cocaine networks in the Netherlands.