The final report discusses key conference findings regarding the usage of mathematical, statistical, and computational STI models. Major challenges, insights, and opportunities can be summarized as follows:
Challenges exist in five areas: (1) Fundamental Research: Few models are validated using empirical data. STI success criteria are not clearly defined, e.g., when is science successful? Few agencies support STI modelling efforts. Federating and cleaning existing open or proprietary data requires resources. (2) Applied Research: Most scholars design models to publish; few build policy-relevant models. Most policy makers do not engage in modeling efforts yet their expert input is needed to design models that make a difference. Few models are production-strength, i.e., validated, well documented, 24/7 service. Few active partnerships among academia, government, and industry exist. (3) Cyberinfrastructure: Unconnected silos of data and code repositories exist in different areas of science. Scholarly results are published in many different journals and are hard to find. Few special issues, reviews, general textbooks exist. ModSTI scholars and practitioners attend many different conferences, might not know about each other. (4) Education: There is a need for improved “model literacy” via formal and informal education combined with more effective communication by researchers on the power of models. A community of teachers/students that share data, code, results, training materials, etc. is missing. (5) Outreach: Engagement with the public is very important. Scientists need to be more actively involved in communicating with the public and engaging with other communities.
Insights and opportunities can be grouped into five areas: (1) Model Needs and Implementation: Successful modeling teams require close collaboration and active partnership between (policy) decision makers and researchers to ensure the usefulness of the models and increase the chances for their adoption. Predictive models require testing, iterative improvements, and a community of users. Models can be used in “evaluation” exercises but are even more valuable when used in support of “situational awareness, proactive steering”. There is a need to identify and call out bad models, since their existence and wide dissemination can harm the reputation of modeling efforts in general. (2) Data Infrastructure: Data quality, coverage, and richness are improving rapidly and support the design and validation of detailed models. Many teams are spending 80% of overall STI modeling effort on data preparation. The setup of data repositories should be explored to increase the amount of time available for model development. Easy access to relevant data will support reproducibility. As many high-quality datasets are held by industry or government, close academia-industry-government partnerships seem desirable. (3) Code Repository and Standards: Well-documented tools are needed that allow decision makers to run their own models. Models need to be reliable and results have to be reproducible. STI modeling community should aim to adopt modeling guidelines and standards and aim to create a shared data and model code infrastructure. (4) Visualization and Communication of Modeling Results: It is important to communicate data quality, model complexity, and modeling results clearly to different stakeholders. Telling visual stories, augmented with high quality data is powerful. More advanced visualizations of model results can be used to have decision makers “fly the future” before writing a check. (5) Funding: Modelling needs increase with reduction of budgets, significant increase in the number of researchers, exponential growth of scientific productivity, larger team sizes, and higher interdisciplinarity. While few agencies and organizations have active funding programs on STI models, modeling of STI programs might be supported analogous to STI program evaluation.
Please cite as:
- • Katy Börner and Staša Milojević (Eds.) (2016) Modeling Science, Technology and Innovation. NSF Conference Report, Indiana University.
Low resolution version is here (1.1 MB).