Empirical Software Engineering as a Science: Challenges and Ways Forward


The Empirical Software Engineering (ESE) community has made great progress in the last 20 years and expanded the field considerably both in scope, volume as well as quality. Nowadays, we have established conferences as well as journals focused on the area, and a majority of the papers published in the top SE conferences such as ICSE are empirical. However, while more established scientific fields such as Physics, Biology and Psychology have clear identities, specific schools of thought, and explicated research methods, I argue this is less so in ESE.

In this talk, I will outline commonly accepted criteria for what constitutes science and then assess to what extent ESE, at present, fulfils these criteria. This way, we can clarify our strengths but also identify weaknesses we can address in the future. In particular, I will consider if we will have a replication crisis in ESE and how we are doing on taking multiple and ‘soft’ aspects into account, e.g. behavioural SE. I will discuss strategies for how we can improve the reliability and validity of our ESE research and conclude with some hopes for how we can be an even stronger science going forward.


Robert Feldt is a professor of Software Engineering at Chalmers University of Technology, Sweden, and at Blekinge Institute of Technology, Sweden. He has broad research interests spanning from human factors to hardcore automation and statistics, and work on testing and quality, requirements engineering, as well as human-centred (behavioural) software engineering. Dr Feldt was an early contributor to search-based software engineering and have recently argued for increased application of psychology and social science to understand and improve software engineering. Most of his research is empirical and conducted in close collaboration with industry partners in Sweden, Europe and Asia, but he also leads more basic research. Dr Feldt received a PhD in Computer Engineering from the Chalmers University of Technology in 2002, studied Psychology at Gothenburg University in the ’90s and has also worked as an IT and software consultant for more than 25 years. He is passionate about empirical research and methods and changing organisations through technological innovation. He is co-Editor in Chief of the EMSE journal and on the editorial boards of three other journals (STVR, ASOC, and SQJ).

Robert Feldt

The Digital Apocalypse according to Silvio Meira, Book 3, Chapter 2019…


It’s hard to list the biggest innovations of the last few decades without citing the Internet. In a list of 2009 that has aged very well, the Internet comes first [and more than 20 of the 30 innovations are digital]. It is like dividing history into three stages: analogical, from the beginning of time until the middle of the twentieth century; digital, the times of the first digital computers, after the second world war [or after the Turing-Church Thesis, and 1936 would be the date] and then in networked, starting with the commercial Internet, in 1995. After smartphones [from 2007] and [near] universalization of personal connectivity, we started to live in a digital and networked world. In some countries, [almost] everything has become digital and connected in a few decades, with the[ir] network completely mediating many services and habits nowadays.

Such pervasiveness of digital performance has created an immense challenge to businesses outside intrinsically digital markets such as social networks: nearly every incumbent now has digital capabilities far below those that would be perceived by their current clients and users as [even] good enough -especially when compared with commonly used platforms like those provided by Google, Microsoft, Facebook, Alibaba and Tencent, just to cite a few. As a result, all types of analog -and even digitized analog- corporations now face an uphill marathon to transform their business models and corresponding operations into a digital first mode, having to simultaneously protect and evolve assets that were created over decades and develop leaner, faster, digital ways of doing business.

It’s going to be -it already is- an incredibly difficult and bumpy ride, even more so because a number of incumbents have already discovered the what and how of being digital. For the rest, it is already looking like a digital apocalypse… and that’s what we are going to talk about.


Silvio Meira is EXTRAORDINARY PROFESSOR at CESAR School, PROFESSOR EMERITUS at the Center for Informatics, UFPE and Senior Research Fellow at the SENAI Institute for Innovation in Informatics, all in Recife, Brazil. Founder and Chaiman of the Board at PORTO DIGITAL, Meira is also a member of the Boards of MAGAZINE LUIZA, CI&T, MRV Engineering and CAPES. Silvio was part of the team that founded CESAR in 1996, was Fellow and Faculty Associate of Harvard University’s Berkman Center [2012 ~ 2015] and Visiting Professor at FGV-RIO’s Law School from 2014 to 2017.

Silvio Meira holds a BSEE [ITA ,1977], MSc in Computer Science [UFPE, 1981] and PhD in Computing [University of Kent at Canterbury, 1985], has supervised 150+ PhD and MSc theses, published 300+ articles in learned journals and conferences and has written extensively in the lay media.

Silvio Meira