This is a full day tutorial I gave at the 15th IEEE International Conference on Requirements Engineering (RE'07) , in New Delhi, India, October 2007.
The slides for the talk.
Book chapter: Easterbrook, S. M., Singer, J., Storey, M, and Damian, D. Selecting Empirical Methods for Software Engineering Research. To appear in F. Shull and J. Singer (eds) "Guide to Advanced Empirical Software Engineering", Springer, 2007.
This aim of this tutorial is to introduce the range of empirical methods used in RE research, to explain how to choose appropriate methods for particular research projects, and how to ensure the methods are used with a high level of scientific rigour. Using a blend of lecture, discussion, and practical exercises, the tutorial will provide RE researchers and practitioners with a foundation for planning, conducting and reviewing and empirical studies. The tutorial will cover all of the principal methods applicable to RE: controlled experiments, quasi-experiments, case studies, survey research, ethnographies, and action research and will relate these methods to relevant meta-theories in the philosophy and sociology of science. Special emphasis will be given to the challenges of investigating the social and cognitive processes that occur when people analyze human activities in order to understand and manage complex, evolving requirements. The tutorial will cover techniques applicable to each of the steps of a research project, including formulating research questions, theory building, data analysis (using both qualitative and quantitative methods), building evidence, assessing validity, and publishing. The tutorial is relevant for researchers, who will be able to conduct and write more credible empirical studies; for reviewers who will be able to provide qualified judgments of papers; and for practitioners, who will be able to more effectively interpret published results within the context of their own organizations.
Despite the widespread interest in empirical research in software engineering over the past decade or so, software engineering still lacks a coherent methodological foundation. There is little guidance available on which research methods are suitable, and how to choose between them. Many researchers select inappropriate methods due to a lack of understanding of the goals underlying the method employed, or due to a lack of knowledge about alternative methods. The problem is particularly acute in requirements engineering, where research crosses social and technological boundaries. Many important research questions in RE concern the social and cognitive processes that occur when people analyze human activities in order to understand and manage complex, evolving requirements.
The aim of this tutorial is to cover the key questions to consider in selecting empirical methods, from the philosophical issues about the nature of scientific knowledge, to the practical considerations for applying empirical methods. We identify and characterize the key empirical methods applicable to RE research, and explain the strengths and weaknesses of each. Our goal is to show how methods can be selected and combined to produce a valid research strategy in response to a well-defined research question.
The tutorial starts with basic issues in empirical studies, moves on a detailed look at each of the principal methods, and concludes with an examination of publication. The curriculum includes the following topics.
The intended audience for this tutorial includes students, researchers, and practicing requirements engineers, that is, both the producers and consumers of empirical studies. This tutorial will help requirements engineers identify and avoid common mistakes by giving them a solid grounding in the fundamentals of empirical research methods. The tutorial will be structured in such a way as to emphasize the principles and design of empirical studies.
An extensive list of further readings can be found on the website for my graduate course, Empirical Research Methods in Software Engineering.