Category: 1.2 Research Methods

The key output of research is a theory or something supporting the building or validation of a theory. A theory, in turn, is knowlege, for example in the form of a model, that lets us predict the future or create reliable output in some form. Scientists usually publish theories for other scientists to review, in journals.

But what about the practitioners that we are creating the theories for in the first place?

In this video I present a way of codifying (presenting) theories as practical pattern handbooks so that practitioners can use these handbooks and apply your theory. This way, we connect science to practice and hopefully help make the world a better place. It also helps to get industry engaged in your work.

An associated (preprint) technical report, soon a journal paper, is available with more information. The slides for the video above are also available.

Many computer science degree programs do a lousy job at teaching science. A high school student, entering university, often has a good idea what science is about, based on their physics and chemistry classes. At least, it involves controlled experiments. At university, this is rarely picked up, and computer science students are given the idea that programming something novel constitutes science. With that idea, they are often bewildered when I teach them rigorous research methods, in particular if those originated in the social sciences (like qualitative interviews or hypothesis-testing surveys).

Continue reading “Traditional Theory Building and Validation in (Computer) Science”

On a whim, I asked my Twitterverse (which includes a fair number of computer scientists) what they think about the following question:

When peer-reviewing somebody else’s work submitted for publication, what should you do if you find that the authors have a different belief than you about what can be known?

Continue reading “Anecdotal Evidence on the Method Wars”

From my excursion into qualitative research land (the aforementioned Berliner Methodentreffen) I took away some rather confusing impressions about the variety of what people consider science. I’m well aware of different philosophies of science (from positivism to radical constructivism) and their impact on research methodology (from controlled experiments to action research, ethnographies, etc.) I did not expect, however, for people to be so divided about fundamental assumptions about what constitutes good science.

One of the initial surprises for me was to learn that it is acceptable for a dissertation to apply only one method and for that method to only deliver descriptive results (and thereby not really make a contribution to theory). In computer science, it is difficult to publish solely theory development research (let alone purely descriptive results) without any theory validation attempt, even if only selective. The limits of what can be done in 3-5 Ph.D. student years are clear, but this shouldn’t lead anyone to lower expectations.

Continue reading “We May not Know What We are Doing…”

The American Statistical Associations’s answer to “statistical techniques for testing hypotheses…have more flaws than Facebook’s privacy policies.” is is now available for free.