So ICSE, the top software engineering conference, rejected our paper, again. The reviewers were actually quite positive: high-quality work, little or no flaws, interesting. One of the reviewers found the paper’s results surprising, asked for more details, and suggested new research directions. The final conclusion of both reviews, however, was the same: The work has no merit because it only explains the world, it does not improve it.
Our paper provides a high-quality model of a key aspect of programming behavior in open source, basically the modeling behind this earlier empirical paper. As such, it is a descriptive empirical paper. It takes a large amount of data and provides an analytically closed model of the data so that we can explain or predict the future (better). That’s pretty standard operating procedure in most of natural and social sciences.
Suggest to a physicist that a paper that explains how a particle spins can only be published if in the same paper the physicist shows how to make the particle spin backwards. Or contribute to human well-being. He or she will give you the finger. Suggest to a sociologist that it is not enough to understand medical practices in a hospital, that he or she in the same paper also has to provide empirical validation on how to improve these practices. You’ll be out the door before you can say I’m sorry.
Only in engineering research do we seem to see no value in seeking knowledge and truth for its own sake. Only in engineering research is merit defined by how some work improves the world. Maybe that’s why many don’t consider it a science. It seems obvious to me that before improvement comes understanding, but in repeated experiences with ICSE and elsewhere, reviewers consistently require that both of it has to be packaged into the same 10 page paper. I think this is ludicrous.
I can accept an argument that some particular knowledge is of little merit because it is uninteresting. I cannot accept the requirement that deep and interesting insight only gains publishable merit by being combined with its application in some context.
Happy new year and a successful 2012 to everyone!
PS: Some explanation for those not working in computer science / software engineering. ICSE is the top conference of the field, by most measures, and ranks higher than the journals. (Which is another weird thing about computer science, but not of concern here.) Computer scientists don’t like to read (a lot) and hate discursive papers (try publishing Grounded Theory work) so many publishing outlets have stringent page requirements, typically 10 pages or 10000 words.