Tutorial: Research Methods in Computer Science

The last decade I supervised a significant amount of PhD students, served in numerous PhD committees and reviewed a countless number of papers. During these occasions I always did my very best to give constructive remarks, and lots of young (someteimes even not so young) researchers have acknowledged that it did indeed help them becoming a better researcher. However, over the years I got this nagging feeling that I was often repeating myself; sometimes wondering whether I was sounding like an old grumpy grandfather instead of an active enthusiastic researcher. Worse, I was often giving tidbits of advice but noticed that the overall picture ---the reference framework so to speak--- was lacking. My advice was sometimes misunderstood, neglected or otherwise ignored and I often wondered if I could nail down this reference framework.

During my sabbatical leave at the University of Zürich (August 2009 - January 2010) in the research group SEAL I was finally presented with the opportunity to develop this reference framework. I did an awfull lot of reading, had several brainstorms and in the end created a tutorial entitled "Research methods in Computer Science". This web-page collects all material and related resources regarding that tutorial.

This page is last updated on Monday, September 30th, 2024.

Tutorial Notes | Goals and Objectives | Chronological Overview | Responses to Questionnaires

Tutorial Notes

Below you can find the most recent copy of the slides used during the tutorial. The long version as presented in Ulster University 2022; the short version as presented on GRASCOMP in 2024

Introduction Text, motivating the contents of the tutorial with some useful bibliographic references.
[PDF]
How to perform research ? (and get "empirical" results).
PDF [1 slide per page]
How to write research ? (and get papers accepted)
PDF [1 slide per page]
Redux version (1 hour)
PDF [1 slide per page]

Goals and Objectives

This tutorial explores the role of research methods in computer science, drawing upon practical examples from empirical approaches in software engineering.

Objectives: After this tutorial, a participant will be able to ...

Name and explain different approaches to conduct computer science research (i.e. feasibility study, case study, comparative study, literature survey, ...).
Understand the peer reviewing process inherent in academic research, including the implications it has for their own research (i.e. writing papers).

Target audience: This tutorial is aimed at PhD students and post-docs who want to improve their research skills. In particular those who are concerned with

(a) fine-tuning research questions;
(b) deciding on the best way to address them and
(c) writing it all up in high-quality research papers.

Chronological Overview

The tutorial "Research Methods in Computer Science" is a concrete result of the sabbatical leave at the University of Züurich in the research group SEAL. Below is a short chronological overview of how the tutorial evolved into its current incarnation.

PhD symposium at University of Zürich. Prof. Serge Demeyer organised a Phd symposium using three 2-hour lectures; one on research methods, one on the reviewing process, and one simulating an actual program committee. The symposium was attended by 21 PhD students at various stages in their research career. Participant feedback at the end of the lectures was positive, so the material was a perfect basis to expand upon.
[More info at http://seal.ifi.uzh.ch/phdseminarresearchmethods/.]
Condensed version at University of Berne. The two introductory lectures (research methods + reviewing process) got condensed into a smaller 1 hour lecture and was taught at an SCG seminar at the University of Berne. Condensing the material helped to sharpen the message which in turn lead to improving the material. This condensed version also served on a few other occasions.
[More info at http://scg.unibe.ch/wiki/softwarecompositionseminar/pastseminars.]
Keynote Address at CSMR2010. In march 2010, Prof. Demeyer gave a keynote address at CSMR2010 entitled Research in Software Evolution -- 'in vitro' vs. 'in vivo'. Here the condensed version of the lecture was mixed up with some concrete examples from over a decade of research experience. The keynote illustrated the richness of the material and confirmed that it should continue.
[More info at http://www.sait.escet.urjc.es/csmr2010/keynotes.html.]
ASE 2010 Tutorial All the above was reworked into a half day tutorial (3 to 3 and 1/2 hour) which was proposed (and accepted) at the ASE 2010 conference, organised in Antwerp 2010. Participant feedback at the end of the tutorial was again positive, yet some suggestions for improvement got incorporated into the ICSM2011 edition.
[More info at http://soft.vub.ac.be/ase2010/program/tutorials.]
ICSM 2011 Tutorial With the ASE 2010 edition, the material became stable. Some minor improvements were made and the tutorial was now presented at ICSM2011 in Williamsburg, Virginia. For this occasion, the slides were published over SlideShare http://www.slideshare.net/icsm2011.
[More info at http://www.cs.wm.edu/icsm2011/.]
IFI 2012 Summer School (Zürich) The University of Zürich organised a summer school for their PhD students. Obviously it was a nice opportunity to present the material in its established form.
SENECA 2016 "Writing and Moving on" Seminar for PhD students. After a few year's of hiatus I got invited again to present the tutorial in Madrid, Spain. The occasion was the SENECAs "Writing and Moving on" Seminar for PhD students (June 2016, Madrid, Spain). A small update on the slides with an inclusion on the CORE ranking for conferences.
IDLab 2017 (Semantic Web team) Another opportunity arose when I was invited by the Semantic Web team IDLab (http://idlab.technology/). After a tour of the group we had a very interactive session with lots of discussion.
Ulster University 2022 After several years of suspension (the Corona Pandemic) I was contacted by Ian Cleland (https://pure.ulster.ac.uk/en/persons/ian-cleland) in the School of Computing -- Ulster University (http:https://pure.ulster.ac.uk/en/organisations/school-of-computing). We spread the tutorial over 2 half days. In between I visited the Ambient Assisted Living Lab, which was a blast.
GRASCOMP Seminar 2024 I was asked to give the opening keynote at The GRASCOMP Seminar (September, 9th 2024); a PhD day for all researchers in the walloon region. I gladly accepted, but had to reduce the slide deck to cover one hour.

Responses to Questionnaires

After some incarnations of the tutorial I hold a little questionnaire among the participants. Below is an analysis of the responses.

Zurich 2010 Seminar [PDF]
ASE 2010 Tutorial [PDF]
ICSM 2011 Tutorial [PDF]
IDLab 2017 Tutorial [PDF]