Is this research any good?

Robert Coe, one of the editors of the trusted educational research companion Research Methods and Methodologies in Education outlines some of the questions you should consider when determining what makes a piece of research good or bad. This is a helpful way for researchers to self-assess their own work as well as other research work. 

Perhaps even harder than defining educational research is defining good research. Given the breadth of approaches to doing educational research, it seems unlikely that there will be any universal set of quality criteria. We present here a list of questions that it may be appropriate to ask in evaluating the quality of a piece of research.

• What are the research questions/aims? 
• Are they clearly stated?
• Are they relevant/important? 
• Does the research actually address them?
• Is the methodology appropriate to them?
• Could the research add to existing knowledge?
• Does the research build systematically on what is already known?
• Are any assumptions or beliefs of the researcher(s) made clear?
• Is it clear who funded or supported the research and whether there are any potential conflicts of interest?
• Are any definitions of terms or constructs clear?
• Are these definitions appropriate (not too broad/narrow)?
• Is it clear how phenomena have been represented?
• Are any constructs operationalised appropriately?
• Is any interpretation of constructs (e.g. measures, scores, variables) supported by a convincing validity argument?
• How realistic or representative are the contexts in which the research was done? Are they described adequately?
• Are any samples adequate? In what sense are they representative?
• Is there enough information about the participants? Who were they? Are we told what the study meant to them?
• How were participants chosen? Who is included/excluded? Is any non-response disclosed?
• Are the claims clear and explicit?
• Are there implicit causal claims?
• Does the evidence support the claims?
• How far are the claims generalised? Is any generalisation justified?
• Are alternative explanations offered/challenged?
• What is arbitrary? How might things have been done otherwise? Are the choices made by the researchers transparent?
• Has there been any selection in what is reported?
• If the data might have been interpreted or analysed differently, could this have led to different conclusions?

Learn mor about conducting educational research