|
Academic
Services |
Critically Reviewing the Literature |
|
|
AIM AND INTRODUCTION |
Before any research planning or data collection takes place it is of central importance to ensure that you have adequately reviewed the literature relevant to your research question. The main reasons why you need to do this can be summarised as :
Some of the key issues you need to evaluate about an article have usefully been summarised by the American Psychological Association (1983) as follows :
|
EVALUATION CRITERIA |
To some extent your ability to review articles is something that requires practice and experience. However, the process can be aided by following a checklist of things to look out for and comparing the paper under review to the criteria. In its simplest form Bradford Hill (1965) cited by Hawkins (1985) suggested the following key questions should be asked about each section of a paper :
Introduction .......................... Why did they start the research?
Methods ............................... What did they do?
Results .................................. What did they find?
Discussion ............................. What do the results mean?
A more detailed checklist of points to look out for is now given section by
section below.
NB Do bear in mind that the type of research (quantitative or qualitative) will
effect the information you are evaluating. Some of the issues mentioned below
may not be applicable to all research studies but the general principles will
apply.
|
Title A minor issue this one, but it
is helpful if the title is succinct but descriptive of the article content.
Gimmicky or catchy titles are all very well but do they enable you to tell what
the article is about? Of course a poor title does not necessarily indicate a
poor paper and certainly isn't a key criterion in assessing the quality of a
paper. |
|
Author Some idea of the author(s) academic background can be gained by looking at their job title, qualifications, and where they work. Articles normally provide this basic information and it may give you an indication of the ability of the writers to carry out valid research. If you know them to be acknowledged experts in the subject area covered then there is good chance the research work will be of high standard (though it would be dangerous to assume this without further critical review of the paper!)
|
|
Abstract This should provide a handy summary of the content with indications of
the aims, methods, results and importance of the study. Muir Gray (2001,
p.107-8) suggests
reading abstracts is an excellent way to "identify junk" but he
also makes the valid point that abstracts are often written with a bias
towards highlighting the positive aspects of the research. If the findings
are noted as positive then carefully check the methods used (negative
findings may perhaps give an indication of lack of bias but you still need
to be check). The abstract can, therefore, give an indication of how well
the study was conducted and whether it is worth reading. |
|
Source An assessment of where the article was published should give some clues
as to its potential value. The key issue is whether it is a peer reviewed
journal. In other words do articles submitted to the journal go through a
rigorous review process before they are accepted for publication. Some
journals are undoubtedly less fastidious about having articles assessed
before publication. |
|
Introduction This section is where the research problem/ clinical question should be defined clearly. You should expect to find here clear descriptions of the research aims, an outline of theoretical issues and the hypothesis should be introduced. Information should include the current state of knowledge about the research topic and an indication of the gaps in knowledge which the current study will hope to fill. Overall you should get an answer to the question "why was the research done?"
|
|
Literature Review Here you will find a survey of current knowledge highlighted by a
thorough review of the existing literature. The review should indicate any
theoretical implications of the research in relation to previous work. The
review should include up to date references and be based on as wide and
thorough a search of sources as possible. A key issue is whether
researchers have been unbiased and have presented any evidence which
actually contradicts their own ideas. |
|
Methodology This section should provide a clear and concise account of methods used. Such detail should be sufficient to allow the research to be replicated by other researchers. The study design and data collection methods should be clearly outlined. Overall you should be able to assess how the research was done. The methods sections may be divided as follows: a) subjects The participants, the test conditions and procedures for experimental
and control groups should be described in detail. This should enable you
to assess whether the sample selection method was valid. You would expect
information on the number of subjects and who took part. A small sample
may be fine as long as it is representative and numbers will generally be
smaller for qualitative studies. However, it may be that a large sample
size is needed to provide a representative group size and small numbers
may have an impact on the power of statistical analysis (Polgar &
Thomas, 2000, p.278). The type of research will obviously impact greatly
on the information you should be looking for. For example, in
the case of a study using a randomised controlled trial, there should be information on
the method for randomising the allocation of subjects to experimental and
control groups. b) Apparatus or Instruments Any special equipment or instruments (e.g. questionnaires, standardised assessments) should be described. The validity and reliability of apparatus or instruments needs to be indicated. In other words the adequacy and appropriateness of the methods used for collecting data should be made clear. c) Procedures What happened in the experiment and the
steps executed should be described here. The treatments and settings
should be outlined and a reputable researcher should detail any flaws in
the procedures or any other factors adversely effecting the research
work. In simple terms, what was done and how it was done should be
recorded here. |
|
Results
The results section reports what has been discovered as a result of the
research undertaken. The normal format is for the results of the research to be
reported factually and formally without discussion and then for a prose summary
to be given of the statistics etc. Other results including summary statistics
may usefully be presented in tables or figures which, if they are well
constructed, should aid understanding of the findings. Any statistical tests and
measures used should be described allowing the reader to evaluate whether the
appropriate tests were applied. A good researcher should mention all the
relevant results, even those that actually go against the hypothesis. |
|
Discussion
The issues raised by the findings should be discussed and resolved in this
section. A good discussion section will relate the findings back to the
literature and to the aims of the research as outlined in the introduction. The
author is expected to examine, interpret and qualify the results and draw any
inferences from them. It should be possible to assess the contribution made by
the study and decide how far it has helped to resolve the original problem.
|
|
Conclusion and recommendations The paper should end with some conclusions about the importance (or otherwise) of the findings. The author should not make any statements here which are not supported by the facts found. Some speculation is acceptable but only if it is described as such and does relate logically to the data or theoretical basis of the study. Recommendations on the basis of the findings are often stated here and may include comments on possible improvements to the research or future areas for more study.
|
|
References and bibliography Consistent citing of references is a sign of good practice here. The list should be appropriately extensive and up-to-date. The separation of citations into References (ie papers actually referred to in the text) and Bibliography (other material read to support the research but not directly cited or quoted) is normal practice.
|
|
|
FURTHER POINTS |
Apart from the above criteria, one of the key criticisms often levelled at research papers is their use of stilted language and impenetrable jargon. A certain amount of academic "posturing" is not unknown and the results (even if important) may be submerged under a welter of statistics only comprehensible to a professor of mathematics! Having said that, most articles will, not unreasonably, assume that the reader should have some background knowledge of research terminology and methods, including statistics.
A good paper should be well-written and readable, with a sound structure, presentation and a logical sequence. Overall it should have an appropriate intellectual level and provide evidence of at least some originality.
Further useful information on this topic can be found via the following links:
How to read a medical journal article
Critical appraisal and using the literature
|
|
CHECKLISTS |
Aside from the general criteria outlined above it is worth mentioning that there a number of checklists available which are aimed at assessing particular types of research paper e.g. qualitative studies. Many can be found in textbooks such as Greenhalgh (2000) but many others are also available on the Internet. A few examples are:
| Case Control | Case Control checklist (CASP) |
|
Causation or harm (aetiology) |
|
| Clinical Guidelines | |
| Cohort Studies | Cohort Checklist (CASP) |
|
Diagnosis/screening |
|
|
Economic evaluations |
|
|
Prognosis |
|
|
Qualitative Research |
|
|
Randomised Controlled Trials |
|
|
Systematic Reviews |
|
| Therapy |
|
|
READINGS |
American Psychological Association (1983). Publication Manual. 3rd ed. Hyattsville, APA.
Bell J (1999) Doing your research project. 3rd ed. Buckingham, Open University Press. (Chapter 6: literature review)
Bower D, House A and Owens D (2001) Understanding clinical papers. Chichester, Wiley.
Bury T and Jerosch-Herold C (1998) Reading and critical appraisal of the literature. IN Bury T and Mead J (1998) Evidence-based healthcare: a practical guide for therapists. London, Butterworth-Heinemann. (Chapter 7)
Craig J V & Smith R L (2002) (eds) The evidence-based practice manual for nurses. London: Churchill Livingstone (Chapter 4: Critical appraisal 1, pp.86-113)
Crocker L M (1977). Linking Research to Practice : Suggestions for Reading a Research Article. American Journal of Occupational Therapy, 31(4). 34-39.
Crombie I K (1996). The pocket guide to critical appraisal. London, BMJ.
Drummond A (1996). Research methods for therapists. London, Chapman Hall
French S (1993). Practical Research : A Guide for Therapists. London, Butterworth / Heinemann.
Greenhalgh T (2000). How to read a paper: The basics of evidence based medicine. 2nd ed. London, BMJ. (The original articles used in this book can be found at : Greenhalgh (BMJ)
Hart C (1998) Doing a literature review. London, Sage.
Muir Gray JA (2001) Evidence-based healthcare. 2nd ed. Edinburgh, Churchill Livingstone.
Parry A (1987) Guidelines to Appraising Research Papers in Journals. Physiotherapy 73 (7) : 375-378.
Polgar S and Thomas SA (2000) Introduction to research in the health sciences. 4th ed. Edinburgh, Churchill Livingstone. (Chapter 22: critical evaluation of published research)
|
Copyright Statement : © University of Plymouth, School of health Professions, Millbrook House, Exeter |
|
Written by Graeme Barber |