 |
Introduction to Research Methods in Political Science: |
II. RESEARCH DESIGNS AND REPORTS
Revised version of essays by Charles McCall,
California State University, Bakersfield
Subtopics
Good research is not
accidental. It requires careful planning as well as careful
execution. Before launching a research project, social scientists prepare
a research design, in which they set forth their plans for the research they intend
to undertake. Often, such designs are formal in nature, as when a scholar
prepares a research grant request to a funding agency, or when a student
submits a proposal for a dissertation, thesis, or other research project.
Even when a research design is
not a formal requirement, it is a step in the research process that should not
be bypassed. With a good design the student will be able to foresee and
avoid numerous obstacles and pitfalls, save time in the long run, and produce a
superior finished product.
The length and complexity of
research designs obviously vary considerably, but any sound design, be it for a
massive, government or foundation supported study, or for an undergraduate term
paper, will do the following things:
1. Identify the problem clearly
and justify its selection.
2. Review previously published
literature dealing with the problem area.
3. Clearly and explicitly
specify hypotheses central to the problem selected.
4. Clearly describe the data
which will be necessary for an adequate test of the hypotheses and explain how
such data will be obtained.
5. Describe the methods of
analysis which will be applied to the data in determining whether or not the
hypotheses are false.
1. As you can see, step 1 requires you to state the problem and
justify its choice; that is, you should explain your reasons for selecting
it. It is correct to say that the choice of a topic depends principally
on an investigator's interest and values. Nonetheless, there are some
general criteria which we can use to evaluate the relative worth of
topics. For example, how important is the topic for development of
knowledge? How would the answer to questions which you pose for yourself
affect other parts of the knowledge structure within
political science? Would it extend theories to areas not yet
explored? Would it replicate earlier findings in a different
setting? Would it resolve present inconsistencies in our interpretation
of evidence or create inconsistencies by calling into doubt currently accepted
interpretations? A research project which adds to our knowledge in such
ways is bound to be considered important.
While the promise of acquisition
of knowledge is an important criterion in evaluating research proposals, it is
not the only standard we apply. If additional research on the problem
will improve the social, political, or economic well-being of people, that fact helps justify the choice of a research
topic. Would the answers to the questions that you pose for yourself
affect the achievement of accepted social values? Might such answers
cause changes in accepted social values? Positive answers to such
questions suggest that the research proposal is one which is important.
A third criterion which we use in
evaluating a choice of research problems is the criterion of economy. How
easily can the topic be studied? We are not likely to establish a moon
colony simply to discover whether or not the relationship between sense of
political efficacy and political participation which we've discovered
repeatedly in Western nations persists among those living on our permanent
satellite. No one is likely to support a proposal to expend great amounts
of time, skill, and money on a project likely to add little to our knowledge or
to human welfare.
2. Before beginning any research project, you should
familiarize yourself with the work that has already been done in your area, so
that your project can build on and extend (or, if necessary, correct) the
findings of those who have preceded you. A summary of this review of the
literature should be included in your research design. Such a review is helpful
not only in systematically gathering and organizing existing substantive
knowledge about the topic but also in sensitizing scholars to the methods and
procedures which have been used by others investigating the same general area.
One way to get started on a “lit review” of journal articles and other scholarly publications is to find a journal article on your topic; the list of “references” or “works cited” in this article will in turn lead you to other relevant material. Your campus library has holdings, hard copy and/or electronic, of many scholarly journals. In addition, the library may subscribe to JSTOR (http://www.jstor.org) (short for “journal storage”), which allows you to access full text of articles from a large and growing list of publications. Visit your library’s Website, or talk to a reference librarian to find out whether your institution is a JSTOR subscriber and, if so, how to access it at your campus. Even if your campus doesn’t subscribe to JSTOR, you can still obtain citations and article abstracts, but will then need to go to other sources for the full text of the article. If your campus library doesn't have what you are looking for, it may be able to obtain it for you through interlibrary loan (also known as "document delivery"). Caution: articles from many journals are subject to a 3 - 5 year embargo before becoming available through JSTOR.
Another good source for finding citations of scholarly articles is Google Scholar (http://scholar.google.com/; not the general Google search engine with which you are probably already familiar). Once you’ve located an article, you can also click on “Cited by” for a list of later publications referencing that article. While freely available, Google Scholar will generally provide only citations and abstracts of articles, not full text.
Yet another place to find citations of data-driven research is the Interuniversity Consortium for Political and Social Research (ICPSR) (http://www.icpsr.org). You can search the “Bibliography of Data-Related Literature” for publications using data in the ICPSR archives. Alternatively, you can go to “Find and Analyze Data,” locate a dataset that you think might be of interest, and click on “View related literature.” All of this information is freely available, but to actually obtain the data used by the author(s), you need in many cases to be at a campus that is an ICPSR member (see below).
3. The third major step in a research design is to clearly
state the hypotheses to be tested. As used here, a hypothesis is a
declarative statement positing a relationship between variables. In its
simplest form a hypothesis contains these elements:
an independent variable; a dependent variable; the nature of the relationship
between the two.
The development of hypotheses
belongs to that area of inquiry which has been called the logic of
discovery. The logic of discovery deals with the identification of good
reasons for suggesting a hypothesis — reasons
for putting it forward or testing it, not reasons for accepting it. One
good reason for testing a hypothesis is
that it is commonly believed. Gerald Pomper in
his book, Elections in America, noted that the literature of political
science was replete with claims that party platforms were unrelated to
legislative enactments. Pomper also noticed
that there was very little systematic evidence in the literature supporting the
hypothesis; thus he decided to test it. His research falsified the
hypothesis, which had been a standard one in political science, and therefore
has been of great importance to the discipline.
A second way in which hypotheses
are developed is through the use of analogy. Hypotheses are often based
on metaphors drawn from our environment. Darwin explained that it was his
perception of the analogy between Malthus' theory of human population and
processes in the animal kingdom which led him to develop the theory of
evolution. The analogy between chess and international politics is an old
one in political science, and we have discovered that a careful study of games
can be quite suggestive for the development of hypotheses regarding the ways in
which nations treat each other.
Another successful use of analogy
in the social sciences is evident in the application of formulae developed to
describe the spread of communicable diseases as formulae to describe the spread
of rumors in society. Not all analogies are good ones. The test of
a good analogy is that it suggests numerous and significant hypotheses and that
those hypotheses are not refuted by systematic testing.
A third way of developing
hypotheses is retroduction. That method which
Arthur Goldburg has defined "as an imaginative
leap, a flight of fancy, taken to account for observed phenomena" is
insufficiently understood. Many of you will know the tale regarding the
discovery of Archimedes' Law that a solid body will, when immersed in water,
suffer a loss in weight equal to the weight of the displaced water. As
the story goes, Archimedes had long been perplexed by a problem set for him by
the tyrant of Syracuse. He had commanded that a votive crown of pure gold
be prepared for placement in one of the temples, but gossip concerning the
goldsmith suggested that some silver had been mixed with the gold in the
manufacture of the crown. Archimedes was charged to discover without
injuring the crown whether or not that was the case.
The story continues that
Archimedes, while taking a bath noticed that his limbs were unusually light
when in the water and that in proportion as his body was immersed in the tub,
the water spilled over its edges. That perception led to instantaneous
realization that he could solve the problem. But if the myth has any
basis, the fact that Archimedes after long study was hit so quickly by the
solution to the problem that he was moved to leap out of the tub and run home
naked shouting "eureka!" does not mean that the solution was not a
result of his long study.
Regardless of the way in which
hypotheses have been discovered or developed, it is crucial that they be
stated clearly. If they are clearly stated, the distinction between
independent and dependent variables should be obvious. Naturally, careful
statement of hypotheses also entails clear, conceptual definition of the
major variables involved.
4. After you have stated your hypotheses clearly, you are ready
to proceed to the fourth major stage in the research design, the description of
the data required to check the hypotheses and the explanation of how the data are to be obtained. Your hypotheses determine in large part the kind of
data required and suggest methods for collecting or obtaining them.
Let us assume that you plan to
test the assertion that conservatism is related to social class such that the
higher peoples' social class, the more likely they are to be
conservative. Your first task is to operationalize the variables. How will you identify "conservatives" and
"liberals" in terms of operations which others could repeat?
How will you identify in similar terms a person's social class?
Suppose you decide to operationalize conservatism on the basis of a scale based
on a series of questions about political issues. You then need to specify
the questions to be included in the scale and describe the method by which
responses to those questions will be scored and summarized. Now what
about social class? Suppose you decide to operationalize social class on the basis of income. Suppose further that you decide to
group your data into the following categories: less than $40,000, $40,000 to
$99,999, and $100,000 and over.
Your next step is to explain how the necessary data are to be obtained. In general, this might involve either primary analysis of data which you will collect especially for this project, or secondary analysis of data that has already been collected for some other purpose. "Secondary" doesn't mean second rate. In fact, a number of sources provide data of the highest quality that are far beyond what you could hope to produce yourself (assuming that you lack unlimited time, money, and organizational resources).
Some of these sources are available by institutional subscription only. Check with your instructor or reference librarian to find out what's available at your campus. These include:
There are a number of other sources of data that are freely available. For a list of just a few of these, see http://www.ssric.org/data/other.
Wherever they come from, the data that are to be analyzed must be
described fully and accurately. The goal is to give readers sufficient
information to allow them to successfully evaluate your proposed project and
the analysis and conclusions which you will present later.
The main point to remember is
that in principle the data you plan to analyze must be capable of falsifying
the hypothesis if the hypothesis is in fact untrue. What will count
against it? That is probably the most important question in building
theory. Indeed, the principle which should govern the selection of
evidence to test the hypotheses is the principle that the chances of
discovering decisive negative evidence should be maximized. The best
design is one which would most clearly and quickly expose the error in a
working hypothesis. That means that we go out of our way to look for
negative evidence.
5. You are now ready to turn to the fifth major element in a
good research design, the identification of the methods of analysis appropriate
for treating your data. Be as specific as possible in describing the
approach you will use, but do not propose to employ a technique that you do not
really understand in the hope that this will somehow give your design an aura
of erudition.
A research report contains all the
elements of a research design, and in addition presents, interprets, and
evaluates the findings of the research. If you
have written a sufficiently careful research design (and have been rather
lucky), you may be able to incorporate your design in its entirety in your
research report. In practice, however, few research projects go
precisely as planned, and the original design will likely have undergone at
least some modification before the finished report is prepared.
In presenting your findings, the
necessary evidence should be set forth and the conclusions based upon it should
be clearly stated. Many beginning scholars fall victim to the temptation
of presenting too much evidence and analysis in the central portion of the
paper. Remember the problem you set out to attack. Interesting findings
which are tangential to the problem should be presented in footnotes or in an
appendix. If you include them in the main body of the text, you may
only succeed in directing the reader's attention away from your principal
focus.
It is in this central portion of
the paper that you will present most of your tables and graphs.
Throughout the report, your main objective must be to communicate clearly, and
this applies just as much to your tables as it does to your sentences.
Tables and graphs should be placed as close as possible to the discussion of
the data which they contain. They should be so structured and so titled
that the reader can understand the figures presented in them without reference to
the text surrounding them. Proofread your tables and graphs with extra
care for it is very easy to make an error in copying figures.
The concluding section of the
paper, like the opening one, performs several functions. You should begin
by summarizing the findings clearly and concisely, showing how they confirm or
disconfirm the hypotheses with which you started. This section also
includes your own critique of your research indicating the shortcomings of your
work and suggesting how they might be overcome with sufficient resources.
Finally, the concluding section of the report should identify further questions
raised by the findings of your study which demand future research.
These exercises use the search term “voter turnout,” but feel free to try another term of your choosing.
[1] This essay is a revised version of “How to Write Research Designs and Research Reports,” © Charles McCall and the CSU Social Science Research and Instructional Council (SSRIC), 1998. Used with permission.
Last updated
April 28, 2013 .
© 2003---2013 John L. Korey. Licensed under a 
Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.