Empty citations?

By Sam Graham

Last week, I had lots of students coming in about their reflective writing essay. The essay had students evaluate whether self-directed teams increased productivity by having them link their experiences from four group meetings with literature about SDTs, while simultaneously considering the groups strengths and weaknesses and getting them to articulate what they had learned through the process. It was quite a beast of an essay in terms of possible complexity, and, from the essays I saw, students had generally done well at linking all of these together.

The most common question they had – at least early in the week – was how to find sources. The questions weren’t so much about how to research the topic well as they were about finding good sources that would let them tick off their minimum five sources requirement. I explained the research process below and tried to explain that the purpose of reading other sources was to refine their thinking, not just to fulfil the assignment criteria of five sources or give (faux) credibility to their writing.

Queue this morning, reading about The Citation Project.

Sandra Jamieson (Drew University) and Rebecca Moore Howard (Syracuse University) are leading an analysis of students’ citations in order to understand how they use source material. So far, they have looked at 1,911 citations from 174 student papers from 16 colleges or universities around the US. An interview by Project Information Literacy summarises things nicely.

Below is a summary of the facts that they found with a combination of their and my interpretation:

  • Citations are usually from within the first two pages of the source. This seems to indicate that students are hunting for a supporting quotation rather than reading and interpreting the entire text.
  • Of the citations analysed,
    • 42% were direct quotations,
    • 16% were patchwritten,
    • 32% were paraphrased,
    • 6% are summarized.

While it might be encouraging that 74% of citations were direct quotations or paraphrased, the authors argue that it is the summaries that demonstrate an understanding of the larger arguments in the text. In their shorter analysis of 18 papers, Howard, Serviss and Rodrique (2010) concluded that “these students are not writing from sources; they are writing from sentences selected from sources.” (The definitions of study weren’t particularly difficult to attain in this study; it required taking a paragraph or more and rewriting it in 50% of less words.)

  • Most sources were cited just once. This seems to indicate that students were taking supporting details rather than engaging with entire reports/arguments. It might also indicate that students are putting in citations to satisfy a requirement for a minimum number of sources.
  • Of the citations analysed,
    • 24.4% were to journal articles (mostly academic),
    • 17.9% were to books (though including non-academic books)
    • 24.5% came from web-based sources,


    • 26% come from sources that are two pages or less in length
    • 44% of the citations are to sources that are no longer than four pages.

This seems to indicate that students have only a partial understanding of what constitutes an academically appropriate source.

One interpretation of the above findings is that students are not following the research process in the diagram above to formulate an informed thesis, and are instead finding sources to back-up their pre-conceived ideas. This process might look something like this:

Another interpretation is that students are unable to summarise long arguments, as described by Howard, Serviss, Rodrigue (2010). Instead of bringing together various elements from a report or argument into a short summary, they instead choose what they think is the single most important element and repeat that in different words. (I’m not sure if this is a co-interpretation or an alternative interpretation.) This might explain why students were using sources at a sentence-level rather than at a wider-level. It might also explain why they are citing each source just once; if they are looking for the most important part, they might identify it in one place and cite only that part.

To me, this also fits into the idea of a zombie academy. If students are required to cite a certain number of sources, we shouldn’t blame them for picking out a sentence from the first page to tick this off the requirement list. If they’re sent the message that grades should come first, we shouldn’t blame them for putting these ahead of real understanding and learning, even if their efforts are counter-productive. Where the focus of good citation is to not plagiarise, we should understand when students aim for a 0% Turnitin score but don’t think to aim for good citation practices.

This is, I understand, a bit of a leap. For the moment I’m just trying to fit things together.

Rebecca Moore Howard, Tricia C. Serviss, and Tanya K. Rodrigue. “Writing from Sources, Writing from Sentences,” Writing and Pedagogy, 2.2 (Fall 2010): 177-192

Tags: , , , ,

4 responses to “Empty citations?”

  1. David DeBrot says :

    This is a well-reasoned and relevant piece on what is probably occuring consistently among our students. This raises for me the question of how our own staff are assessing the students’ use of citations and support (ticking a box as well) and if the students sense that there own arguments are not being examined/analyzed as a whole.

    One practical step that I believe could compliment a more wholistic analysis of arguments from the texts students read is a spoken assessment where students have to verbally defend/explain their own analysis of a written piece or set of texts. In fact, various students or groups of students could be set up in a debate format to argue through the piece or set of texts and could be assessed on how well they support their analysis of the arguments.

    What do others think of this? Is this a practice we see happening here, and if so, what is a way to advocate and support true learning (as Sam has mentioned here) as opposed to a zombie appetite for potential empty ritual around citing published work?

    – Dave

  2. Matt says :

    Hey Sam – yep interesting piece, nice work! Something that kept coming up from OB students down here was that they were becoming frustrated because they didn’t understand how they could possibly write 1,500 words about Self-Directed Teams when McShane (coursebook) had only devoted about half a page of a 600 page textbook to the definition of SDTs! They hadn’t realised that they could draw upon all of the concepts in McShane to better explain their experiences in their work group meetings. I was getting things like, “But this only explains individual behaviour!” etc, etc. Once they made the connection that individuals make up groups and that studying individual behaviour within groups is also relevant, the ‘penny dropped’! This was a real learning experience for me as I hadn’t predicted this whatsoever, even though I had explicitly said in both workshops that they should draw on a number of sources to complete the task requirements, not to mention that McShane is full of stuff!.

    Your bit on citations is timely as only 2 days ago I had an ‘HD’ student in here writing a literature review for Workplace Orientation. On reading over some of her work, she had clearly failed to put a ‘voice’ to it, meaning her piece appeared to have been put together by merely ‘dropping’ reformulated/paraphased sentences/quotes into her work literally one after the other resulting in no apparent engagement/analysis with what she had read. As a result, there was very little discussion nor comparison of the literature and hence no ‘voice’ to what she was writing and it did come across as though she was merely dropping in citations in order to impress her lecturers. This was a challenging ‘drop-in’ as she had left this assessment to the last minute and had no time to go back and really think about what the literature was saying and how it relates to workplace applications. The best I could do at that point was to encourage her to, as I say, put her ‘voice’ to it. This makes me wonder, is this an isolated case on her behalf, or is she considered a high achiever because of her ability to show that she can ‘drop-in’ 20 or so references in a reasonably cohesive essay? Plus, if an HD student is doing this, what are the other ones doing who haven’t been in to see us?

  3. Sam says :

    There seems to be a connection between Dave’s question about how the use of sources is assessed and Matt’s case of the HD student doing what would look, at a quick glance, to be technically great: it’s a *lot* of work to really assess how sources are used, and, realistically, lecturers can’t be expected to assess them to the extent that this study did. Bear in mind that in this study the analysis was done by 23 people, and they weren’t doing it several times a semester!

    I remember marking level 5 graph descriptions. After a while I felt like a computer: I would process the grammar and structure, and would recognise accurate descriptions without really absorbing the meaning. I’m sure I often let through some pretty major errors and shallow descriptions! But with a lot of marking, it’s what seemed reasonable.

    So, I really like the idea of testing students orally, where they have to respond on the spot to an idea, which really tests whether they’ve absorbed and processed the ideas, not just a few quotes from a set number of sources. This would also acknowledge that the use of sources won’t, realistically, be assessed in great depth.

    At my university (undergrad), we had two one hour tutorials a week usually with one academic and two students. The students would take turns reading their essay, and we’d spend an hour talking about the points raised. The essays weren’t assessed, but it was embarrassing if they were poor. If you just made an essay that looked good (sentence-level citation, not reading full papers/chapters) it was an awkward hour. And then, there was a strong reason to actually engage with the material – there was going to be an interesting conversation afterwards! I know this model can’t be copied everywhere, but I do wonder how the principles could be applied.

    I think our comments here have related to teaching and learning. What’s our role in this?

  4. Magee says :

    Hi Lads,

    I think the genre of the work in the study Sam has referred to is important to consider. Reflective writing is much harder to support than purely academic writing as it is based on feelings. If it is our experience we are writing about, the motivation to defend it is lower because in our minds it is already factual.

    Currently I am drafting a reflective essay on my practicum experience. 15 years of teaching made me over-confident and I wrote the essay off the top of my head – this weekend I am retrospectively inserting quotes that support what I already knew/believed about teaching. As Sam’s research suggests I haven’t learnt very much at all in this subject. The referencing and citation is like a dictionary quest using text book indexes and google searches.

    However, the specifications of the assignment were to define my personal pedagogical philosophy and then support it. So that is what I did, and in that order.

    I think that by modifying the question to something which asked students to explore – lets say 6, pedagogical methods and then based on experience justify those which they support – ie putting the cart before the horse would encourage deeper research.

    In terms of teaching and learning, when designing reflective assignments, I think it is more effective if criteria stipulates that academic research come before empirical analysis. Students follow instructions when writing something which has an audience rather than when reading which is a private accumulation of knowledge.

    The idea of verbal assessment is interesting, but assessment of students’ verbal skills in a foreign language can involve personality assessment and assessors can be subjective and therefore influenced by the confidence or likability of a candidate.

    So that’s my two cents

Tell us what you think

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: