[ARCHIVE] What is process tracking and how is it used to deter AI misuse?
We are probably all familiar with version history in Word or Google Docs and may have heard of students being asked to or choosing to share version history to provide evidence that they wrote something. By “process tracking” we are referring to a wide range of practices that involve gathering data about how a piece of writing was created. Depending on how process tracking is done, such data can include
- How much time was spent actively working on the document
- What text was copied and pasted
- Which websites text was copied and pasted from
- Which sections of the text were typed
- How much editing and revision there was
- How much text was edited using AI grammar and style suggestions
- A replay of the writing process, including early versions and editing
- What dates and times the document was worked on
We note that the last year has seen a proliferation of different software approaches to tracking the student writing process in order to prevent or detect AI misuse. In some cases, this functionality is offered alongside AI detection, and in some cases it is framed as an alternative to AI detection. Some schools or individual teachers purchase proprietary software that may or may not sync with an LMS. Another approach is for instructors ask students for edit access to documents and then use free browser extensions to view process history. In yet another model, students opt to track their own history and view a report that they may be required to share with their teacher.
Some considerations include
- Is asking for data on the writing process history overly intrusive? Is it a form of surveillance or close observation more akin to in-class writing? Are there lines to be drawn in terms of what should be tracked and what should not?
- Alternate writing processes and accessibility: Students may want to or need to write using different software (such as a phone app or a dictation app) than the process tracking system mandated by the instructor offers.
- How effective is process tracking at deterring AI misuse? We note that students could still retype AI text, spend time in a document, and insert edits. To what extent would the effort involved in doing that reduce misuse? To what extent will software that effectively simulates a human writing process become widely available?
In initial task force discussions, we find that members have a variety of sometimes conflicting views on process tracking. Here we offer some preliminary perspectives.
Prompt: What are the possibilities, limits, and implications of process tracking programs for faculty in reading and writing-intensive disciplines ?
Kofi Adisa:
I typed a rant, deleted it, and I started again. And again. If I were a student drafting my response to the prompt, I might feel as if my process is idiosyncratic, perhaps even a secret; however, if I knew as a student, I had to share my process or worse to see that it was being tracked and that information was somehow in the purview of my professor, I probably would be too self-conscious and worried that my process was judging my writing. As a professor, I never ask students to do something for the class that I couldn’t do if I were a student. The closest I ever got to requiring students to share their process was to have them share their notes or annotations on or about a particular reading. I cannot imagine having them walk me through their reading and writing process or even wanting to track their process with some kind of technology. Yet, I know there are faculty at my institution who welcome this kind of intrusiveness. (I see it as an intrusion; it’s like asking me what my diary entry was when I was a young boy or some other kind of personal matter that’s unnecessary for me to respond to a prompt.)
My colleagues have argued that tracking student writing helps students “see” their writing differently. They also argue that it gives them insight into how students think about a topic. That’s why they “love” having students use G-docs. And I will admit that seeing “version history” is enlightening, but is it something worth evaluating? Is having students share any moment of their process necessary for me, their professor? I don’t think so. However, as more and more students use AI tools, I believe some faculty may rely too much on the surveillance of writing than the actual teaching of it.
I may come back to these paragraphs and rewrite them. Again. The human part of writing is having an authentic voice, a sound. Not every student desires this part of the writing process, nor do they always value reading the voice of a text. But I try to draw students’ attention to it so they can see and hear it for themselves and in themselves. (MTK)
Antonio Byrd:
Writing Studies has been (justifiably) obsessed with process for decades, going back to the 1960s when writing became a subject of research. Foundational scholars such as Janet Emig, Peter Elbow, and Donald Murray advocated for a focus on teaching the writing process. They opposed the dominant product-focused pedagogy, which focused on learning the features of four modes of writing – narrative, expository, expository, and persuasive. Teaching for product meant asking students to follow formulas, adhere to correct grammar, punctuation, mechanics, and neatness. But the entire experience writing itself – from ideation to final product – was more complicated, as Emig wrote in her book The Composing Process of Twelfth Graders. Murray describes the writing process accordingly:
What is the process we should teach? It is the process of discovery through language. It is the process of exploration of what we know and what we feel about what we know through language. It is the process of using language to learn about our world, to evaluate what we learn about our world, to communicate what we learn about our world.
There is one caveat: Murray distiless writing process to three stages: prewriting, drafting, and revision. As Writing Studies developed different perspectives on what mattered to the writing process, from cognition to expressivism to sociocultural influence, teachers fell into the trap of making the writing process a series of stages. My graduate students and I reviewed the process movement and we all agreed that the different stages are helpful for structuring a class over 16 weeks but in reality the writing process is more complicated, is, dare I say Writing Studies’s favorite words, recursive and/or iterative. To remind ourselves and to teach our students this reality is to embrace post-process theory, which apparently never really took off but nevertheless echoes in our disciplinary language.
There’s more in the process movement! I recently assigned my graduate students a couple of chapters from Writing Workflows: Beyond Word Processing. Based on qualitative research, Tim Lockridge and Derek Van Ittersum argue for analyzing the writing technologies we use in our writing process. What tools do we load up on our computers or what notebooks and pens do we grab to do the work of composing? Workflow and the friction we encounter while working with the writing tools we’ve picked remind me that writing is networked, ecological.
And how do we account for the writing process? I loved portfolio-based assessment when I was a graduate teaching assistant for University of Wisconsin-Madison. I disliked rubrics and enjoyed writing in response to my students’ writing memos, which detailed how all the pieces in their portfolio connected to learning about writing, and the final product which did and did not matter to me.
Why have I used this space to draw a super rough sketch of the process movement? Because process-tracking software, I think, doesn’t align with our values in Writing Studies. We care about text, but we also care about context and subtext. We care about the narrative our students weave about their process, and we care about their takeaways, like what transfers and what students learn about themselves as writers. I guess if you use a version of labor-based contract grading that asks students to keep track of their time working on writing, process tracking fits in your pedagogy. But I understand the writing process as more than time on text or text copy and pasted, but, ideally speaking, a transaction of love between writer and reader. I accept the expansiveness of the writing process, with all its messiness, with all the anxiety students may feel, with all the triumph we both find at the end. I believe if you create a classroom where writing is zero stakes (we’re not trying to publish anything we write after all. Most of the time.), then we have less need to use genAI or at the least mitigate its presence, so we can only connect through the sharing of language.
Leonardo Flores:
Process tracking software, such as Grammarly’s new service “Authorship,” is being presented as a tool that allows people to defend their writing by providing evidence of their process to others in case they are accused of using AI. This is troubling to me on two levels: the first being that it is built with mistrust as a baseline. And because there are power imbalances in many educational and writing situations, it is a simple matter for faculty, publishers, or institutions to require the use of this software as a form of surveillance against those who are doing the writing. So what is offered as an opt-in tool can very easily become what Jeffrey Moro called “cop shit,” which he defined as “any pedagogical technique or technology that presumes an adversarial relationship between students and teachers” (2020). This subverts what is a good thing in writing pedagogy– focusing on process– and turns it into a prescriptive and auditable form of surveillance.
My second concern is that it can and will be used by educators to avoid updating their pedagogical practices to account for the impact of generative AI. Like AI detection software, it is simply another weapon in an AI arms race. AI detection has been proven to be ineffective and problematic, it has still been widely adopted by faculty and continues to be developed and marketed by educational technology companies. Both AI detection and process tracking can be easily circumvented, but the harm they do to writing practices and pedagogical situations are lasting.
We cannot turn back the clock to a time when writing could only be done by humans and was therefore a convenient tool for evaluation of student work. Generative AI technologies are being increasingly integrated into writing tools and workflows and they will become ubiquitous and naturalized for current and future generations. We need to cultivate AI literacy and update our teaching and writing practices to prepare our students for a world inflected by these technologies.
Liz Losh:
We should be mindful of potential stigma associated with certain kinds of writing practices, such as writing large parts of papers on cell phones or using speech-to-text software. If we force students to disclose information about how and when they write, it could undermine trust and egalitarianism. These kinds of writing practices should not raise any red flags for academic integrity, but students may feel shame about what could be seen as non-normative behavior. They might feel uncomfortable about not owning their own computer or having learning anomalies that require adaptive tools. In Jay Dolmage’s “Writing Against Normal” (2012), he observes that too many writing teachers “have not acknowledged that we have a body, bodies; we cannot admit that our prevailing metaphors and tropes should be read across the body, or that our work has material, corporeal bases, effects and affects.” Surveilling these embodied interactions can make many students feel self-conscious. We already saw this during online teaching at the start of the Covid pandemic, when students who attended class from their beds or their cars didn’t want their cameras on.
As I argue in The War on Learning (2014), it is often not really disrespect for academic integrity or academic honesty that aggravates educators. It is academic labor. Some students don’t invest their labor in the learning enterprise and faculty feel that they must take on more labor in the interest of fairness. Unfortunately, just like a timeclock, it is difficult to always establish who is on the job and when. And we as faculty might not want our own time-stamped feedback practices to be visible to students.
Anna Mills:
As in so many other domains like parenting and work, supporting intrinsic motivation goes a long way, but some form of accountability is still needed. Whether or not we allow use of AI, teachers do need to know what a student wrote in order to meaningfully assess and respond. AI misuse is less likely if pedagogy supports student motivation, a sense of belonging, and support to weather the ups and downs of writing processes. However, even with the best pedagogy, some temptation to take shortcuts with AI will remain when students feel insecure or under pressure. We can’t prevent all such misuse, but we can significantly reduce it by making it more of a hassle.
I have been incorporating process tracking in a limited form into my writing classes for the last year and a half. According to a recent poll I did on X, educators are split on this topic, but it seems to me a much better approach than AI text detection, which is inaccurate, likely biased, easily circumvented, and not transparent. Accountability doesn’t have to mean an adversarial relationship; I try to approach it in a spirit of collaborative reflection. If a report shows a big copy/paste or only a few minutes of time spent, I meet with the student to understand what happened. If they tell me they used their phone to dictate something, we discuss the content a little, and then I generally take their word for it. If it seems they did misuse AI, I offer them a chance to rewrite. Going forward, I want to find a way students can comment in the margin of their process report so it helps them reflect on the strategies they used and how they worked with sources or AI.
We don’t consider in-class writing to be surveillance, and I hope we can establish norms for process tracking that are parallel to the kind of observation we do in class. In my class, students see their writing process report first, and I ask them to share only total time spent, which text was typed and which was copied and pasted, and how Grammarly was used–not early versions or timestamps. As with in-class writing, we need to provide accessible alternatives. I offer to work out another way of showing a student’s process, such as an extra conference with me, if the student is not comfortable with process tracking.
This semester, several students have shared with me that they have cheated with AI in the past, and they see other students cheating with it. Acknowledging that this is tempting and asking for some transparency is reasonable. I’m trying to send the message that their learning matters to me, and I want to know their words are theirs.