Writing online: what really changes?

You know you wanna

Before I roll out the details of this little mini-project, allow me to summarize.  This project was the first of many in an attempt to characterize the differences between online writing and more traditional formats.  Students in my Dual-Credit Biology course were divided into two groups upon culmination of a biochemistry unit.  Half of the class wrote a comprehensive unit summary in a traditional format which was turned in directly to the teacher.  The other half of the class composed a summary in an online forum for a much wider audience (Ning network).

Summaries were analyzed for word count, readability and effect on content-based exam scores.  Exam scores and readability were closely aligned.  However, strong differences were noted in average word count.  Students writing in online forums used significantly fewer words to achieve the same overall impact.  Implications of using online forums for future enhancement of student summarization are discussed.  My take?  Writing online is a potentially powerful tool for summarization of course content.

How well I could write if I were not here!

The set-up

A building-wide focus for instructional improvement in my high school this year, has been to launch and maintain an instructional technology integration initiative.  The school year began with a cohort of twenty teachers who were willing to engage in training above and beyond the professional development for all staff.  I began meeting with these teachers in mid-June for three-hour technology training sessions.  The first meeting consisted of a “care and feeding” session for the 15″ MacBook Pros, Olympus digital cameras and iPod nanos the cohort teachers received.  From that session forward, training sessions integrated this hardware, as well as emerging online technologies, with solid instructional best practices for classroom learning.  These twenty teachers have followed a prescription of immersion.  New technological tools were presented alongside potential uses in a classroom setting.  An online social network was set up to facilitate learning between face-to-face meetings.

I teach one course during the regular school day.  Principles of Biology is a course in which students earn 101-level college credit through Missouri Western State University.  This course is populated by students who enjoy learning.  Work conducted by these students formed the basis of comparison in this study.  Our course network is also based on the Ning platform.  Though this network features discussion forums as well as blogs, both are examples of online writing in some form or other. As you will see, in this class, I utilize the forum much more than the blog.  This will certainly have an effect on the results of this project.

The main event

What effects, if any, does a move toward online writing generate? Actually, little has been done to characterize the impact this new “genre” of writing is having on student achievement.  In fact, blogging has been said to be different enough as to warrant a new genre of writing called “connective writing.”  As Will Richardson has said:  it is

“…a form that forces those who do it to read carefully and critically, that demands clarity and cogency in its construction, that is done for a wide audience, and that links to the sources of the ideas expressed.”

In fact, I mentioned this idea earlier this year in this post on blogging.

How was this done?

This project began at the culmination of a biochemistry and nutrition unit.  Class sessions throughout were widely varied and ranged from cooperative work in small groups, guided webquests, lecture and discussion, and wet-lab investigations.  The final strategy prior to the unit exam was to have students engage in writing a comprehensive summary of their learning throughout the unit.  Students were randomly divided into two groups.  They were then informed that they would be doing one of two types of comprehensive summary.

The only difference between the two groups of students was writing “environment.”  The “odd” group was to compose the summary on a word processor, and then print as well as e-mail the final copy directly to the instructor only.  The “even” group was to write their summary online, as a blog post, to our classroom network.  Based on previous work with the online class network, these students knew that their work would be accessible online to virtually anyone.  This group of students knew that they were writing for a potentially wide audience, while the students of the first group were writing for me alone.  Since this class is largely an inquiry-based biology class, data was analyzed by the students themselves.  Students were asked to infer from the data and make conclusions online based on the findings.

So what happened?

Overall, 20 students participated in the study- out of a total of 20 enrolled in the class.  Each student in the table is listed only by initials in order to preserve anonymity.  (Table 1) Students who constructed a comprehensive summary of nutrition unit via online blog post for a wide and potentially global audience (even group):

(Table 1)  Students who constructed summary via online blog post for a wide and potentially global audience (even group):

(Table 2) Students who constructed comprehensive summary of nutrition unit via MS Word document handed in directly to teacher (odd group):

(Table 2)  Students who constructed comprehensive summary of nutrition unit via MS Word document handed in directly to teacher (odd group):

An examination of the data will show average exam scores differed less than one-half of a point across the two groups.  Average readability was also comparable, with the average score differing less than one Flesch-Kincaid grade level.  Easily the largest difference between the two groups was the average number of words used per summary.  Students writing online summaries used an average of 239 words less (399 opposed to 638) than those writing in Microsoft Word for me alone.  Variability in all data seemed fairly low for human studies.  Word counts were tightly clustered around the mean for online writers.  Though the results here seem fairly simple to interpret, there are many factors that must be considered in any analysis and subsequent application.

Jibba-Jabba

In this study, students were instructed to create a comprehensive summary of a unit on biochemistry.  The only instructions given were that the summary should provide an understandable context for the main topics of study, and that any source used should be cited.  It was also suggested to students that the mere creation of this summary would help to prepare for the upcoming exam.  Therefore, the only difference between the groups would truly be whether the summary was composed in an online forum or in a word processor.  Also- the online group knew their summary would gain a potentially global audience while the MS Word group knew that their summary would be read only by me.

Before extrapolating too far, it is important to note that when planning to implement any new teaching strategy, the first consideration should be to do no harm.  With that philosophy in mind, the data in this study immediately suggests that when students worked online, they certainly performed no worse than their counterparts who spent their time offline.

With that in mind, the fact that both groups showed no measurable difference in exam scores, is encouraging.  Therefore, even if an instructor wanted to use online writing as nothing more than a novel approach that might excite a few reluctant learners, they would likely do no harm toward content achievement.  I had initially hypothesized that due to the connective nature of this form of writing, students would better assimilate the content of the unit and show higher exam scores.  When considering this initial study alone, that hypothesis was not supported.  Though it is also important to note that the individual classroom climate and culture could impact these results heavily.  Not only would the feel of a classroom influence these results, it is easy to see how previous instruction could change things to a large degree.

To this point, few significant differences have been shown between the two study groups.  However, when looking at raw word count per summary, things quickly diverge.  It was anticipated that the group engaging in online writing would be stimulated to write a higher volume of words.  Hey-  it’s a novel approach.  Writing online is in contrast to a traditional approach where the work is done in isolation from start to finish.  In fact, this is the opposite result seen in the trial.  The group writing online submitted a much lower word count (avg. = 399) than the word processing group (avg. = 638).  This is no small difference and would certainly register as significant on any statistical test.

Why so many fewer words per summary when writing online?  Many of the participants had an idea about this when analyzing the results after the fact.  The following statement by student “RH” typifies a common student response:

“It appeared that the papers had a higher readability and word count, which I kind of expected because I think people tend to write more formally on papers, whereas the blog posts tend to be more opinion and informal writing.”

The aspect of formality is something that was not considered to be a factor prior to the study, and yet it makes solid sense.  The type of online writing these students had engaged in prior to this study was largely of a reflective nature.  Our work online has tended to center around written reflections that helped to synthesize classroom sessions.  A more formal approach to blogging has simply not been utilized as of yet with this class.  This is something that could certainly affect the results of this study.  Online writing has been approached in many ways for many different reasons in classrooms across our building this year.  It is very interesting to think about furthering this study to investigate the details of this interaction between instruction and writing.

First phase data is soon due in from three other participating teachers.  In contrast with the class in this study, several of the other participating classes feature some “reluctant” learners.  It will be interesting to see if the results of those trials differ from these in any way.  I would guess that they will.  The analysis of this new data, will certainly provide a jumping off point for the next round of research in our school.

Wheww… finally

To conclude, this brief study demonstrated several things.  For one, the mere act of writing of a comprehensive summary prior to a unit exam seems to be an effective strategy for a class of mature high school students.  Furthermore, the nature of this summary did little to affect scores in this study.  Students writing in online forums showed an average score almost exactly equal to that of students writing in a more traditional (printed) format.

The most signficant result of this study was the analysis of word count between the study groups.  Students writing online submitted significantly more concise summaries with smaller word counts.  The implications of this data are very interesting.  Furthermore, students who wrote less (when writing online) performed equal to those students who wrote many more words.  This could directly point to the power of online writing in helping students to summarize effectively.  This is no small feat considering the difficulty many of today’s students tend to have with summarization.  Hotshot ASCD guru, Robert Marzano, makes the case that “summarization has a robust and long history of research,” and is one of the “nine most effective instructional strategies a teacher can employ.”  These final results will likely have implications for not only future research trials at our high school, but for immediate classroom action in the area of content summary writing.

You?

What do you think about writing online?  You obviously read online.  You likely even write online.  We would love feedback about what goes on in your mind when you write online in different settings.  What happens when you blog?  How are discussion forums different?  What difference does a global audience make- if any?  Weigh in.  What does change when you write online?

View SlideShare presentation or Upload your own. (tags: students pedagogy)

Artwork thanks:

*”How well I could write if I were not here!” by Esther_G on Flickr.
*”Live Blogging at Woolfcamp” by Sue Richards on Flickr.
*”WORDS” by Feuillu on Flickr.
*Remainder:  me.   😉
.

Sean Nash

Biology teacher in the great state of Kansas. Back at it in the classroom after a 30-year career in Missouri. Former District Curriculum Administrator, Instructional Technology Coordinator, and Instructional Coach. Biology instructor since 1993. Find more about my passions and my work at http://nashworld.me

10 Comments

  1. What do you think about writing online?
    I think writing online is cool. When I write online it’s more like “stream of consciousness” than formal essay-like writing. When I write online I write what I really think and feel, whereas on paper I’m more inclined to stick to what I consider normal and expected. Basically, on paper I write what I think my reader wants to read, so it comes out boring with little personality.

    I love participating in forums. I really enjoy engaging in discussion, I guess it’s that opinion driven writing that I don’t often get to produce on paper.

    What changes when I write online? My personality. I write from my heart, not just my head. I write from “the real me”. I write with emotion, with opinion, but with just as much thought as I do on paper.

    Hmm, I wonder if I can convince the leadership team at my school to let my students write like this. It would be great if they felt empowered to write from “the real them” rather than the boring paper-style writing I’m sick of.

      • @nashworld, you’re right – I meant convincing the leadership to allow my kids to blog. I’m pretty sure blogging sites are blocked here, even del.icio.us was blocked and they weren’t keen to unblock that (apparently it allowed students to communicate with other people). I’ll be teaching students in years 8-10 (13-15/16). We’re in a pretty conservative country town though.

  2. Wow, first let me say that is was a really great study. A group of 20 teachers at your school are using online technologies and they get a Macbook Pro?!?! I’m the lone ranger at my school. I’m so jealous. May I come to your school please?

    I know you did a readability test on each group, but did you notice any redundancy in the writing in the paper group? Did they write more (and repeat things more) because they think a paper is better if it’s longer?

    Ability to summarize is definitely a good skill. I’ve been using Twitter with my AP students and I like how they have to get to the point of what they need in a very limited number of characters. I don’t know if it’s making them better writers, but I definitely plan to ask them at some point what they think.

    It would be interesting for me to run the same test with my students. Rather than a social network, we use a blog and the writing in their posts is still pretty formal. However, the comments to posts are informal.

    It would also be interesting to ask the students in the online group if they developed more interests in the topics because of the online environment. A test as a final measure of learning may not be the best way to evaluate everything that was gained from the online environment.

    Maybe because I have a more reserved personality I don’t feel that writing online makes me feel more free the way that Claire does. Also, because I don’t think any writer can truly remain anonymous online for long, I’m still fairly careful about what I write because administrators, parents, and even some students read what I write. The ability to share my writing with the world is the number one reason I do it.

    • @missbaker, Ha! Come on dowwwwwwn. Our town would be lucky to have the addition of your talents. Don’t tease too much there- I am one heckuva recruiter. Think 20 is cool? Wait until phase two begins in June… and the remaining 70 come online. Wow. Now you see why I chose 20 in the first year. That is going to have to be some seriously shared training. I can’t wait to see how that throws everyone into the same boat. I think it could do wonders for climate & culture overall. Congruency of mission is a powerful thing. I do know what being a lone ranger feels like, and it does feel so much better to have a posse.

      I didn’t notice so much redundancy (and the beauty of this data- is that I will always have it digitally to examine later). But what I did notice is the MS Word need to “talk around” a concept. Say -explaining the structure of glycogen- as opposed to linking to a visual for the reader. That certainly does account for some of the difference.

      A bunch of my kids and I use Twitter, but we haven’t yet for real content. Just communication. But I am certainly open to that.

      Yes, yes, yes… replicating on your end would be very cool. I would certainly link it to this one as an update at the end, or… you could do it as a guest post here. (?)

      I also agree wholeheartedly that a raw exam score means little. To be honest, that one was thrown in there for other folks who tend to gravitate there due to familiarity and accountability. If I hadn’t done that one… it would have been the first question asked. 😉 I think an investigation of the real learning that occurs there is forthcoming, and will be far more important.

      And no- I don’t feel “free” when I write here. Not even close. However, it is easy for me to tailor tone & detail considering the potential audience. I think the critical audience makes me consider everything I think -and thus type- all the more. So for reflection, it is awesome.

      Thanks for the thoughtful reply.

  3. I am surprised that little difference was found between the two groups being researched. I, like you, also predicted that the on-line forum would out preform the traditional word processing group. If it were me, and I knew a global audience would be reading my paper, I would probably write a more in depth paper than I would just for my instructor.

    With this same thought, I will be interested to see how the “reluctant” students fair with this project. If by “reluctant” you mean a student who can perform academically but is difficult to motivate, I predict this will produce a positive outcome, and this group will score higher than the traditional word processing group.

    On the other hand, if by “reluctant” you mean a student who falls behind academically from their peers, I believe you will find the opposite results will be found. If it were me, I would do anything I could to hide my learning difficulties, and making my possible deficits public for the whole world would be the last thing I would want to do.

    I concur with missbaker that writing online does not give me a feeling of freedom. Although while I do not feel freedom, it is no reason for not seeking new avenues for our students to express their intellectual thoughts and wonders.

    • @karen, – Hey Karen! You are correct- I consider “reluctant” learners to be tough cases with regard to motivation & effort. I would also think that the larger audience would be quite stressful for students with learning difficulties. However, if you check the slide presentation embedded above, you’ll see it starts with a screenshot of several of the online networks at BHS.

      You’ll see that I took a screenshot of one called “LA Resource.” Immediately after, there is a screenshot of a login/password for that site. What I explained in the presentation was that putting up a wall in certain circumstances is a good thing. With that wall, and only invited guests allowed in, these students are writing volumes more than in the past! In fact, just ask the boss lady (invited guest on many) how often she is drawn in to “read what I wrote” today. Powerful.

      Thanks so much for joining in. I respect your opinions on pretty much all things educational. 😉

  4. So, I am wondering if the word choice was more precise and concise (ACT skills) on the blog vs the essay writing. I wonder if there is a way to “test” that. I’m also wondering about details, examples and transitions used. I remember when I conducted my action research a few years ago and I looked at readability, word count, etc., that some of the kids who had a higher readability didn’t necessarily have a higher quality essay. It was just one factor.

    When school is back in session, would you be up for an examination of the student work holistically?

    By the way, I didn’t go much further with the blog vs. word “study” in my class because the blog posts were so, well, insufficient compared to what we were doing in class. Interestingly, we in Com Arts have taught the kids that deep thinking writing is process based and constructed response is short, sweet and to the point. They were viewing my blog assignments like constructed response questions and not well-developed responses with deep thinking, which is what I wanted. Plus, we WERE doing process writing, and the blog was not working for that at all.

    • @Kelly Lock, I agree… I am a person of details when it comes to writing. Though I teach biology, and only grade science literacy content… I do a ton of “consulting” with kids on their writing. It never seems to be enough though.

      Evaluating precise & concise at that level might be a bear, but I would be more than interested. I think if it were that simple, more would have done it. But then again- perhaps not. Of course I would be up for a closer look. Actual scoring of student writing is what I had hoped to work with you on. I can’t wait for that. We have too few discussions about teaching & learning as it stands. 😉

      Also- let’s look at some of the nuances of: “the blog posts were so, well, insufficient compared to what we were doing in class.” Fun stuff. Get recharged in the next couple of weeks. I certainly intend to!

  5. I am impressed with your study and the way you present the information. Not only do you give concise background information, but you give excellent data to back up you findings. This is an impressive blog post and study!

    I know that when I write online, I am much more inclined to present my ideas in a clear, consistent manner. I understand that my online writing has the potential to reach an international audience and I must present myself and my country in a positive light.

    Your blog creates an excellent analysis of the study and research. You added quotes from the top of the educational field. Well done and I look forward to reading your posts!

Leave a Reply

Your email address will not be published. Required fields are marked *