Communication for CIDUI International Conference - http://cidui.upc.edu (5-7 July 2006), Barcelona ( pdf - 1.08 Mb | Wiki-slides)

"Writing documents collaboratively in Higher education using Traditional vs. Wiki methodology (II): QUANTITATIVE results from a 2-year project study"

Xavier de Pedro * (1, [email protected]), Maria Rieradevall (1, [email protected]), Pilar López (1, [email protected]), Dolors Sant (2, [email protected]), Josep Piñol (3, [email protected]), Lluïsa Núñez (4, 5, [email protected]), Miquel Llobera (6, [email protected])

(* corresponding author)
(1) Dept. d'Ecologia, University of Barcelona (UB). Avda. Diagonal 645, 08028 Barcelona
(2) Dept. de Fisiologia Vegetal (UB). Avda. Diagonal 645, 08028 Barcelona
(3) Dept. de Biologia Animal, Biologia Vegetal i Ecologia, Autonomous University of Barcelona. Edifici C (Campus UAB). 08193 Bellaterra
(4) Dept. de Biblioteconomia i Documentació (UB). C/ Melcior de Palau 140. 08014 Barcelona
(5) Aplicacions, Àrea de Tecnologies (UB). C/ Melcior de Palau 140. 08014 Barcelona
(6) Dept. de Bioquímica i Biologia Molecular (UB). Avda. Diagonal 645, 08028 Barcelona

Group collaboration, Wiki, Cooperative learning.

(up to approximately 200 words)

If the first part of the results from the recent projects in innovation teaching using Wiki methodology for doing group assignments in university degrees showed the qualitative results (De Pedro et al. 2006), this second part focuses more on the quantitative results. Data came from three sources: (a) individual surveys to students using either Traditional or Wiki methodology, including closed questions (1 to 5), (b) a table where they auto-recorded the time invested per week and per type of tasks to complete the group assignment, and (c) their academic marks in the group assignments, evaluated with common criteria for both methodologies. Conclusions from the results obtained stated four main ideas: (1) Wiki methodology in big groups - 15 students - showed a clear enhancement in quality (higher marks) with less overall time devoted to finish the work; (2) Results for smaller groups were not clear, and performance using Wiki methodology seemed to depend on other factors; (3) an "editor in chief" is very important and needed for higher final quality of the work produced, regardless of methodology followed, even if this task was apparently facilitated by Wiki methodology; and (4) students using Wiki methodology devoted less time than the ones using traditional methodology in the items: "writing the work", "exchanging files and other material", and in some cases, in "structuring" and "homogenization of the style, markup, and content" of the work, as well as in "meetings in person".

i. Objectives

This communication is the second part of the one entitled "Writing documents collaboratively in Higher education using Traditional vs. Wiki methodology (I): QUALITATIVE results from a 2-year project study" (from now onwards, called as “the first part”, De Pedro et al. 2006). This one focuses more on the quantitative results obtained in the frame of the two projects cited there (UniWiki – 2003MQD00167, and UniWiki-Redice 2005ICE-UB).

We wanted to know if Wiki methodology reduced the need to use some other ways of distant communication, if it reduced typical organizational problems at doing group assignments, if students obtained better yield of the time invested, and also the degree of satisfaction that students obtained using each methodology. Moreover, we wanted to know if Wiki methodology allowed reducing the time invested onto some tasks commonly needed to complete the group work.

ii. Description of the work

Introduction

The results shown here are framed within the same context as reported in the previous communication (De Pedro et al. 2006).

Several TikiWiki sites were configured (Figure 1), one for each subject participating in the UniWiki project, to allow single cooperative virtual communities for each subject (one per subject , Table 1).



Each TikiWiki collaborative portal site was later on manually configured to host different work groups, since AulaWiki module to automatically allow and configure work spaces was not yet developed by then (De Pedro & Reyes 2006). Each work group had manually created by the teacher their own private forum, file galleries, Wiki pages, ... as many TikiWiki resources, and as private or public as needed for the specific case of configuration needed for each subject.

Methodology

We defined in “the first part” what we called Traditional methodology and Wiki methodology (De Pedro et al., 2006), and we use the same terminology here.

Permission denied

Data shown here (Table 2) were collected through individual surveys (Annex 1) with closed questions (answers ranging from 1 to 5), plus individual sheet per student containing a table to allow them to record the time they invested to fulfill their tasks in the group work ("self-recording table", from now onwards; Annex 2). Students had to hand in the sheets to their teachers at the end of the term inside a closed envelope labeled with their name on it, and they were not opened until the grading of the group work was completely finished.

Permission denied

The last set of data came from the academic marks they obtained from the grading of their group work, which was performed without taking into account the type of methodology deployed by students (same criteria for grading the final document made in group through any of the two methodologies).

iii. Results and Conclusions

Results

There where a total of 229 surveys and 223 self-recording tables collected (divided in methodologies and subjects in Table 1 in the accompanying communication; De Pedro, 2006).

Among the items answered in the collected surveys, some results are shown specifically for the subject "Evaluation of Environmental Impact, course 2004-2005" ("AIA 0405") in detail, from which the more extensive and balanced data-set was gathered (circa 30 students with Traditional methodology versus circa 30 with Wiki methodology).

Methodological issues:

Students with Wiki methodology did not use as much the more common ways of communication (phone, chat or instant messaging, e-mail) and of exchange of documents (floppies, USB-drives, ...), but they employed more the Wiki technology. Moreover, meetings in person were reduced in the groups of students that followed Wiki methodology.

Organizational issues

In general, first-cycle students showed less difficulties to hold meetings during the week than those of second cycle. This is totally justifiable due to the bigger dispersion in subjects of last-years students, in spite of the case that subject was compulsory or optional in the degree.
The difficulties of internal organization inside groups were very heterogeneous, and they did not seem to be tied to the work methodology but more to the characteristics of each group of students.

Marks and time yield

In very big work groups (AIA, 15 students for group) there is a clear advantage in the yield of the time invested for the students that worked using Wiki methodology respect to the ones using Traditional, with more frequent occurrence of high marks, for the same or less time invested (Figure 2).



In smaller groups (2-4 pupils for group, generally), where it is not essential to use special methodologies of work in group, there are not evident differences, and the success in the use of the tool might be more related with other factors:

• students were appropriately motivated by teachers
• students controlled, previously to their use in class for production, the functioning of the new tool that they were going to use. This would imply that all students would have joined necessarily some suitable specific guided formation on the Wiki methodology, and on the correct use of the Wiki technology
• students had difficulties to meet in person (all the members of the work group), with as much frequency that they felt it was necessary for them
• students didn't refine at the end the writing, markup, grammar and spell checking, removing duplicities of information, synthesizing again, if needed, etc.

Student satisfaction

There are some differences in the perception of the satisfaction about the method between the pupils of first cycle and those of second cycle. On the one hand, students of first cycle compulsory subjects, if they can meet every day in class and that are more usual to the multimedia technologies (Functional Ecology), then they need more tutelage and direction of the work, and they do not seem to appreciate the advantages of the methodology so much. The students of optional subjects of the first cycle that can not meet every day in class (because most of them work besides studying, as it happens in SIMA "Health and Environment" from Nursery degree), they do appraise it indeed more positively than with the traditional method.

Students of second cycle feel satisfied and recognize the advantages of the Wiki methodology, especially regarding the subject AIA and EAUB. In some specific cases, however, they did not use the tool due to technological handicaps and because they were not used to this methodology of work.

"AIA 0405" specific case study
Students that used Wiki methodology showed a higher satisfaction degree for the tool than the students that didn't use it but knew about it because their colleagues told them the details (Figure 3), and they showed a little less satisfaction regarding the Internet connection (Figure 4). This could imply that even if some colleagues tell a student about some advantages and satisfaction using Wiki methodology, they only become satisfied with it when they try it themselves and have a good experience with it (and not just due to comments from the teacher or their colleagues about it).

Figure 3. Satisfaction of students about Wiki methodology. Case of subject "AIA 0405

Figure 4. Satisfaction of students regarding the Internet connection. Case of subject "AIA 0405

Students that worked with Wiki thought the the methodology they used served them to increase their final quality in the work and document, and the ones who used traditional methodology didn't think that their work methodology helped them as much as their colleagues with Wiki methodology thought (Figure 5).

Likewise, all students thought that Wiki methodology facilitated that they could interact with the parts of the document led by other members of the work group, and that the other ones could interact with theirs. This can be deduced from that students with Wiki methodology had the feeling that it allowed them to improve their interaction (Figure 6), and the others declared that traditional methodology, with which they worked, didn't facilitate this expected (and pursued by lecturers as a goal from the subject objectives) interaction among students.

Figure 5. Perception from students that the methodology they used improved final quality of the work. Case of subject "AIA 0405"

Figure 6. Perception from students that the methodology they used improved interaction among the members of the work group. Case of subject "AIA 0405"

Time invested by items

It could be analyzed only in four subjects (AIA-0304, AIA-0405, EAUB and SIMA). Some of the items were specific of each subject and their comparison has no real meaning in practice for all subjects, since the survey, common to all, contained questions about items that were not used at all in some subjects, but were in others.

Thus, we highlight (in bold) some items out of the 16 proposed (Annex 2), due to its interest for the global comparison:

• T1 = Choosing the topic
• T2 = Searching information
• T3 = Structuring the document information,
• T4 = Writing of the work,
• T5 = Including images in the document
• T6 = Process of giving homogeneity to style and content,
• T7 = Final markup to the document, in web page
• T8 = Final markup of the document, in paper,
• T9 = Preparing oral presentation
• T10 = Exchange of files and materials,
• T11 = visits-interviews with topic experts
• T12 = Journeys,
• T13 = Tutorship in person with teacher/s
• T14 = Tutorship by telematic means with teacher/s
• T15 = Meetings in person of the members of the group, and
• T16 = Telematic communication among the members of the group: phone, wiki, e-mail, chat, etc.

The time devoted to structuring the work (item T3) is not necessarily inferior using Wiki methodology respect to traditional methodology, even though there was an exception (AIA 0405).

The time devoted to writing the work (item T4) has been inferior with Wiki methodology. This is a positive result of support to the Wiki tool and methodology.

Another aspect that we wanted to explore was if the Wiki methodology favored the process of giving homogeneity to the markup, style and content of the work (item T6). Students from AIA and SIMA subjects that worked with Wiki methodology dedicated less time to giving homogeneity than those with traditional methodology, except for the EAUB subject, having in all cases similar presentation quality. It remains unclear if these differences in results are only a consequence of the type of methodology or a consequence of the different interest and commitment of each working group in the matter of work.

In all experiences, Wiki methodology saved, in average for the whole group, about 50% of the time invested on final markup of the work (item T8), compared to students working with traditional methodology. But the teaching staff felt that the single students that made this markup did not have to invest less time through Wiki methodology than the ones with traditional methodology. Suitable data was not collected to verify this hypothesis.

In general, Wiki methodology enables a time saving in the exchange of files and materials (item T10) of about 50%.

Regarding the time dedicated to journeys (T12), even though it seemed that the Wiki methodology saved time in general, results were not conclusive since the time for journeys was tied to the specific characteristics of each work (there were works where longer journeys were required for interviewing experts, and on this matter, Wiki methodology was not able to bring savings).

The time devoted to meetings in person (T15) was in global reduced with Wiki methodology in a clear way, even though in a single case the results were inverted (EAUB).

"AIA 0405" specific case study

On the one hand, students that used Wiki methodology invested less time in (Figure 7):

• choosing the topic (item 1 or T1),
• structuring the document information (T3),
• homogenization of the single discourse of the document (T6),
• exchange of files and other material (T10), and
• meeting in person of the work group members (T15).

On the other hand, Wiki methodology meant a major inversion in time, respect to traditional methodology, in:

• including images in the document (T5, although there were very little students that included images in the document, in both work methodologies).
  This result was expected since we there was a misconfiguration of TikiWiki in our specific hosting provider, which was solved long ago, but after the experiences were carried with students in the framework of the project reported here (course 2004-2005). By then, students had to upload images not directly to the single page were they were going to use them (as explained in De Pedro 2006, which is probably less time consuming than including them in MS Word documents), but they had to upload them to "Image Galleries" feature, search them there, copy their link, and go back to the text document to include them there by manually adding some wiki code to reference that image in that position. This longer and more manual process obviously meant longer times to include images, and also some disagreement from students (and teachers), of course, which had to learn the process to include them in the wiki page already. Many students didn't include images in the wiki pages, but waited to have the final OOo Writer or MS Word document to include them there, at that final stage of the group work, just before printing it in paper.

Tutorship (T13, T14) is not commented here but in a next section (below).

Tutorship

Students who had worked with Wiki methodology invested more hours of tutorship in person in all cases with data ("AIA0405" and "SIMA"; both with the same teacher) than their colleagues with traditional methodology. They had to do some extra meetings inside class hours for showing them the basic functioning of the Wiki technology and solving doubts, since only between 5 % and 10 % of the those students had done the specific extra course (convertible as "free choice" credits) on Wiki technology offered. Looking at data from these specific students who attended the extra course, the time they invested in tutorship in person was much lower than the average (data not shown).

"AIA 0405" specific case study
Time dedicated to tutorship in person was higher in students with Wiki methodology (Figure 8), whereas it was the opposite when tutorship was requested through telematic means (e-mail. forums, internal Tiki messaging, etc.) was much lower in students with Wiki methodology than in those who used Traditional methodology, for the most relevant case ("AIA 0405"), where students knew already e-mail, web browsing, searching information in computers, etc (Figure 9).

The different tutorship in person might be referred to their biggest need to consult the teacher the topics what are bring written in the Wiki or discussed in the electronic forums of the subject. Whereas students from traditional methodology would already speak more in person among the different members of their work group, and therefore, they would need less to go to the teacher to consult basic things.

On the other hand, the pupils who worked with Wiki methodology would have needed less time of tutorship through telematic means, since the Wiki would already allow to ask, to comment or even to discuss them all about specific things of their work or document without having to meet in person.

Figure 8. Time invested in "in person" tutorship. Case of subject "AIA 0405"

Figure 9. Time invested in tutorship through telematic means. Case of subject "AIA 045"

Not all students who worked with Wiki methodology were satisfied with the contributed support by their teacher to do the work (in the frame of the assignment in the practical classes of AIA subject), whereas those who used the Traditional methodology they did remained very satisfied (Figure 10). It can be deduced from here that, being the tasks and intended support from the teacher the same towards all groups of students (regardless of the work methodology they used), the difference had to be specifically on the work methodology based on Wiki. It has to be stated, also, that only four out of 31 students who used the Wiki joined previously the extraordinary course of 10 hours (convertible as "free choice" credits) on collaborative writing using Wiki. The rest received little more than two hours of formation / training in Wiki usage, distributed along the first two meetings in person with students within the practical classes of Evaluation of Environmental Impact. This way, it would be expected as logical that students who worked with Wiki methodology should remain with the sensation that it would have been more necessary for them to have had increased methodological support (because the support about the specific content from AIA subject was them same for all groups of students).



Consequently, it becomes important that fact that, students who have to use Wiki methodology, could previously receive a specific course to get used to Wiki methodology (no only to Wiki technology) through a slow, gradual and comfortable way. This way, they will be able to extract the maximum profit out of their work with the minimum disorientation due to the collaborative work methodology based on Wiki, whenever they have to use it at production level in real situations (other subjects at university or in their professional jobs out of university later on).

Conclusions

• Wiki methodology in big groups (15 students) showed a clear enhancement in quality (higher marks) with less overall time devoted to finish the work.

• Results for smaller groups were not clear, and performance using Wiki methodology seems to depend on other factors, which might vary among:
  • previous training using the Wiki technology,
  • motivation of the teacher to convince the students to give the new methodology a try
  • difficulties to attend meeting in person to discuss or write together the document.

• Students using Wiki methodology devoted less time than the ones using traditional methodology in the items: "writing the work", "exchanging files and other material", and in some cases, in "structuring" and "homogenisation of the style, markup, and content" of the work, as well as in "meetings in person".

• Students using Wiki methodology were more satisfied with the methodology for writing documents collaboratively than the ones with traditional methodology. those results came from the subject where bigger and more balanced data sets could be gathered to analyze answers from their surveys (AIA 0405).

• It is important that students who have to use Wiki methodology could previously receive a specific course to get used to Wiki methodology (no only to Wiki technology) through a slow, gradual and comfortable way, in order to extract the maximum profit out of their work whenever they have to use it at production level.

Acknowledgements

1. AGAUR-DURSI (2004 - 2005): Ref. 2003MQD00167
2. UB-REDICE (2005-2006)
3. Ourproject.org server and admin vjrj
4. Library of Biology Faculty (UB)
5. Tikiwiki.org community

iv. Bibliography

• De Pedro, X., 2004. "Dedicación temporal del alumnado a trabajos en grupo usando Wikis en entornos libres de aprendizaje colaborativo u Ofimática Web: análisis de un caso en ciencias experimentales". EDUTEC 2004, Educar con la Tecnologías: de lo excepcional a lo cotidiano: 6. (document: http://uniwiki.ourproject.org/dl70 )

• De Pedro, X., Rieradevall, M., López, P., Sant, D., Piñol, J., Núñez, Ll., Llobera, M. 2005. Disseny, implementació i avaluació d'experiències de treball col·laboratiu en la docència d'assignatures de ciències experimentals. Informe final (AGAUR, Ref. 2003MQD00167). (document: http://uniwiki.ourproject.org/ifuw )

• De Pedro, X., Rieradevall, M., López, P., Sant, D., Piñol, J., Núñez, Ll., Llobera, M. 2006. "Writing documents collaboratively in Higher education using Traditional vs. Wiki methodology (I): QUALITATIVE results from a 2-year project study". "4º Congreso Internacional de Docencia Universitaria e Innovación (IV CIDUI)."

• De Pedro, X. & Reyes, J. 2006. “TikiWiki en contextos educativos (II): los sistemas de evaluación de los aprendizajes”. V Jornadas de Software Libre, Universidad Politécnica de Cataluña (http://www.jornadespl.org; full text: http://gclub.ub.es/dl53).

• Dumais, M., 2004. "Technologie: Le Wiki, le tableau blanc de l'intelligence collective", en Le Devoir.com. (http://www.ledevoir.com), http://www.ledevoir.com/2004/05/03/53624.html

• Fernández, A., 2004. "Los wikis se extienden en Internet", in IBLNEWS, http://iblnews.com/news/noticia.php3?id=101774

• Goodnoe, E., 2005. "Wikis Make Collaboration Easier", a InternetWeek (http://www.informationweek.com), Aug. 29, 2005. http://www.informationweek.com/story/showArticle.jhtml?articleID=170100392&pgno=1

v. Annexes

Annex 1

Survey included here as pdf (4 pages - 150 Kb): http://uniwiki.ourproject.org/dl91

Annex 2

Table of self-recording time invested by students included here as pdf (1 page - 114 Kb): http://uniwiki.ourproject.org/dl92