The observation protocol has been developed using a combination of design research (Cobb et al. 2003) and grounded theory (Strauss and Corbin, 1998). Because an observation protocol involves assigning categories and numerical values to qualitative data, grounded theory seemed to be a natural choice for developing the categories to be observed. Grounded Theory distinguishes between three types of coding. In open coding, the raw qualitative data is analyzed and organized into natural categories (the open codes) that the researchers use to describe and organize the concepts they observe in the data. In axial coding, the codes themselves are the object of study. Axial coding involves organizing the open codes into a structure, and identifying categories of open codes. Selective (or closed) coding occurs at the point where no new categories are found by the open and axial coding process. At this point, the codes are finalized, a theory is constructed, and closed coding is used primarily to convert qualitative data to quantitative data. Closed coding is a natural fit for an observation protocol that relies on recording observed behaviors in time, so we followed the full procedure for developing open, axial, and closed codes as a way of constructing the protocol.
A design experiment (Cobb et al. 2003) is primarily an approach to theory building, by using cycles of invention, testing, revision in order to refine a theory. In the core design experiment methodology, a theory of learning or teaching suggests a particular intervention. This intervention is then implemented and tested. Unexpected results are used to refine the theory and design new cycles of intervention and testing until some sort of equilibrium is reached. The final product would be an improved theory. We did not use a full design research methodology, because the development of a protocol lacks an educational intervention, and because the final product is an assessment tool, rather than a theory. However the protocol has been developed using a design cycle of invention, testing, and refinement. In keeping with design research methodology, we value each cycle for the contribution to our greater understanding of the problem, and report on all of the cycles in the results.
