Susan Van Gundy | February 6, 2012  •  In reflecting on developments in the U.S. education system over the past year, technology has remained at the center of the reform-oriented zeitgeist, promising to make learning experiences more accessible, more personal, and more relevant. Education is, by its nature, a socially constructed process. Yet the robust feedback loops that exist on campuses and in classrooms among educators, learners, and instructional content have not yet extended to online environments at the scale that we envisioned.

Susan Van Gundy is deputy director and co–principal investigator of the National Science Digital Library, which is hosted at UCP. (©UCAR. Photo by Carlye Calvin.)

Despite years of applying Web 2.0 approaches in education, technology-enhanced academic interactions have generally remained within closed institutional platforms. Online content is still largely restricted through publisher licensing, and data describing the circulation of ideas and resources are generally not shared beyond the platform owners. The feedback data exist, but are not networked. In contrast, our personal lives are infused by Web 3.0 social data hyperlinked among people, content, groups, ideas, and information that dominate our decision making for shopping, dining, entertainment, and travel. If we can better enable this scale of open information flow for education, it can have significant potential to transform teaching and learning.

The National Science Digital Library team at UCAR has been working on more open approaches to connecting the dots in education, with new data protocols and exchange systems for sharing usage and utility data about online educational content. Just as smart electrical grids monitor and adjust flow through complex multiscale systems, NSDL is tapping the emergence of socially connected educational communities to enable smart systems for digital learning resources that can reflect production, storage, transmission, and consumption activities across diverse networks.

For more than ten years, NSDL has been active in the aggregation, contextualization, and dissemination of open digital learning content generated through grants from NSF and other federal agencies as well as produced by nonprofits such as museums, research labs, and professional societies. As with many new technologies, NSDL’s early efforts as a digital library research-and-development project emphasized the technology as an end product. As we moved into production, we quickly transitioned to a focus on the content and how best to curate resources for effective educational use and reuse. Along the way, as we made improvements on both the technology and the content, we built the most valuable contributions of NSDL to the science, technology, education, and mathematics community—a network of education, research, technology, and policy partners and a carefully cultivated culture of collaboration. It is through the shared knowledge of this particular connected community that the concept of learning resource paradata (see below) has emerged as a mechanism to make learning networks even smarter.

From its inception, NSDL was tasked to demonstrate the impact of digital libraries on instructional practice and, if at all possible, on student outcomes. This proved to be challenging in the highly heterogeneous, highly distributed, and relatively anonymous usage environment of the open Web. Outside of a few controlled research studies, online resource providers lacked the proper feedback mechanisms to examine the texture of usage at multiple scales. Evaluation methodologies borrowed from conventional library practices, or those used to assess the efficacy of formally adopted curricular materials, did not effectively represent the authentic use of open learning content by educators on the ground. Traditional Web metrics also left us wanting. What does it mean that 100,000 users came to our homepage? What does it tell us when users click on a resource? Was it useful? Did they incorporate it into their classrooms? Did they adapt it for a different context? How easy was it to implement? What do we need to improve about that resource?

It became clear that we needed to reconceptualize our notions of impact to match the changing realities of educational practice. In response, we have begun to focus on the online educator communities where new practices are nucleating, and where educators are rapidly expanding their consumption and production of expert-generated, peer-generated, and self-authored content. 

Paradata: Describing how information is used

The rapid growth of online communities of practice allows for authentic examination of digital resources as they are being discovered, created, and used. As educators, subject matter experts, and learners increasingly interact around digital content in complex collaboration environments online, we have an opportunity to make this collective knowledge production visible, and to evaluate what is working and what needs are going unmet with regard to the current generation of digital learning resources. This will allow us to meaningfully inform the requirements for the next generation of learning tools. Most importantly, we can now gather that intelligence as it emerges across diverse platforms, aggregate and interpret it, and feed it back into the workflows of user communities as a new layer of data to inform decision making around digital content.

NSDL launched the STEM Exchange as a means to bring content into the hands of educators through online communities, and to establish return feedback loops of data created by the activities of communities around that content—a type of data we have defined as paradata, adapting the term from its application in the social sciences.

The concept of learning resource paradata is grounded in the premise that teachers trust teachers to understand and inform what might be most useful to a particular pedagogical context. Paradata provides a mechanism to openly exchange information about how resources are discovered, assessed for utility, and integrated into the processes of designing learning experiences. Each of the individual and collective actions that are the hallmarks of today’s workflow around digital content—favoriting, foldering, rating, sharing, remixing, embedding, and embellishing—are points of paradata that can serve as indicators about resource utility and emerging practices. This is an idea best served at scale, and it will only realize its potential as it allows online communities to connect with each other and knowledge from real-world applications to accrete as shared content diffuses across networked platforms.

To accelerate that diffusion, and our collective understanding of it, the NSDL STEM Exchange is sharing its underlying concepts and ideas with the U.S. Department of Education and Department of Defense to seed the development of the Learning Registry, a new initiative that is building a common technical infrastructure for exchanging descriptive information and usage data about learning resources from federal agencies, nonprofits, and commercial providers alike. Like other smart systems, the Learning Registry’s capacity for meaningful pattern identification will be enhanced as it scales up. The more that connected communities use the Learning Registry to discover and share content and give back anonymized and aggregated paradata, the greater our collective intelligence will be, and the more our actual needs and demands will drive the future of digital content. 

As we gain the ability to know, not just that a resource exists, but that…

• Faculty at University X integrated Resources Y and Z into a course module tagged as Introduction to Meteorology.
• 189 high school teachers who are registered with PBS Learning Media favorited Animation Z in the month of September.
• Master Teachers in the CTE Online community aligned Podcast A to California State standard B and Math Common Core standard C.
• One community college member on the OER Commons site created slideshow D and assigned it Creative Commons license CC BY-SA 3.0.
• 75 Florida teachers at the CPALMS portal recommend Educational Gaming Site E for use with English Language Learners.

… we may gain insights into the nature of online communities themselves, and the role of online collaborations in redefining the teaching profession for the digital age.

The Learning Registry launched a beta version in November. It has already gathered a vibrant open community of collaborators beyond NSDL, including content providers, learning communities, and technical tool developers. Learn how your knowledge network or online community can participate at the LearningRegistry.org website, and let us know what you think as paradata begins appearing in your own online workflows.

This article is adapted from a November 2011 guest blog post for the U.S. Department of Education / American Institutes for Research, appearing at ConnectedEducators.org.

See all News