Computing integrated activities scored for programming concepts
Data files
Jun 19, 2024 version files 42.78 KB
-
Integrated_Computing_Concepts.xlsx
22.64 KB
-
README.md
20.14 KB
Abstract
Educators across disciplines are implementing lessons and activities that integrate computing concepts into their curriculum to broaden participation in computing. Out of myriad important introductory computing skills, it is unknown which—and to what extent—these concepts are included in these integrated experiences, especially when compared to concepts commonly taught in introductory computer science courses. Thus, it is unclear how integrated computing activities serve the goal of broadening participation in computing. To address this deficit, we compiled a database of 81 integrated computing activities, constructed a framework of fundamental programming concepts, and scored each activity in the database for the presence of each concept. The dataset also includes different activity features, including discipline, programming language, student age, and duration of activity.
https://doi.org/10.5061/dryad.k0p2ngfgj
Associated journal article: Margulieux, L. E., Parker, M. C., Uzun, G. C., & Cohen, J. D. (2023). Levels of computing concepts used in computing integration activities across disciplines. Journal of Technology and Teacher Education, 31(2), 167-202. Waynesville, NC USA: Society for Information Technology & Teacher Education. https://www.learntechlib.org/p/221815/
Please contact Lauren Margulieux, lmargulieux@gsu.edu, for additional information about this dataset.
The listed computing integrated activities were scored for which programming concepts they included. A blank cell means the activity did not include the concept, and a 1 means it did. Activities were scored binarily, either they included the concept or not. The number of times the concept is used within an activity was not counted to avoid uneven representation across shorter or longer activities.
Initial Programming Concept Framework
Integrated computing activities are often theoretically framed in a CT framework, especially for science and math activities for which national standards (e.g., the Next Generation Science Standards and Common Core) explicitly include CT. Thus, we explored CT frameworks for the programming concepts that they include (Aho, 2012; Armoni, 2016; Barr & Stephenson, 2011; Brennan & Resnick, 2012; Denner et al., 2012; Denning, 2017; Grover & Pea, 2013; Lye & Koh, 2014; Palts & Pedaste, 2020; Tang et al., 2020; Weintrop et al., 2016; Wing, 2010; Yadav et al., 2014). As Tang et al. (2020) highlight in their systematic review of CT frameworks aimed at assessment, many of the frameworks focus on problem-solving competencies rather than programming concepts. The main exception to this trend is Brennan and Resnick’s (2012) framework, which includes a category for computational concepts that lists sequences, parallelism, loops, events, conditionals, operators, and data. We used these concepts as the initial foundation for our scoring scheme. In addition, Weintrop et al.’s (2016) definition of CT for math and science integration includes specific data practices, such as creating, manipulating, and visualizing data. We transformed these practices into programming practices, such as inputting variable values, using operators to calculate variables, and various visualization concepts.
To supplement the CT frameworks, we also considered how different block-based programs grouped blocks into menus. These groupings were recently analyzed in Lin and Weintrop’s (2021) landscape report of block-based programming languages. Their analysis of 36 languages grouped blocks based on concepts. The conceptual groupings included: variables/data structures, logic operators, sprite/character appearance, operators (numerical, textual, color), customize block (i.e., function), movement, sound, sense/input, interacting with the physical/virtual environment, output, debug, comment, and extension. These categories were added to our initial conceptual framework.
From this theoretical basis, we began an iterative scoring of computing integration activities. Our goal was to refine the framework based on concepts that were missing and differentiate large categories into more specific concepts, such as separating the concept of loops into for and while loops.
Revisions to Programming Concept Framework
Our full, revised programming concept framework can be found in Table 1, with definitions and examples for each concept that we considered when analyzing the integrated computing activities. When considering each concept definition, it is important to differentiate the role of the programmer and of the user, which can often be the same person in integrated activities. The programmer is the person creating the program by interacting with the blocks, such as initializing a variable. The user is the person interacting with the program as it runs, such as responding to prompts to change the value of variables. More details about the evolution of the framework can be found in Margulieux, L. E., Parker, M. C., Uzun, G. C., & Cohen, J. D. (2023). Levels of computing concepts used in computing integration activities across disciplines. Journal of Technology and Teacher Education, 31(2), 167-202. Waynesville, NC USA: Society for Information Technology & Teacher Education. https://www.learntechlib.org/p/221815/
Table 1
Definitions of programming concepts used in integrated computing activities with examples
| Concepts | Definitions | Examples |
|---|---|---|
| **Algorithms **– nature of the program | ||
| Sequence | The order of blocks was critical to correct execution | Defining the variables before using them to calculate other variables |
| Parallelism | Used multiple code sequences in the program | Multiple sprites programmed with their own behaviors |
| **Variables (and Data Structures) **– how data is stored or handled | ||
| Variable | Used a variable | Assigned a string to the variable “adjective” and used that variable later in the program |
| sub-concept: +=/change | Used a += or change block to calculate a new value for a variable | Increase the value of variables to visualize the Fibonacci sequence |
| sub-concept: calculated variable | Used arithmetic operators or the value of other variables to calculate the value of the variable | Subtract the value of other angles from 180 to find the value of the last angle in a triangle |
| List | Used a list to assign multiple values to a variable | Used a list to provide multiple options for the variable “adjective” |
| Operators – how data is processed or analyzed | ||
| Arithmetic | Used +, -, *, /, or other operators to calculate numerical values | Divide a wavelength in half to draw the first half of the wave |
| Relational | Used <, <=, >, >=, or == | Compare the value of different fractions |
| Boolean/Logical | Used and, or, not, true, false, or other logical operators | Determine the quadrant of a graph based on the x- and y-axis value |
| **Loops – **how code is repeated | ||
| For loop | Used a loop that repeated for a specific number of times entered by the user or a forever loop that repeated infinitely | A program that repeats a sequence multiple times to draw a symmetrical shape |
| Sub-concept: loop index | Used an index to count or control how many times to loop | An index within a loop that counts the number of times a loop has run |
| While loop | Used a loop with a termination condition | Loops until a variable becomes negative |
| sub-concept: nested loop | Used a loop within another structure, like a loop | Repeat a drawing sequence within a larger repeated sequence to draw symmetrical shapes |
| **Conditionals – **how the program makes decisions | ||
| If-then | Used a statement to determine whether a condition was met to determine whether code should be executed | Compare a variable to a predetermined value to determine whether to display a message |
| If-else | Used a series of statements to determine which matched a given condition | Compare a variable to a predetermined value to determine which message to display |
| sub-concept: nested conditional | Used a conditional within another structure, like a loop | Repeat a conditional statement for the changing value of a variable within a loop |
| Functions – chunks of code that are easy to reuse | ||
| Define/call | Called a function defined by the user | Define a function that draws a shape |
| Parameter | Included a parameter as part of the function definition | Define a function that draws and shape using a parameter for the size of the shape |
| Visualization – how components move around the screen | ||
| Movement | Moved sprites or objects to visualize a process | Characters in a story move throughout the story |
| sub-concept: pen up/down | Used the pen up or pen down blocks | Using pen up and down to differentiate between different shapes |
| Cartesian | Used Cartesian coordinates to determine the location of a sprite or object | Start a character in the bottom left corner by setting their location to negative x and y values |
| Angles | Changed the direction of sprites or objects using angles | Draw a triangle based on the angles of vertices |
| **Components – **audiovisual components to be visualized | ||
| Multimedia | Added a multimedia component to the program such as a sprite, sound, pen, or stamp | Add two characters and a background to a digital story |
| sub-concept: button | Added a button for the user to click to trigger an event | Add a button that says “Click here when ready” |
| sub-concept: counter | Added a counter for the number of times an event has occurred | Count how many times a sprite has touched another sprite |
| Properties | Changed the default properties | Change the color of a pen |
| **Output – **the program communicating with the user | ||
| String | Displayed a string entered by the programmer | A character is programmed to say a line |
| Variable | Displayed the value of a variable, either string or numeric | The program prints the final value of a variable |
| String and Variable | Displayed a string and value of a variable in the same output statement | A character says “And you picked”, + a variable selected by the user |
| **Input – **the user communicating with the program | ||
| Event | Triggered the execution of code based on the action of the user, sensor, or internal feature | A character in a story won’t talk until the user clicks them or a message is broadcasted |
| Variable: String | The user entered a string into the program | A poem generator asks for a verb |
| Variable: Numerical | The programmer entered a starting variable value or asks the user for input | The user picks a number |
| Cleaning/transforming | Used a cleaning or transforming process on user-inputted data | Change all text to lowercase |
Scoring Activities for Concepts
Activities were scored by two of three raters for whether they included a concept. We decided to score activities for whether they included a concept rather than the number of times a concept is used. Our goal was to determine which concepts are more commonly used across computing integration activities, and scoring for the number of times a concept was used in an activity might have skewed the results based on incidental features of the activities. For example, if an activity used five variables, the conceptual knowledge required is not necessarily different than if an activity used two variables. Thus, scoring for times a concept was used was more likely to result in over-representation of the concept rather than information about which concepts were most commonly used in computing integration.
The raters were three CS education researchers. Two have a Ph.D. in areas related to CS education, one from educational psychology and one from CS and have worked and published in computing education for at least five years. The other has a Master’s degree in CS and is earning a Ph.D. in instructional technology with an emphasis on CS education. Each activity was scored by at least one rater with a degree in education and at least one rater with a degree in computer science.
To determine interrater reliability, we used Cohen’s Kappa. Cohen’s Kappa is appropriate to compare binary data from two raters, such as whether a concept is used in an activity. Cohen’s Kappa compares the observed agreement, in this case 96%, to the chance of agreement, calculated from the base rate of each binary option from each rater. For example, if rater one marked 80% of concepts as present in each activity on average while rater two marked 20% of concepts as present, then the chance of agreement would be low. A higher Kappa indicates higher agreement, with 0.81 to 1.00 indicating nearly absolute agreement (McHugh, 2012). For the current study, Kappa was 0.92, indicating nearly absolute agreement.
Selection Criteria: Features and Limitations
Non-CS Disciplinary Learning Objectives
The first selection criterion for activities to include in the analysis was the inclusion of learning objectives in a discipline other than computing. No restrictions were placed on which other disciplines qualified, and we found activities from language arts, math, science, art, music, foreign language, history, social studies, and even spatial skill development for young children. One indirect benefit of requiring non-computing disciplinary learning objectives was that many included activities have substantive lesson plans. These lesson plans make the activities more accessible to teachers by including TPACK-related information, such as disciplinary learning objectives for the activity. As a result, the authors recognize the limitations of requiring non-computing learning objectives but also that it provides a level of authenticity and accessibility for the included activities.
One of the major sources of computing integration activities affected by this requirement was the ScratchEd website. Scratch is a popular language for computing integration activities, aided by an extensive repository of student- and teacher-created projects that users are encouraged to remix into their own projects. The thousands of programs in this repository are of widely varying complexity and quality, and most of them are listed with a topic but without explicit learning objectives. To draw from this wealth of activities without comprehensively including projects, we identified lists of vetted computing integrated activities using Scratch to include in the analysis. These lists were "Integrated Scratch Programming in the Curriculum," "Scratch Projects Across the Curriculum," "From Music to Math: Scratch Across Every Subject," and "Scratch Cross-Curricular Integration Guide." Similarly, resources related to the Snap! language had plentiful examples of projects across disciplines with limited explicit non-CS disciplinary learning objectives.
Block-Based Programming Languages
Because computing integration activities are becoming popular, an initial search revealed too many activities to score in one analysis. To narrow the scope of the analysis, the next selection criterion was that the activity had to use a block-based programming language. This criterion has benefits and limitations. One of the main benefits for the goal of the current analysis was that block-based activities include a range of concepts, regardless of their syntactic or semantic difficulty (Grover & Basu, 2017; Papadakis et al., 2014). This benefit means that concepts that best serve the activity can be included for learners with little to no programming experience (Weintrop & Wilensky, 2018). The associated limitation, however, was that concepts are also restricted by the blocks that are built into the language. Most popular languages use a low-floor, high-ceiling design that includes blocks for all concepts that would be taught in an introductory programming course, though (Grover, 2021; Weintrop & Wilensky, 2015). Another limitation was that prominent, text-based integration activities, such as Bootstrap’s curricula in Algebra and Physics, are excluded.
This selection criterion also notably excluded commonly used science simulation platforms, like NetLogo and PhET. These platforms include a large range of simulations for scientific phenomena and other models beyond science. While the simulations allow users to easily access the source code, the primary interface does not include the program used to create the simulation. In addition, the source code, except for some adapted NetLogo simulations, is text-based. Though the programs are heavily commented to make them understandable, they do not meet the inclusion criteria for the current dataset. More programming-centric and block-based options for scientific simulations, like StarLogo Nova, were included.
Access
Accessibility of the activities was the final criterion for inclusion. Following the accessibility criteria used by Lin and Weintrop (2021), we included activities only if they could be found online, were free of cost, did not require a physical device like robotics toolkits, and were updated recently enough that they ran on current versions of languages and operating systems. The requirement to be found online is not expected to substantially narrow the analysis because Lin and Weintrop found that 90% of block-based programming languages ran in a web browser. Exclusion for use of physical devices is a corollary to the requirement to be free of cost. We felt that these criteria would result in a dataset that had the broadest and most equitable applications because many public schools in low-income areas in the US cannot afford physical computing or robotics kits.
Search Criteria
Users need to recognize that the current dataset was based on a review of computing integration activities but not a systematic review. Unlike systematic literature reviews of scholarly work on a given keyword or topic area, there were no databases of indexed computing integration activities that span our inclusion criteria. Some repositories for certain languages exist, such as ScratchEd’s repository of Scratch projects and the Exploring Computational Thinking repository of Pencil Code and Python activities. However, computing integration activities are not published through a central organization, so they can be difficult to find.
In lieu of a systematic review, we attempted to build a database that represented activities from a variety of disciplines, student ages, designers, and languages. To create this database, we included any activities that we were already aware of, such as Action Fractions, links from lists of computing integration activities, such as "Scratch Projects Across the Curriculum," links from CSforAll’s curriculum directory, and a general Google search for "‘integrated computing’ activities" and "‘computational thinking’ + programming" or "‘computational thinking’ + coding." We examined the first 100 returns for these searches. However, many of the activities found through Google search were excluded based on our criteria, primarily for not including non-CS learning objectives.
We included activities as whole units, whether they were single-class lessons or extended curricular units that included multiple lessons, like Coding as Another Language. Treating individual lessons from curricular units as individual activities would have created an over-representation of extended units (e.g., 72 lessons for the Kindergarten, 1st, and 2nd grade curricular units from Coding as Another Language instead of 3 activities). Our database included 81 activities from the following sources:
• CANON Lab
• Code.org’s CS Connections
• Code.org’s Hour of Code
• Coding as Another Language curriculum
• CS+ units from University of California San Diego
• CSforALL’s Curriculum Repository (including 144 curricular units at the time of searching)
• CT4Edu
• Everyday Computing
• Exploring Computational Thinking
• Google search
• Google’s CS First
• Integrated computing activities from Georgia State University
• Project GUTS
• ScratchEd
• The Tech Interactive
• TVO Learn
• UCL Scratch Maths
We analyzed the distribution of these activities’ characteristics based on primary discipline, student age, programming language, and minimum time to complete. Based on discipline, we recognized that we had only two from history or social studies and searched for additional activities. While we found many projects on ScratchEd’s website, they did not meet the selection criteria. Required courses, including Language Arts, Math, and Science had a sufficient number of activities, matching their representation in the school day. We also had a wide range of activities based on student age and minimum time to complete, so we did not search for any additional activities based on these characteristics.
To explore the representation in our database based on programming languages, we used the categories identified by Lin and Weintrop (2021) to ensure coverage of different types of block-based languages. The database has activities from Pencil Code (i.e., block-based implementation of a text-based language), Scratch (i.e., multimedia focused on animations and storytelling), AppLab (i.e., mobile app development), StarLogo Nova (i.e., simulations), and ScratchJr (i.e., pre-reading language). We decided against requiring languages from Lin and Weintrop’s other categories for data science, physical computing, and task-specific languages because they did not match our inclusion criteria. We explored other common languages to include, like Alice, Snap!, and App Inventor, but we did not find activities that matched our criteria.