Reliability for paste rubbing after caustic paste application
Data files
Sep 15, 2026 version files 14.95 MB
-
PasteRubbingComparison_GoldandScorer_Intern1_260614.xlsx
8.68 KB
-
PasteRubbingReliabilityDiscrepancies_Intern1_260614.xlsx
6.95 KB
-
README.md
4.63 KB
-
ReliabilityFor_PasteRubbing_Analysis.pdf
207.91 KB
-
ReliabilityFor_PasteRubbing_Analysis.Rmd
11.47 KB
-
RubbedPasteScoringTest_Blank_260614.xlsx
9.89 KB
-
RubbedPasteScoringTest_Intern1_260614.xlsx
10.90 KB
-
RubbedPasteScoringTest_key_260614.xlsx
10.90 KB
-
RubbingScoringSystem_260614.pdf
14.68 MB
Abstract
Disbudding, a painful procedure, permanently damages the horn-bud tissue either with a hot iron (thermal) or caustic paste (chemical). The resulting wounds persist for an average of 7-9 wk for hot-iron or 14-18 wk for the caustic paste method. For caustic paste disbudding, the paste is wet for multiple hours as it absorbs into the calf’s tissue. After application, calves may rub the paste onto their pen, housing, group mate, or on themselves, affecting non-target tissue, and creating disturbance to the initial application. To create consistency among our research team when identifying paste disturbance from rubbing, we have developed supporting materials. This Dryad entry contains information for evaluating observer reliability when describing caustic paste disturbance from rubbing in dairy calves. The scoring system and definitions we have used in previous research are included, along with a 60-photo test containing examples of the different types of disturbance in individually housed dairy calves. These include the size of the disturbance of the initial paste application on the head (small or large), if the paste appears on the housing environment or not, and if paste is present on other parts of the body. Examples of undisturbed paste are included in the scoring system and test photos. A video recording describing the scoring system, areas of emphasis within the definition, and potential observation challenges such as how to score imperfect edges when paste is applied to a horn bud with hair are discussed. These resources are intended to be used to test observer reliability before data collection begins, during data collection, or afterwards. Included are: (1) the gold standard key for comparison, (2) a blank, formatted Excel sheet for data entry, and (3) an example response from an intern. Finally, the R code (kappa R package irr, version 0.84.1) used for calculating inter-observer reliability of paste disturbance is presented. The outcomes of the analysis of the gold standard vs. an intern are shown. These data and resources can be used in future research to support reliable scoring among observers.
The photo test, gold standard, and analysis used for calculating inter observer reliability of disturbance from paste rubbing by the calf after caustic paste application are presented.
Description of the data and file structure
- RubbingScoringSystem_260614.pdf; This file presents the paste rubbing system developed for use after caustic paste application in dairy calves. A total of 5 categories (no disturbance, small disturbance, large disturbance, paste on hutch, or paste on body) are presented along with examples and their definitions. The reliability for identifying the presence and absence of each disturbance is evaluated in the RMD.
- PasteRubbingDescriptionVideo.MP4 (Supplemental Info link); This video goes into detailed description of each rubbing category presented in Rubbing scoring system 260614.pdf. Additional tips on how to differentiate the categories of disturbance, areas of emphasis within the definition, and potential challenges such as how to score imperfect edges when paste is applied to a horn bud with hair are discussed.
- PasteRubbingTest.zip (Supplemental Info link); 60 photos presenting examples of rubbing after caustic paste application, which should be scored to calculate inter observer reliability.
- RubbedPasteScoringTest_key_260614.xlsx; This file presents the gold standard used to calculate inter observer reliability for each of the categories scored. "Photo" corresponds to the numbering of the photos in the test, while 1 indicates a rubbing category is present and 0 indicates absence. The definitions presented in Rubbing scoring system 260614.pdf were used to score these photos.
- RubbedPasteScoringTest_Blank_260614.xlsx; This file is a blank sheet which can be downloaded and used to score the presence and absence (1/0) by observers for each of the rubbing categories. Inter observer reliability can then be evaluated by comparing the the observer's score to the gold standard/key provided. The photo column corresponds to the photo number in the test folder. For every photo and tissue type, a 0 or a 1 should be entered based on the presence or absence. If 0s are not entered when categories do not appear, the reliability will not be calculated properly.
- RubbedPasteScoringTest_Intern1_260614.xlsx; This file includes the presence and absence (1/0) by an intern labeled as Intern1 for anonymity, for each of the rubbing categories for which reliability is being evaluated.
- ReliabilityFor_PasteRubbing_Analysis.Rmd; This RMarkdown file uses the Rubbed paste scoring test key 260614.xlsx listed above to calculate the inter observer reliability for 4 rubbing categories presented in Rubbing scoring system 260614.pdf. In this example, the Rubbed paste scoring test Intern1 260614.xlsx is used to calculate reliability for an intern. The Excel file Rubbed paste scoring test Blank 260614.xlsx can be downloaded, filled in, and calculate reliability against our gold standard. Rmd files can be downloaded and run in R. This file is annotated.
- ReliabilityFor_PasteRubbing_Analysis.pdf; This file is the PDF output of the corresponding RMarkdown file, and includes the code and output previews. This file is annotated.
- PasteRubbingReliabilityDiscrepancies_Intern1_260614.xlsx; This file was created in the ReliabilityFor_PasteRubbing_Analysis.Rmd and exported to review disagreement in scoring between the gold standard and Intern1. The photo column shows the number of the photo scored where interns and the gold standard either "match" or have a "discrepancy". We are most interested in reviewing the disagreement, so this Excel specifically focuses on creating a visual for rubbing categories where there is disagreement between the observer and gold standard. If a photo had matches for each of the rubbing categories, it was dropped from this data frame before being exported.
- PasteRubbingComparison_GoldandScorer_Intern1_260614.xlsx; This file was created in the ReliabilityFor_PasteRubbing_Analysis.Rmd and exported to check for disagreement in scoring between the gold standard and Intern 1. It shows the "Photo" number and the 0/1 score for both the Gold standard and Scorer Intern1 in separate columns for each rubbing category presented in Rubbing scoring system 260614.pdf.
Code/Software
All analyses were run in R using RStudio. Software versions, including those for all packages used, are included in ReliabilityFor_PasteRubbing_Analysis.Rmd, and the associated PDF export.