catSurv online so that any web-enabled survey software can query our server in order to administer an adaptive inventory (see our published work using computerized adaptive testing in political science here and here).
It depends. In theory, anyone well-versed in web development should be able to fully integrate
catSurv operations into the operations of a given survey software. But because most researchers are not trained in web development, we’ve worked out how to interface with catSurv.com when using the Qualtrics survey software.
We’ve created a catSurv Google Group forum to ask questions—to us and to the computerized adaptive testing community—relating to all things
catSurv and adaptive survey implementation. We encourage you to post questions or problems that arise when implementing your adaptive inventory to the Google Group, but we ask you to please report any
catSurv software bugs as a GitHub issue.
First, you’ll need to do some work in R using
catSurv to set up your adaptive inventory. Make sure the latest version of
catSurv (v1.2.0) is installed and loaded into R.
In what follows, we assume some working knowledge about the
catSurv package. See our package documentation and the supplementary materials of our paper for detailed explanations and examples of the
catSurv software and its functionality. Here, we’ll simply go over a few important steps necessary to interface with catSurv.com with your own adaptive inventory.
For this example, we use the 20-item agreeableness battery from the 100-item IPIP representation of Costa and McCrae’s Five Factor Model. We’ve fit a graded response model (GRM) with 774,410 response profiles collected by the myPersonality Project and 1500 response profiles collected by YouGov in July 2018 for the full 20-item battery. See
?grmCat for more information about item parameter estimation. Our goal is to administer a 4-item adaptive inventory.
While we cannot share all of this raw sample data, the estimated item parameters for the model are stored in a
Cat object available in the
catSurv package. In addition to item parameter estimates, the
Cat object stores all research-chosen features of the adaptive algorithm such as the statistical routine for item selection. In sum, the
Cat object needs to contain all information governing and used by the adaptive algorithm. We have chosen defaults for all options
catSurv makes available to the researcher, but researchers may want to customize their algorithm. See
?'Cat-class' for a detailed description of all of the customizable features stored in the
Cat object can be loaded into your R environment with the
data() function. See
?agree_cat for more details about this battery and item parameter estimation.
data(agree_cat) agree_cat@discrimination # GRM discrimination parameters
## q86 q6 q66 q46 q36 q26 q56 ## 1.3963428 1.3325832 1.8201505 1.2308572 0.9626278 1.2288147 1.1692755 ## q76 q13 q96 q82 q9 q22 q32 ## 1.2768165 0.8477620 1.3225958 -0.9240555 -1.6871667 -0.9958325 -1.2405438 ## q92 q42 q52 q62 q2 q72 ## -1.7112276 -0.9724382 -1.0603658 -1.0066092 -1.0676238 -0.9820362
agree_cat@selection # minimum expected posterior variance item selection
##  "EPV"
Cat object in hand, you now need to set up your Qualtrics survey. The following six steps outline the basic procedure. We discuss each step in detail below.
Catobject into the Qualtrics survey as “embedded data.”
catSurvto clean and obtain the answer profiles.