Monte Carlo Analysis for Single-Subject Experimental Designs
Import Data
The app requires that the data be in a "Long Format." This means that the app expects only one measure of responding per row. Other factors, such as subject number or condition, should be repeated for each row of data. Load the example data to see the expected format.
The app also expects a file that includes columns for subject/participant, session numbers, and respoding. If you do not need a subject/participant identifier, just add a dummy code to your data file. Session number is necessary to plot your data, if your data uses something other than sessions (i.e., date), you must recode that into session numbers.
Finally, the data must be in a comma-seperated file format (.csv).
Example Data
Click the "Example Button" below to reset to table back to the included example data.
Note: After you load your data or use the example data set, the data will appear on the next tab (View & Modify)
Data Table
It is possible to modify the data displayed above, but it is not recommended. If you modify values on the data table, click the "Update" button to make sure the new data is loaded into to the app.
Select Columns
In the drop menus below, you must select the columns that are for "responding"\behavioral data, the session indicator, and subject\participant identifier. The menus will only function once data has been loaded into the app.
log Proportion Responding
If you are planning on using a Monte Carlo simulation to examine either transient changes in behavior (sugh as a probe desgin) or whether there is a trend in the data, then we recommend you consider using log proportion responding.
log Proportion responding as a measure is great for detecting changes in behavior. It provides you with a measure of behavior on session B as a proportion of session A. By looking at the logarithm of the proportion, increases in behavior have the same impact on our measure as decreases in behavior. For example, the standard celeration chart uses logrithms on the y-axis. See Friedel et al. (2019) for a full description of log proportion responding.
The app separates the calcualtion of log proportion responding from the Monte Carlo simulation. You can use the app to calculate log proportion responding, conduct a Monte Carlo analysis, or do both. For that reason, if you want to calculate log proportion responding and use it in the Monte Carlo analysis then you must go back to the "View & Modify " tab and change the "Responding column" to the newly calculated "log prop. resp." column.
Warning: If you have more than one measure of behavior per session (such as Behavior A and Behavior B for each session) then the app will not calculate log proportion responding correctly. We recommend you calcluate the measure outside of the app and then upload the data.
Click the following button to calculate the natural logarithm (base e).
Sample Selection
The app can not inherently distinguish between the "real" sample and the rest of the data. On this panel, you must use the inputs to filter the data so that only "real" sample is highlighted. If you have selected the column indicators on the "View & Modify" panel then a simple figure will be displayed below. The data you have selected for including in the "real" sample will be highlighted as you update the filters.
Select a grouping factor if you want different Monte Carlo analyses based on that grouping. If no grouping factor is selected, then there will be only one sample. For example, with the Example Data you may wish to have a different Monte Carlo Analysis for each "Group". Thus, by selecting "Group" as a grouping factor you will get a seperate Monte Carlo analysis for each unique group in the data set and the results of each analysis will be independent of one another.
Note: The sample selection (to the right) can be difficult to use. If you are struggling to highlight the proper sample using this tool, then consider including an extra column in your data that indicates what data should be in the sample of interest. If you upload the updated data file then you can just use the new column to select/filter your data.
Use the boxes below to select filters for the data you want included in your "real" sample. When completed, click he "Update Filter" button below, which will highlight the selected data in the plot.
Run Monte Carlo
To run the Monte Carlo simulation, you must:
- Upload data on the "Import" tab.
- Indicate the columns that hold behavior, sessions, and participant/subject numbers on the "View & Modify" tab.
- Select a filter on the "Sample Selection" tab.
After you run a simulation, a plot will be displayed below with different panels for each group (if there were any groups). The red line on the panel displays the mean(s) for the data the "real" data that was selected based on your filter. The gray bars are a histogram of the samples of data that were simulated by the Monte Carlo.
The "Value for randomization" box below displays a value that is used to produce the random samples for the Monte Carlo Simulation. You can use the value that was supplied when the app was loaded. If you use the same value, the Monte Carlo will give you the same result. Therefore, if you select your own value or re-use a previous value you can replicate your Monte Carlo outcome.
A basic summary table will be displayed to show that the script is complete. On the "MC results" tab, you will find a figure and on the "Downloads" tab a method to export the data.
Monte Carlo Results
Figure
When you have completed the Monte Carlo analysis a figure will be displayed (by groups, if groups were specified). The figure(s) will show a histogram of the means for each of the 1,000 samples that was simulated by the Monte Carlo Analysis. The color for each bar are only included so that you can easily distinguish adjacent bars on the histogram because the figure may appear small.
The dashed black line shows you the mean of your "real" data. If the dashed line is in the center of the distribution, then that indicates that the Monte Carlo analysis could easily simulate data that looks similar to your "real" data. In typical statistical langauge, your "real" data are not statistically significant.
If the dashed line is at the extreme end of the distributions, then that indicates that the Monte Carlo analysis was not able to reliably simulate data that looks similar to your "real" data. In typical statistical langauge, your "real" data are statistically significant.
Download MC Files
Note: the default name of the download files will contain the value used for randomization (labeled as RV in file name) so that you can replicate your results.
Click the "Download Figure" button if you would like a copy of the histogram on the "MC Results" panel.
Download FigureClick the "Download Data" figure if you would like to download a full copy of the means of each simulated sample. Due to limitations of space on the server you can only download the means of the samples. The specific values that were randomly selected by the Monte Carlo for each sample are not saved.
Download Monte Carlo dataProgram Information
Monte Carlo Analysis for Single-Subject Experimental Designs
Jonathan E. Friedel, Ph.D.
https://orcid.org/0000-0002-1516-330X
Links
Files archived at the time of manuscript publicationGitHub repository
References
Friedel, J.E., Galizio, A., Berry, M.S., Sweeney, M.M., & Odum, A.L. (2019). An alternative approach to relapse analysis: Using Monte Carlo methods and proportional rates of response. Journal of the Experimental Analysis of Behavior, 111 (2), 289-308. https://doi.org/10.1002/jeab.489