Welcome on Models for missing data

Conditions

Mon, 01 Jan 0001 00:00:00 +0000

Conditions #

Modify the default behavior of pred-type means.

Syntax #

trend conditions:
 Botanist: JA

Usage #

See the object types entry for a more thorough introduction to this topic.

Installation

Mon, 01 Jan 0001 00:00:00 +0000

Installation #

Obtaining a copy of the repository #

New here? #

To clone the repository, open a terminal¹ and run the following command from the directory into which you’d like to place the project (e.g., from ~/repos).

git clone https://github.com/lzachmann/models-for-missing-data.git [DIRNAME]

DIRNAME is optional, and specifies the name of the directory into which the project will be cloned on your local machine. You could call it “m4md” for instance, if you wanted something a bit shorter than the default “models-for-missing-data.”

Likelihood function

Mon, 01 Jan 0001 00:00:00 +0000

Likelihood function #

The likelihood statement specifies the probability distribution that will be used to describe the response variable. The available options depend on your data, so knowing what type of variable you’re working with is essential.

Syntax #

# Likelihood. For counts, one or both of: poisson, negative-binomial.
likelihood:
 - poisson
 - negative-binomial

Usage #

In some cases – a model for soil stability observations – there is only one plausible, implemented likelihood function (ordinal-latent-normal). In other cases – species richness – there might be several available options (poisson, negative-binomial, and their zero-inflated counterparts). In cases where you specify multiple likelihoods, the model API will create a separate analysis for each, and the relative performance of each can be evaluated using model diagnostics, posterior predictive checks, and information criteria.

Mock data walkthrough

Mon, 01 Jan 0001 00:00:00 +0000

Your First Forecast (Mock Data) #

This is the simpliest path to perform a forecast. All data is pre-committed, so there’s nothing to download or prepare. The walkthrough has three steps.

Prerequisites #

A working M4MD repo. [TODO: formalize this]

Step 0: Understand the mock data #

The mock dataset represents a single park unit (ELDO) with two vegetation strata (A and B), three sites per stratum, and three transects per site — surveyed annually from 2000 to 2025. Each transect records how many of 100 sample points hit a plant (y_hits out of n_points = 100). A single climate covariate, precipitation (ppt), declines gradually over the training period and drives the vegetation response.

Model builder

Mon, 01 Jan 0001 00:00:00 +0000

Model builder #

The shell script compile-jags-file.sh builds JAGS model files using several arguments. The arguments – which correspond to the likelihood, the deterministic function, group-level effects parameterization (random intercepts and slopes vs. random intercepts only), the presence or absence of additional covariates and, finally, the path to write to – are supplied via the ‘MODEL’ block in the YAML file for an analysis. For purposes of development or debugging, compile-jags-file.sh can be sourced from the command line with an optional output directory as such:

Non-ignorable missingness

Mon, 01 Jan 0001 00:00:00 +0000

Statistics is basically a missing data problem!

– Little 2013

Nearly all samples – whether by design or by accident – are incomplete. We very rarely make a complete census of all individuals in a population or all sites on a landscape. Sometimes we don’t collect, or can’t collect, complete information for individual samples or measures. For instance, we might know an animal was alive when it was last seen, so we know it survived at least that long, but know nothing about its current status. Or we might have information on the coverage of an invasive species down to a certain patch size, beyond which patches are too small or numerous to survey.

Response data

Mon, 01 Jan 0001 00:00:00 +0000

Response data #

The response data include the observations we are trying to model, as well as columns with identifiers (indices, IDs, datetime strings) for all relevant sampling design information. Elements of the sampling design often seen in long-term monitoring data include the following:

Design element	Examples
observational units	transect, quadrat, plot
sampling units	plot, site
stratification	stratum
date / event times	`MM/DD/YYYY`, `YYYY`

Format #

The response data are stored as flat files. The key characteristic of a flat file is that each row represents a single observation, while the columns describe values associated with the observation and the design features described above. These files are typically text files with no special word processing or markup. The file can be CSV, XLS, XLSX, GZ, or RDS. For ease of use, readability, and other reasons, we generally recommend CSV.

Response info

Mon, 01 Jan 0001 00:00:00 +0000

Response information #

The usual case #

In the usual case, we are working with a single vector of observations (a single column of data in a flat file). Below, we see the first few lines of the data block for an analysis of count-type data from Little Bighorn Battlefield National Monument (LIBI), in Montana (a park within the Rocky Mountain Network). The file, rich.yml, lives in the directory assets/uplands-config/ROMN/LIBI.

Climate futures tutorial

Mon, 01 Jan 0001 00:00:00 +0000

Forecasting with Real-World Data #

What You’ll Need #

Step 1: Prepare Your Covariate and Scenario Data #

Step 2: Fit the Model #

Once your data files are ready, follow the same fit step as the mock walkthrough:

Covariate data

Mon, 01 Jan 0001 00:00:00 +0000

Covariate data #

Covariates are variables that are expected to change with the response variable – they covary in some way with the observations we seek to model. The definition of covariates varies widely online and in the literature. For our purposes, we use the term covariate to describe any variable (whether continuous or discrete) that might influence the mean of the response variable we are interested in. In many cases their effects are of direct interest in the analysis (weather or terrain, for instance). In other cases, a covariate might be “nuisance variable” – a fact or feature that is of no particular interest in itself, but nonetheless might be necessary to build a proper model and develop robust inference. Examples of nuisances include sudden changes in protocol or observers.

Disentangling concepts of status, trend, and trajectory

Mon, 01 Jan 0001 00:00:00 +0000

The terms status and trend are ubiquitous in resource monitoring and management settings. To be useful and robust, however, they require precise (mathematical) definitions. It has been my experience that misunderstanding these terms can lead to misapplication of model predictions and to researchers and managers drawing the wrong conclusions from the data. In this post we show how relatively simple, even intuitive, definitions for each of these terms clarifies their intent, and improves the insights provided by models of monitoring data.

Finite population correction

Mon, 01 Jan 0001 00:00:00 +0000

Finite population correction #

When the size of the sample collected is a significant proportion of the overall population, then we must use a finite population correction, a topic we introduce briefly in another post.

Syntax #

finite population correction: true
finite population info:
 file: assets/uplands-data/ROMN/LIBI_SampledUnsampled_Ratio.csv
 columns with the number of sampled and unsampled sites:
 - '# sampled sites'
 - '# unsampled sites'
 covariate info includes unsampled sites: true

Usage #

Link function

Mon, 01 Jan 0001 00:00:00 +0000

Link function #

The deterministic link function for the regression on the mean ensures that the mean we are trying to model has the appropriate support for the probability distribution from which our observations arise.

deterministic model:
 - linear
 - exponential

Location info

Mon, 01 Jan 0001 00:00:00 +0000

Location info #

Site location information (site location info), if provided, is used to evaluate spatial autocorrelation in model residuals.

site location info:
 file: assets/uplands-data/ROMN/LIBI_GRKO_metrics_adjwt_20170414_forCSP.csv
 coordinate columns:
 - xcoord
 - ycoord

Here again, we point to the name of the file (file) and supply the names of the columns containing the spatial coordinates (coordinate columns). Note that proper residual analysis requires Cartesian coordinates. If the locations come directly from a geographic coordinate system, they’ll need to be transformed to a projected coordinate system.

Requirements

Mon, 01 Jan 0001 00:00:00 +0000

Requirements #

The requirements include R (v4.X.X) and RStudio (if you’d like to work with models-for-missing-data interactively), and JAGS (v4.3.0). Git and Git Bash (for Windows users) are convenient if you’d like to stay up to date with the latest changes.

The requirements (including a few non-essentials) are described in the latest Dockerfile. You’re likely to need everything you see beneath install2.r in the instructions below. You can install these from your R console with install.packages().

FROM rocker/geospatial:4.1.2

# Copy Latin Modern font files to fonts directory and refresh fonts cache.
COPY Latin-Modern-Roman-fontfacekit.zip /tmp
RUN unzip tmp/Latin-Modern-Roman-fontfacekit.zip -d /usr/share/fonts && \
 fc-cache -f -v && \
 rm tmp/Latin-Modern-Roman-fontfacekit.zip

# Install JAGS.
RUN apt-get update -y && apt-get install -y jags

# Graphics and other required packages.
RUN apt-get install tree && \
 Rscript -e "update.packages(ask = FALSE)" && \
 install2.r --error \
 abind \
 spsurvey \
 hrbrthemes \
 ggthemes \
 ggridges \
 cowplot \
 HDInterval \
 rjags \
 coda \
 R2jags \
 runjags \
 bayesplot \
 extraDistr \
 MCMCpack \
 magick \
 gifski \
 gganimate \
 multidplyr

Covariate info

Mon, 01 Jan 0001 00:00:00 +0000

Covariate info #

Covariate information (covariate info), if provided, is joined to the response information prior to model fitting. It’s used only in models for which predictors are specified.

covariate info:
 file: assets/uplands-data/ROMN/LIBI_Covariates_AllSitesAllYears_20201104_Through2016_with_exotics.csv
 event date info:
 date-time column: Year
 date-time format: Y!

As with the other statements in the data block, we point to the name of the covariate file (file; see the covariate data section for more detail) and, if different from the date attributes in the response data, supply info required to properly parse covariate datetime fields.

Custom data guide

Mon, 01 Jan 0001 00:00:00 +0000

Running a Forecast with Custom Data #

Required Input Files #

Forecast Config YAML Reference #

Running the Pipeline #

Rscript forecasting/forecast/forecast-pipeline.R \
 --config path/to/forecast-config.yaml

Output Files #

Group-level effects

Mon, 01 Jan 0001 00:00:00 +0000

Group-level effects #

The basic motivation for random group-level effects is that sites matter. They are an essential feature of the sampling design. Having no such random effects would imply that sites don’t matter and observations can be pooled (as if they were collected from a completely random sample).¹

Group-level effects incorporate differences among sites (that may be impossible to model with covariates) to inform park-level trends. They allow each site to have its intercept (and potentially, its slope) drawn from a common / underlying distribution of intercepts (and slopes). We make inference at the park scale using the parameters of the distribution from which the individual site intercepts and slopes are drawn.

Marginal effects

Mon, 01 Jan 0001 00:00:00 +0000

Marginal effects #

Summaries of the association between a change in a regressor and a change in the response variable.

Syntax #

drivers: true

Usage #

When drivers: true, the model API creates additional JAGS objects, which are used to evaluate the marginal effects of explanatory variables in the model. By default, it shows the effect of each variable over its empirical range, holding all other variables at zero (the mean of scaled continuous variables or the reference level of categorical variables).

Quickstart

Mon, 01 Jan 0001 00:00:00 +0000

Quickstart #

Assuming you’ve got the models-for-missing-data project code (see Installation) and have all of the dependencies (see Requirements), the last remaining pieces you’ll need to begin running models are the data and analysis config files. We’ve included examples of each in the “assets/” directory. See the guide for more information about each of these components of the models-for-missing-data workflow. You can find an example of the data here, and an analysis config file for these data here.

Unequal inclusion probabilities

Mon, 01 Jan 0001 00:00:00 +0000

The Sonoran Desert is among the most extreme environments on Earth. Sampling in these remote, rugged landscapes requires a different approach. When the Park Service established monitoring in Organ Pipe Cactus National Monument they used an approach to select sites based on the cost of travel to sites on the broader landscape, visiting less “costly” sites with higher probability than more costly sites. The cost surface that defined the probability of inclusion of sites was developed using terrain data, and a tool that estimates the time to travel to any arbitrary location on the landscape.

Posterior predictive checks

Mon, 01 Jan 0001 00:00:00 +0000

Posterior predictive checks #

By default, the posterior predictive checks produced by the model API include stratum_id and unit_code. We include these elements by default because they correspond directly to subscripts in the model. You can specify additional variables for post-hoc model checking via ppc facets. Performing posterios predictive checks is an essential step in modeling process. These checks essentially allow you to evaluate whether your model can generate new data that are similar to your observed data. The posterior predictive check process involves generating many new data sets from your model from what is called the posterior predictive distribution, and comparing charateristics (e.g. the mean and/or variance of the data set) of those simulated data sets those same characteristics of data used for model fitting. If you have a very good model that fits your data well, then in the case of the mean, you should expect the true mean of your data set to fall right around the 50th percentile of the distribution of means from the simulated datasets (the same goes for variance or other quantites). A posterior predictive check identifies this quantile (which is often referred to as a Bayesian p value). A Bayesian p value that is very close to 0 or 1 (a rule of thumb that is often used is greater than 0.8 or less than 0.2) may suggest a poor model fit.

Sampling and populations

Mon, 01 Jan 0001 00:00:00 +0000

We sample for a very practical reason. It’s usually impossible to get information on the whole population, so we use a sample to make inferences about the population. In our case, the population is typically all sites in a stratum or all sites – in all strata – at the scale of an entire park. Typically, the inference we seek entails three questions.

What’s the best estimate of the population mean?

We can generate a sample mean, \(\bar{x}\) , from our sample. This is the best estimate of the population mean.

Sync changes

Mon, 01 Jan 0001 00:00:00 +0000

Sync changes #

Checking for updates #

Project updates #

To get the latest models-for-missing-data code, navigate to your project directory using a terminal, and run git pull.

Submodule updates / individual submodules #

cd <submodule path> # e.g., model-api
git fetch
git checkout gh-submodule
git pull origin gh-submodule

Submodule updates / all submodules simultaneously #

First, be sure to do any necessary housekeeping (remove, stash, or commit changes). To get the latest updates for each of the submodules, run:

Variances

Mon, 01 Jan 0001 00:00:00 +0000

Variances #

As with group-level effects for the intercept or time slope terms, each site can also have its own variance.

Syntax #

variances:
 stratum-level fixed:
 level: stratum
 type: fixed
 response column(s):
 - native.rich
 - native.forb.rich
 site-level hierarchical:
 level: site
 type: hier
 response column(s):
 - native.rich
 - native.forb.rich

Usage #

The entries beneath response column(s) have to be column names, not variable descriptions. The model API will create all combinations of models specified in these blocks. Thus, in the example above, we would obtain four distinct models.

Covariates

Mon, 01 Jan 0001 00:00:00 +0000

Covariates #

The predictors to use for modeling the mean of the response variable.

Syntax #

additional covariates:
 - Botanist (JA)
 - Botanist (JA), deficit.pregr

Usage #

For categorical variables (e.g., Botanist), the optional parenthetical declaration can do one of two things.

It sets the reference level for dummy-coded variables. In the case of Botanist (JA), the reference level becomes the botanist with initials JA. If not specified, the reference level will be set using R’s default handling for factors (using the first level of the factor sorted in ascending alphabetical o.rder as the reference level)
It can also be used to implement sum-to-zero effect coding, which we call deflections. For instance MgmtZone (deflections). In the case of deflections, the coefficients for each level of the categorical variable sum to zero. By default the model returns only the first \(k-1\) coefficients, but the \(k^{\mathrm{th}}\) coefficient can be computed as a derived quantity. See this StackExchange post for more on this calculation.

See this resource for additional background on dummy vs effect coding.

Exposure / offset

Mon, 01 Jan 0001 00:00:00 +0000

Exposure / offset #

Initial values

Mon, 01 Jan 0001 00:00:00 +0000

Initial values #

Initial values determine the starting point of the sampler, and can play a large role in ensuring efficient sampling and convergence. Implementing some degree of randomness is also important when defining initial values to ensure that a model is able to converge properly for a range of initial values. Providing initial values that are within or near the posterior distribution of the parameter of interest serves not only to speed up convergence, but also to optimize the efficiency of the sampler (i.e. make it more random and less autocorrelated) in cases when the sampling algorithm is doing hyperparameter tuning during the early phases of sampling.

Interpreting coefficients

Mon, 01 Jan 0001 00:00:00 +0000

Making sense of the effects of variables included as predictors #

Some aspects of covariate effects are readily apparent – for instance, the sign of a coefficient in a model says at least something about the general directionality of the effect, positive or negative. However, a deeper understanding of a model typically requires inferences that go well beyond simple measures of the directionality or significance of effects – it requires understanding the size of effects.

Stratum-varying fixed effects

Mon, 01 Jan 0001 00:00:00 +0000

Assume we have three strata, \(s_0\) , \(s_1\) , and \(s_2\) , where \(s_0\) is the “reference” stratum – in other words, \(s_0\) is the stratum for which the 0/1 indicator is 0 across the board in the indicator matrix below (the first row):

\[\begin{bmatrix} 1 & 0 & 0 \\ 1 & 1 & 0 \\ 1 & 0 & 1 \end{bmatrix}\]

B_0 + (B_1 + B_1_s1_offset * s1 + B_1_s2_offset * s2) * x_1 

# in stratum s0
B_0 + (B_1) * x_1 

# in stratum s1
B_0 + (B_1 + B_1_s1_offset * s1) * x_1 

# in stratum s2
B_0 + (B_1 + B_1_s2_offset * s2) * x_1 

# lm(y~x1*x2)
model.matrix(~x1*x2, tibble(x1 = runif(5), x2 = runif(5)))

The offset term

Mon, 01 Jan 0001 00:00:00 +0000

Counts of things naturally scale with the length or duration of observation, the area sampled, and sampling intensity ( Citation: McElreath, 2018 McElreath, R. (2018). Statistical rethinking: A bayesian course with examples in r and stan. Chapman; Hall/CRC. ) . For instance, the longer the river stretch we survey, the more fish we’ll tend to find.

Offset terms are used to model rates – e.g., counts per unit area or time. In the context of the model, the offset term transforms the response variable from a rate to a count.

Mon, 01 Jan 0001 00:00:00 +0000