Skip to main content
SpringerLink
Log in

Prediction-Corrected Visual Predictive Checks for Diagnosing Nonlinear Mixed-Effects Models

  • Research Article
  • Published:
The AAPS Journal Aims and scope Submit manuscript

Abstract

Informative diagnostic tools are vital to the development of useful mixed-effects models. The Visual Predictive Check (VPC) is a popular tool for evaluating the performance of population PK and PKPD models. Ideally, a VPC will diagnose both the fixed and random effects in a mixed-effects model. In many cases, this can be done by comparing different percentiles of the observed data to percentiles of simulated data, generally grouped together within bins of an independent variable. However, the diagnostic value of a VPC can be hampered by binning across a large variability in dose and/or influential covariates. VPCs can also be misleading if applied to data following adaptive designs such as dose adjustments. The prediction-corrected VPC (pcVPC) offers a solution to these problems while retaining the visual interpretation of the traditional VPC. In a pcVPC, the variability coming from binning across independent variables is removed by normalizing the observed and simulated dependent variable based on the typical population prediction for the median independent variable in the bin. The principal benefit with the pcVPC has been explored by application to both simulated and real examples of PK and PKPD models. The investigated examples demonstrate that pcVPCs have an enhanced ability to diagnose model misspecification especially with respect to random effects models in a range of situations. The pcVPC was in contrast to traditional VPCs shown to be readily applicable to data from studies with a priori and/or a posteriori dose adaptations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Orloff J, Douglas F, Pinheiro J, Levinson S, Branson M, Chaturvedi P, et al. The future of drug development: advancing clinical trial design. Nat Rev Drug Discov. 2009;8(12):949–57.

    PubMed  CAS  Google Scholar 

  2. Karlsson MO, Savic RM. Diagnosing model diagnostics. Clin Pharmacol Ther. 2007;82(1):17–20.

    Article  PubMed  CAS  Google Scholar 

  3. Sheiner LB, Beal SL, Dunne A. Analysis of nonrandomly censored ordered categorical longitudinal data from analgesic trials. J Am Stat Assoc. 1997;92(440):1235–44.

    Article  Google Scholar 

  4. Holford N. The visual predictive check—superiority to standard diagnostic (Rorschach) plots. PAGE 14 (2005) Abstr 738 [www.page-meeting.org/?abstract=738].

  5. Karlsson MO, Holford N. A tutorial on visual predictive checks. PAGE 17 (2008) Abstr 1434 [www.page-meeting.org/?abstract=1434].

  6. Post TM, Freijer JI, Ploeger BA, Danhof M. Extensions to the visual predictive check to facilitate model performance evaluation. J Pharmacokinet Pharmacodyn. 2008;35(2):185–202.

    Article  PubMed  Google Scholar 

  7. Wang DD, Zhang S. Standardized visual predictive check—how and when to used it in model evaluation. PAGE 18 (2009) Abstr 1501 [www.page-meeting.org/?abstract=1501].

  8. Wallin JE, Friberg LE, Fasth A, Staatz CE. Population pharmacokinetics of tacrolimus in pediatric hematopoietic stem cell transplant recipients: new initial dosage suggestions and a model-based dosage adjustment tool. Ther Drug Monit. 2009 Jun 11.

  9. Gibiansky L. Parameter estimates of population models: comparison of nonmem versions and estimation methods. PAGE 17 (2008) Abstr 1268 [www.page-meeting.org/?abstract=1268].

  10. Brynne L, McNay JL, Schaefer HG, Swedberg K, Wiltse CG, Karlsson MO. Pharmacodynamic models for the cardiovascular effects of moxonidine in patients with congestive heart failure. Br J Clin Pharmacol. 2001;51(1):35–43.

    Article  PubMed  CAS  Google Scholar 

  11. Beal SL, Sheiner LB, Boeckmann AJ. NONMEM user’s guides. Ellicot City: MD: Icon Development Solutions; 1989–2006.

  12. Lindbom L, Pihlgren P, Jonsson EN. PsN-Toolkit—a collection of computer intensive statistical methods for non-linear mixed effect modeling using NONMEM. Comput Method Programs Biomed. 2005;79(3):241–57.

    Article  Google Scholar 

  13. Lindbom L, Ribbing J, Jonsson EN. Perl-speaks-NONMEM (PsN)—a Perl module for NONMEM related programming. Comput Method Programs Biomed. 2004;75(2):85–94.

    Article  Google Scholar 

  14. Harling K, Ueckert S, Hooker AC, Jonsson EN, Karlsson MO. Xpose and Perl speaks NONMEM (PsN). PAGE 19 (2010) Abstr 1842 [www.page-meeting.org/?abstract=1842].

  15. Jonsson EN, Karlsson MO. Xpose—an S-PLUS based population pharmacokinetic/pharmacodynamic model building aid for NONMEM. Comput Method Programs Biomed. 1999;58(1):51–64.

    Article  CAS  Google Scholar 

  16. R: a language and environment for statistical computing. R Foundation for Statistical Computing; 2009 [2009-12-30]; Available from: http://www.R-project.org.

  17. Box GEP, Draper NR. Empirical model-building and response surfaces: Wiley; 1987.

  18. Bergstrand M, Hooker AC, Wallin JE, Karlsson MO. Prediction corrected visual predictive checks. ACoP (2009) Abstr F7 http://2009.go-acop.org/sites/all/assets/webform/Poster_ACoP_VPC_091002_two_page.pdf.

  19. Ma G, Olsson BL, Rosenborg J, Karlsson MO. Quantifying lung function progression in asthma. PAGE 18 (2009) Abstr 1562 [www.page-meeting.org/?abstract=1562].

  20. Dansirikul C, Silber HE, Karlsson MO. Approaches to handling pharmacodynamic baseline responses. J Pharmacokinet Pharmacodyn. 2008;35(3):269–83.

    Article  PubMed  CAS  Google Scholar 

  21. Bergstrand M, Karlsson MO. Handling data below the limit of quantification in mixed effect models. AAPS J. 2009;11(2):371–80.

    Article  PubMed  CAS  Google Scholar 

  22. Friberg LE, de Greef R, Kerbusch T, Karlsson MO. Modeling and simulation of the time course of asenapine exposure response and dropout patterns in acute schizophrenia. Clin Pharmacol Ther. 2009;86(1):84–91.

    Article  PubMed  CAS  Google Scholar 

  23. Kuhn E, Lavielle M. Maximum likelihood estimation in nonlinear mixed effects models. Comput Stat Data Anal. 2005;49(4):1020–38.

    Article  Google Scholar 

  24. Khachman D, Laffont CM, Concordet D. You have problems to interpret VPC? Try VIPER! PAGE 19 (2010) Abstr 1892 [www.page-meeting.org/?abstract=1892].

  25. Laffont CM, Concordet D. A new exact test to globally assess a population PK and/or PD model. PAGE 18 (2009) Abstr 1633 [www.page-meeting.org/?abstract=1633].

  26. Comets E, Brendel K, Mentré F. Computing normalised prediction distribution errors to evaluate nonlinear mixed-effect models: the npde add-on package for R. Comput Meth Programs Biomed. 2008;90(2):154–66.

    Article  Google Scholar 

  27. Comets E, Brendel K, Mentré F. Model evaluation in nonlinear mixed effect models, with applications to pharmacokinetics. J Soc Fr Stat. 2010;151(1):106–28.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Pharmaceutical Biosciences, Uppsala University, P.O. Box 591, 751 24, Uppsala, Sweden

    Martin Bergstrand, Andrew C. Hooker, Johan E. Wallin & Mats O. Karlsson

  2. Lilly PKPD&TS, Windlesham Surrey, UK

    Johan E. Wallin

Authors
  1. Martin Bergstrand
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew C. Hooker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Johan E. Wallin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mats O. Karlsson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martin Bergstrand.

Electronic Supplementary Materials

Below is the link to the electronic supplementary material.

ESM 1

(PDF 403 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bergstrand, M., Hooker, A.C., Wallin, J.E. et al. Prediction-Corrected Visual Predictive Checks for Diagnosing Nonlinear Mixed-Effects Models. AAPS J 13, 143–151 (2011). https://doi.org/10.1208/s12248-011-9255-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1208/s12248-011-9255-z

KEY WORDS

Search

Navigation