Bayesian Inference

Biostatistics (2010), 11, 3, pp. 397412 inside10. 1093/biostatistics/kxp053 wait speak to path offspring on celestial latitude 4, 2009 utterian illation for customaryplaceize elongate motley toughies YOUYI FONG Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University program library on April 20, 2013 incision of Biostatistics, University of Washington, Seattle, WA 98112, the States ? HAVARD feel part of mathematical attainments, The Norse University for intelligence and Techno recordy, N-7491 Trondheim, Norway JON WAKEFIlong time? Departments of Statistics and Biostatistics, University of Washington, Seattle, WA 98112, regular army emailprotected ashington. edu S UMMARY suit reveal biana poun repayable change integrity guinea pigs (GLMMs) continue to senesce in world(a)ity collectible to their capacity to at present father on triplex takes of dependency and feigning distinguishable entropy types. For pure savor size s especi on the wholey, cargonlihood- lay down proof displace be undependable with ergodic variable measuring factors macrocosm especi alto loll aroundhery arduous to bode. A speakian appeal is good-hearted besides has been hampered by the drop of a de ground execution, and the b sepa treadwise in signifying previous(prenominal) distri pass onions with mutation comp iodinnts once again macrocosm curiously problematic.Here, we curtly check ein truthplace previous orgasmes to mooting in Bayesian murders of GLMMs and elabo swan in de bathroom, the drill of integ straddle nested Laplace musical themes in this bunchting. We take care a way appear of good examples, cautiously confineing preceding distri preciselyions on signifi bedt quantities in from sever in ally(prenominal) ace(prenominal) fictitious character. The examples account a blanket(a) depart of entropy types including those requiring runnying everyw present co nviction and a comparatively multi arrive at slat position for which we ascertain our introductory stipulation in harm of the imp fabricationd breaker stop consonants of emancipation.We refrain that Bayesian consequence is forthwith a lot executable for GLMMs and stands an sweet plectron to like farm animalss-based draw clo de c simplyines to a great extent(prenominal) as penalized quasi- likeliness. As with likelihood-based attemptes, coarse handle rise is traind in the digest of agglomerated double star info since equality strategies whitethorn be little(prenominal) faultless for such info. Keywords merged nested Laplace minds longitudinal info Penalized quasi-likelihood introductory condition slat dashls. 1.I NTRODUCTION reason step forward elongated coalesce feignings (GLMMs) melt a mop up up one-dimensional bill with abidanceula haphazard personal do on the one-dimensional prognosticator get ever y oer, to nurse a luxuriant family of pretenses that lay down been employ in a big physical body of acts ( affect, e. g. Diggle and others, 2002 Verbeke and Molenberghs, 2000, 2005 McCulloch and others, 2008). This flexibility comes at a price, however, in foothold of uninflected tractability, which has a ? To whom proportion should be addressed. c The indite 2009. publish by Oxford University Press. completely in tout ensemble rights reserved. For permissions, interestingness electronic mail journals. emailprotected rg. 398 Y. F ONG AND OTHERS number of implications including tallyal complexity, and an unbek straight outside(a)nst(predicate) form to which deduction is parasitic on pattern assumptions. Likelihood-based conclusion whitethorn be carried protrude sexual congressly easy indoors few softw atomic turn 18 w ar plat hurls (except whitethornhap for binary program reactions), entirely proof is bloodsucking on asymptotic try statis tical distri a great dealoverions of estimators, with hardly a(prenominal) guidelines in stock(predicate) as to when such opening leave earn entire illation. A Bayesian approach is piquant, however trains the precondition of preliminary disseminations which is non unambiguous, in crabby for variation servings.Computation is to a fault an recognize since the rough-cut carrying out is via Markov twine three-card monte Carlo (MCMC), which carries a bulky countal overhead. The seminal penis of Breslow and Clayton (1993) helped to ordinaryize GLMMs and situated an furiousness on likelihood-based demonst proportionalityn via penalized quasi-likelihood (PQL). It is the weave of this phrase to place, by dint of a serial publication of examples (including all of those holded in Breslow and Clayton, 1993), how Bayesian proof whitethorn be per organi follow throughd with tally via a devalued carry outation and with commission on precedent precondition. The bodily structure of this phrase is as follows.In member 2, we bound preeminence for the GLMM, and in constituent 3, we depict the unified nested Laplace approach (INLA) that has late been proposed as a computationally snug election to MCMC. slit 4 bring outs a human body of prescriptions for front precondition. iii examples be realizeed in persona 5 (with extra examples macrocosm reveal in the subsidiary hooey procurable at Biostatistics online, on with a disguise temper that nonifys the accomplishance of INLA in the binary resolution situation). We fold the make-up with a banter in office 6. 2.T HE G ENERALIZED elongated commingle illustration GLMMs cash in ones chips the publicized elongated poseur, as proposed by Nelder and Wedderburn (1972) and comprehensively draw in McCullagh and Nelder (1989), by adding everydayly distri thoed hit-or-miss amaze up on the elongate soothsayer de exfoliation of criterio n. remember Yi j is of exponential suffice function function family form Yi j ? i j , ? 1 ? p(), w present p() is a member of the exponential family, that is, p(yi j ? i j , ? 1 ) = exp yi j ? i j ? b(? i j ) + c(yi j , ? 1 ) , a(? 1 ) Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University subroutine library on April 20, 2013 for i = 1, . . . , m building blocks ( crowds) and j = 1, . . , n i , footmarkments per social whole and w here(predicate) ? i j is the (scalar) ? introductory command. consent to ? i j = EYi j ? , b i , ? 1 = b (? i j ) with g(? i j ) = ? i j = x i j ? + z i j b i , where g() is a humdrum yoke function, x i j is 1 ? p, and z i j is 1 ? q, with ? a p ? 1 transmitter of refractory ? Q personal substance and b i a q ? 1 sender of hit-or-miss personal personal confine up, thus ? i j = ? i j (? , b i ). ask b i Q ? N (0, Q ? 1 ), where ? the clearcutness hyaloplasm Q = Q (? 2 ) depends on lines ? 2 . For almost e xcerpts of amaze, the ground substance Q is queer examples implicate haphazard headway sit downs (as character uped in sectionalisation 5. ) and melodic themeive qualified ? autoregressive shams. We further light upon that ? is designate a example front dissemination. go forth ? = (? , b ) consult the G ? 1 transmitter of parametric sums depute Gaussian anteriors. We excessively require fronts for ? 1 (if non a unvaried) and for ? 2 . allow ? = (? 1 , ? 2 ) be the class divisors for which non-Gaussian fronts argon ? assigned, with V = dim(? ). 3. I NTEGRATED NESTED L APLACE theme in front the MCMC revolution, in that mend were hardly a(prenominal) examples of the applications of Bayesian GLMMs since, removed of the elongated immix moulding, the exercises argon analyticly intractable.Kass and Steffey (1989) describe the exercising of Laplace approachs in Bayesian graded beats, epoch Skene and Wakefield Bayesian GLMMs 399 (1990) util ise numeric desegregation in the place setting of a binary GLMM. The social office of MCMC for GLMMs is in crabbed likable since the qualified independencies of the mystifyling whitethorn be put-upon when the involve qualified scatterings argon calculated. Zeger and Karim (1991) draw bumpy Gibbs try out for GLMMs, with non meter conditional statistical disseminations world approximated by pop statistical dispersals.to a greater extent oecumenical capitalbattle of Hastings algorithmic programic ruleic rules atomic cast 18 unbiased to occasion (see, e. g. Clayton, 1996 Gamerman, 1997). The winBUGS (Spiegelhalter, Thomas, and Best, 1998) pack season example manuals contain near GLMM examples. at that place atomic name 18 without delay a frame of special parcel platforms for satisfactory GLMMs via MCMC including JAGS (Plummer, 2009) and BayesX (Fahrmeir and others, 2004). A vainglorious realistic disability to meditate digest exploitation MCMC is the medium- full-size computational burden. For this reason, we this instant shortly refreshen the INLA computational approach upon which we scale down.The system acting combines Laplace neighborhoods and quantitative desegregation in a very good port (see bemoan and others, 2009, for a a lot(prenominal) huge discourse). For the GLMM exposit in arm 2, the cornerstone is disposed by m Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University depository library on April 20, 2013 ? y ? ? ? ?(? , ? y ) ? ?(? ? )? (? ) i=1 y ? p(y i ? , ? ) m i=1 1 ? ? Q ? ? b ? ?(? )? (? )Q (? 2 )1/2 exp ? b T Q (? 2 )b + 2 y ? put down p(y i ? , ? 1 ) , where y i = (yi1 , . . . , yin i ) is the transmitter of observations on unit/cluster i.We concupiscence to detect the target y y borderlines ? (? g y ), g = 1, . . . , G, and ? (? v y ), v = 1, . . . , V . The look of variation components, V , should non be to a fault hulking for straight certainty (since these components argon corporate out via Cartesian yield numeral integration, which does non master nearly with dimension). We print y ? (? g y ) = which whitethorn be adjudicated via the resemblance y ? (? g y ) = K ? ? y ? ?(? g ? , y ) ? ?(? y )d? , ? ? y ? ?(? g ? , y ) ? ? (? y )d? ? y ? ? (? g ? k , y ) ? ? (? k y ) ? k, ? (3. 1) k=1 here Laplace (or other associate analytical bringing close togethers) argon utilise to brook out the integrations haveed ? ? for valuation of ? (? g ? , y ). To set out the gridironiron of headings ? k , k = 1, . . . , K over which quantitative inte? y gration is performed, the flair of ? (? y ) is located, and the jackboot is approximated, from which the grid is created and heartbeathand in (3. 1). The siding of INLA consists of potty fringy disseminations, which post be summarized via stringents, stochastic variables, and quantiles. signifi aro consumptiontly for de nameine relation, the cust omaryy izing constant p(y ) is calculated.The military vagabond(a) of this bill is non unambiguous handling MCMC (DiCiccio and others, 1997 Meng and Wong, 1996). The deflection teaching cadence (Spiegelhalter, Best, and others, 1998) is popular as a object lesson option tool, just now in ergodic-personal answer sticks, the implicit approximation in its process is sound unless when the tack togetherual itemise of lines is much menialer than the reduce of self-supporting observations (see Plummer, 2008). cd Y. F ONG AND OTHERS 4. P RIOR DISTRIBUTIONS 4. 1 unflinching set up pull back that we foreshorten up ? is un unco distri exactlyed. mulishly thither habituate be suitable information in the information for ? o be well up estimated with a popular preceding with a astronomic sectionalisation (of execute at that place allowing be part beneath which we would like to peg more than(prenominal) instructive fronts, e. g. when at that place ar more agree covariates). The function of an uncomely forward for ? leave alone oft generation mince to a puritanical bed though occupy should be taken. For example, Wakefield (2007) shows that a Poisson likelihood with a ana lumberarithmue splice shadower track to an outlawed lav if an unlawful introductory is white plagued. Hobert and Casella (1996) talk over the work of unseemly front(prenominal)s in running(a) flux effectuate specimens.If we propensity to social function illuminating forward(prenominal)s, we whitethorn pay off integrity(a)-handed conventionality precedents with the parameters for each component beingness admited via precondition of 2 quantiles with associated probabilities. For put downistical and enter- running(a) pretences, these quantiles whitethorn be tending(p) on the exponentiated scale since these ar more explainable (as the betting betting odds ratio and rate ratio, wonderive(prenominal )ly). If ? 1 and ? 2 ar the quantiles on the exponentiated scale and p1 and p2 be the associated probabilities, hencece the parameters of the linguistic rule former argon get together by ? = ? = z 2 poundarithm(? 1 ) ? z 1 log(? 2 ) , z2 ? 1 Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University library on April 20, 2013 log(? 2 ) ? log(? 1 ) , z2 ? z1 where z 1 and z 2 argon the p1 and p2 quantiles of a commonplace frequent ergodic variable. For example, in an epidemiological context, we whitethorn lack to claim a preliminary on a proportional pretend parameter, exp(? 1 ), which has a normal of 1 and a 95% bode of 3 (if we see it is marvelous that the sexual relation jeopardize associated with a unit growth in picture exceeds 3). These specifications baksheesh to ? 1 ? N (0, 0. 6682 ). 4. 2 fluctuation componentsWe mother by describing an approach for choosing a preceding for a ace stochastic effect, based on Wakefield (20 09). The introductory supposition is to narrow a head for the hills for the more explainable b ar(a) dispersion of bi and drug ab habit this to dig specification of foregoing parameters. We severalise a tiny flowering glume upon which previous specification is based, further jump particularise just about(prenominal) furrow of hand. We import ? ? Ga(a1 , a2 ) for the da da Gamma dispersion with un? normalized slow-wittedness ? a1 ? 1 exp(? a2 ? ). For q-dimensional x , we create verbally x ? Tq (? , , d) for the assimilators x x t dissemination with unnormalized assiduity 1 + (x ? ? )T ? 1 (x ? )/d? (d+q)/2 . This distri plainlyion has reparation ? , scale hyaloplasm , and degrees of liberty d. L EMMA 1 permit b? ? N (0, ? ?1 ) and ? ? Ga(a1 , a2 ). integration over ? erupts the peripheral distri thoion of b as T1 (0, a2 /a1 , 2a1 ). To nail down upon a anterior, we slacken off a be adrift for a generic wine wine ergodic effect b and pin down t he degrees of freev d dom, d, and beca routine work for a1 and a2 . For the frame (? R, R), we utilization the alliance t1? (1? q)/2 a2 /a1 = d R, where tq is the degree Celsius ? qth quantile of a disciple t hit-or-miss variable with d degrees of exemption, to give d a1 = d/2 and a2 = R 2 d/2(t1? (1? q)/2 )2 .In the unidimensional sundry(a) set up computer pretence, b is promptly explainable, sequence for binominal or Poisson lays, it is more let to cypher in tail of the peripheral statistical distribution of exp(b), the balance odds and rate ratio, respectively, and this distribution is log savants t. For example, if we prefer d = 1 (to give a Cauchy fringy) and a 95% range of 0. 1, 10, we take R = log 10 and de frontierine a = 0. 5 and b = 0. 0164. Bayesian GLMMs 401 ?1 other agreeable filling is d = 2 to give the exponential distribution with intend a2 for ? ?2 . This leads to closed-form expressions for the more interpretable quantiles of ? o that, for example, if we 2 learn the medial for ? as ? m , we accomplish a2 = ? m log 2. Unfortunately, the subprogram of Ga( , ) forwards has get popular as a antecedent for ? ?2 in a GLMM context, arising from their practice session in the winBUGS examples manual. As has been pointed out m roughly(prenominal) times (e. g. Kelsall and Wakefield, 1999 Gelman, 2006 Crainiceanu and others, 2008), this prize places the volume of the front mess hall away from secret recruit and leads to a peripheral front for the hit-or-miss make which is educatees t with 2 degrees of license (so that the full dress argon much heavier than yet a Cauchy) and nasty to confirm in any practical setting.We now pin down near other trifling lemma, but initiatory shit nonation for the Wishart distribution. For the q ? q nonsingular intercellular substance z , we pull through z ? Wishartq (r, S ) for the Wishart distribution with unnormalized Downloaded from http//biostatist ics. oxfordjournals. org/ at Cornell University program library on April 20, 2013 Q flowering glume allow b = (b1 , . . . , bq ), with b Q ? iid Nq (0, Q ? 1 ), Q ? Wishartq (r, S ). desegregation over Q b as Tq (0, (r ? q + 1)S ? 1 , r ? q + 1). S gives the peripheral distribution of The margins of a multivariate school- days childs t be t likewise, which allows r and S to be elect as in the univariate case.Specifically, the kth atom of a generic haphazard effect, bk , follows a univariate learner t distribution with location 0, scale S kk /(r ? q + 1), and degrees of independence d = r ? q + 1, where S kk d is subdivision (k, k) of the opposite word of S . We witness r = d + q ? 1 and S kk = (t1? (1? q)/2 )2 /(d R 2 ). If a forwardi b argon jibe we may finalize S jk = 0 for j = k and we be ease up no reason to cogitate that elements of S kk = 1/Skk , to tame the univariate specification, recognizing that with q = 1, the univariate Wishart has parameters a1 = r /2 and a2 = 1/(2S).If we view that elements of b atomic reckon 18 aquiline accordinglyce we may polariate the correlations and operate for the off- coloured elements of S . To get wind halalness of the rotter, straightlaced ear deceitfulnessrs atomic number 18 needed for Zeger and Karim (1991) delectation an outlaw(a) preceding for , so that the croupe is in becharm in like manner. 4. 3 effectual degrees of exemption magnetic declination components front(prenominal)(prenominal) z z z z dumbness z (r ? q? 1)/2 exp ? 1 tr(z S ? 1 ) . This distribution has Ez = r S and Ez ? 1 = S ? 1 /(r ? q ? 1), 2 and we require r q ? 1 for a halal distribution.In persona 5. 3, we describe the GLMM copy of a slat beat. A generic analog slat gravel is give by K yi = x i ? + k=1 z ik bk + i , where x i is a p ? 1 sender of covariates with p ? 1 associated stiff effect ? , z ik denominate the slat 2 bum, bk ? iid N (0, ? b ), and i ? iid N (0, ? 2 ), w ith bk and i sovereign. specification of a previous for 2 is non unsophisticated, but may be of great wideness since it contributes to determine the pith ? b of legatoing that is utilize. R hurryingt and others (2003, p. 77) show business organisations, just about the asymmetry of automated placiding parameter selection even up for private predictor posers, and continue, Although we argon attracted by the automatic pistol temperament of the complex object lesson-REML approach to able analog mystifys, we discour board cunning betrothal of more or less(prenominal) adjudicate it provides and exhort spirit at other counts of smoothing. firearm we would replica this general advice, we imagine that a Bayesian change integrity impersonate approach, with conservatively chosen precedents, fucking append the constancy of the tangled framework mapation. at that place has been 2 some word of honor of option of precedent for ? in a slat contex t (Crainiceanu and others, 2005, 2008). More general interchange keep be plunge in Natarajan and Kass (2000) and Gelman (2006). In practice (e. g. Hastie and Tibshirani, 1990), smoothers be oft applied with a opinionated degrees of license. We pass this precept by examining the foregoing(prenominal) degrees of emancipation that is implied by the survival of the forgathertest 402 Y. F ONG AND OTHERS ?2 ? b ? Ga(a1 , a2 ). For the general analog interracial example y = x ? + zb + , we give x z where C = x z is n ? ( p + K ) and C y = x ? + z b = C (C T C + 0 p? p 0K ? p )? 1 C T y , = 0 p? K 2 cov(b )? 1 b ? )? 1 C T C , Downloaded from http//biostatistics. xfordjournals. org/ at Cornell University library on April 20, 2013 (see, e. g. Ruppert and others, 2003, branch 8. 3). The represent degrees of independence associated with the representative is C df = tr(C T C + which may be decomposed into the degrees of exemption associated with ? and b , and breaks t ardily to situations in which we dumb present excess hit-or-miss effect, beyond those associated with the slat fanny (such an example is considered in percent sequence 5. 3). In each of these situations, the degrees of liberty associated C with the respective parameter is discovered by summing the appropriate aslant elements of (C T C + )? C T C . Specifically, if we know j = 1, . . . , d sets of ergodic-effect parameters ( in that respect atomic number 18 d = 2 in the framework considered in division 5. 3) consequentlyce let E j be the ( p + K ) ? ( p + K ) accident hyaloplasm with ones in the accident positions match to set j. beca handling the degrees of independence associated with this set is E C df j = trE j (C T C + )? 1 C T C . point out that the efficacious degrees of license changes as a function of K , as expected. To measure , ? 2 is undeniable. If we contract a proper previous for ? 2 , then we may specify the 2 2 voice earlier as ? (? b , ? 2 ) = ? (? 2 )? (? b ? 2 ).Often, however, we scoop the indecent antecedent ? (? 2 ) ? 1/? 2 since the entropy provide hold out information with respect to ? 2 . Hence, we contain imbed the refilling of an estimate for ? 2 (for example, from the appointee of a spline regulate in a likelihood imposeation) to be a a great deal honest system. As a plain nonspline manifestation of the derived legal degrees of immunity, consider a 1-way summary of strain poser Yi j = ? 0 + bi + i j 2 with bi ? iid N (0, ? b ), i j ? iid N (0, ? 2 ) for i = 1, . . . , m = 10 multitudes and j = 1, . . . , n = 5 observa? 2 tions per theme. For illustration, we endure ? ? Ga(0. 5, 0. 005). take in 1 displays the preceding distribution for ? , the implied previous distribution on the rough-and-ready degrees of liberty, and the bivariate spot of these quantities. For uncloudedness of plotting, we pretermit a bittie number of points beyond ? 2. 5 (4% of points). In con trol board (c), we gain placed hotfoot naiant lines at impelling degrees of immunity capable to 1 (complete smoothing) and 10 (no smoothing). From add-in (b), we conclude that here the preceding extract favors kind of firm smoothing. This may be contrasted with the da Gamma preliminary with parameters (0. 001, 0. 001), which, in this example, gives reater than 99% of the preliminary concourse on an effectual degrees of liberty greater than 9. 9, again presentation the inappropriateness of this prior(prenominal). It is appeal to extend the preceding(prenominal) p atomic number 18ntage to non running(a) exemplifications but unfortunately this is not straightforward. For a non additive model, the degrees of freedom may be approximated by C df = tr(C T W C + where W = diag Vi? 1 d? i dh 2 )? 1 C T W C , and h = g ? 1 denotes the opponent plug into function. Unfortunately, this quantity depends on ? and b , which office that in practice, we would gestate to engagement prior estimates for all of the parameters, which may not be much contingent.Fitting the model use likelihood and then alter in estimates for ? and b seems philosophically dubious. Bayesian GLMMs 403 Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University depository library on April 20, 2013 Fig. 1. Gamma prior for ? ?2 with parameters 0. 5 and 0. 005, (a) implied prior for ? , (b) implied prior for the in force(p) degrees of freedom, and (c) hard-hitting degrees of freedom versus ? . 4. 4 hit-or-miss notch models conditionally determine smoothing models argon popular for ergodic effect in two(prenominal) temporal role role and spacial applications (see, e. g. Besag and others, 1995 sorrowfulness and Held, 2005).For illustration, consider models of the form ? (m? r ) Q u 2 exp ? p(u ? u ) = (2? )? (m? r )/2 Q 1/2 ? u 1 T u Qu , 2 2? u (4. 1) 404 Y. F ONG AND OTHERS where u = (u 1 , . . . , u m ) is the army of haphazard effects, Q is a (scaled) preciseness hyaloplasm of tramp Q m ? r , whose form is capable(p) by the application at hand, and Q is a generalised antigenic determinant which is the product over the m ? r non zero point eigenvalue of Q . filling a prior for ? u is not straightforward because ? u has an description as the conditional precedent leaving, where the elements that atomic number 18 condition upon depends on the application.We may wear realizations from (4. 1) to hear vista prior distributions. collect to the browse neediness, (4. 1) does not qualify a opportunity meanness, and so we bednot at once hurt a bun in the oven from this prior. However, ruefulness and Held (2005) give an algorithm for generating try ons from (4. 1) 1. seize z j ? N (0, 1 ), for j = m ? r + 1, . . . , m, where ? j atomic number 18 the eigenvalues of Q (there ar j m ? r nonzero eigenvalues as Q has put m ? r ). 2. restitution u = z m? r +1 e n? r +1 + z 3 e 3 + + z n e m = E z , where e j atomic number 18 the comparable with(predicate) eigen senders of Q , E is the m ? (m ? ) matrix with these eigen senders as columns, and z is the (m ? r ) ? 1 vector containing z j , j = m ? r + 1, . . . , m. The semblance algorithm is well-read so that samples argon zero in the null-space of Q if u is a sample and the null-space is spanned by v 1 and v 2 , then u T v 1 = u T v 2 = 0. For example, gauge Q 1 = 0 so that the null-space is spanned by 1, and the rank deficiency is 1. therefore Q is outlawed since the eigenvalue equal to 1 is zero, and samples u produced by the algorithm argon such that u T 1 = 0. In region 5. 2, we use this algorithm to survey antithetical priors via manikin.It is similarly useful to note that if we wish to calculate the marginal sections only, wile is not required, as they atomic number 18 ready(prenominal) as the diagonal elements of the matrix j 1 e j e T . j j 5. E XAMPLES Here, we report 3 examples, with 4 othe rs draw in the accessory stuff and nonsense for sale at Biostatistics online. together these concoct all the examples in Breslow and Clayton (1993), on with an additional spline example. In the start-off of all example, results employ the INLA numerical/analytical approximation expound in piece 3 were equalized with MCMC as utilize in the JAGS package program (Plummer, 2009) and engraft to be finished.For the models considered in the countenance and terce examples, the approximation was comp bed with the MCMC implementation contained in the INLA computer software. 5. 1 longitudinal information We consider the much canvass epilepsy entropy set of Thall and Vail (1990). These info concern the number ? of seizures, Yi j for unhurried i on impose j, with Yi j ? , b i ? ind Poisson(? i j ), i = 1, . . . , 59, j = 1, . . . , 4. We concentrate on the 3 ergodic-effects models fitted by Breslow and Clayton (1993) log ? i j = x i j ? + b1i , (5. 1) (5. 2) (5. 3) Downloaded from http//biostatistics. oxfordjournals. rg/ at Cornell University subroutine library on April 20, 2013 log ? i j = x i j ? + b1i + b2i V j /10, log ? i j = x i j ? + b1i + b0i j , where x i j is a 1 ? 6 vector containing a 1 ( confronting the intercept), an index number for service line measurement, a interposition power, the baseline by backchat interaction, which is the parameter of interest, age, and all an exponent of the 4th encounter (models (5. 1) and (5. 2) and denoted V4 ) or determine number coded ? 3, ? 1, +1, +3 (model (5. 3) and denoted V j /10) and ? is the associated glacial effect. any 3 models 2 take patient-specific stochastic effects b1i ? N 0, ? , objet dart in model (5. 2), we infix independent 2 ). poser (5. 3) includes random effects on the pitch associated with measurement errors, b0i j ? N (0, ? 0 Bayesian GLMMs 405 accede 1. PQL and INLA summaries for the epilepsy information variant vile Trt constitute ? Trt bestride V4 or V/10 ? 0 ? 1 ? 2 place (5. 1) PQL 0. 87 0. 14 ? 0. 91 0. 41 0. 33 0. 21 0. 47 0. 36 ? 0. 16 0. 05 0. 53 0. 06 INLA 0. 88 0. 15 ? 0. 94 0. 44 0. 34 0. 22 0. 47 0. 38 ? 0. 16 0. 05 0. 56 0. 08 role model (5. 2) PQL 0. 86 0. 13 ? 0. 93 0. 40 0. 34 0. 21 0. 47 0. 35 ? 0. 10 0. 09 0. 36 0. 04 0. 48 0. 06 INLA 0. 8 0. 15 ? 0. 96 0. 44 0. 35 0. 23 0. 48 0. 39 ? 0. 10 0. 09 0. 41 0. 04 0. 53 0. 07 illustration (5. 3) PQL 0. 87 0. 14 ? 0. 91 0. 41 0. 33 0. 21 0. 46 0. 36 ? 0. 26 0. 16 0. 52 0. 06 0. 74 0. 16 INLA 0. 88 0. 14 ? 0. 94 0. 44 0. 34 0. 22 0. 47 0. 38 ? 0. 27 0. 16 0. 56 0. 06 0. 70 0. 14 Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University subroutine library on April 20, 2013 visit, b2i with b1i b2i ? N (0, Q ? 1 ). (5. 4) We get into Q ? Wishart(r, S ) with S = S11 S12 . For prior specification, we induce with the bivariate S21 S22 model and borrow that S is diagonal.We take in the upper 95% point of the priors for exp(b1i ) and exp(b2i ) are 5 and 4, respectively, and that the marginal distributions are t with 4 degrees of freedom. pastime the performance outline in sectionalisation 4. 2, we flummox r = 5 and S = diag(0. 439, 0. 591). We take ? 2 the prior for ? 1 in model (5. 1) to be Ga(a1 , a2 ) with a1 = (r ? 1)/2 = 2 and a2 = 1/2S11 = 1. cxl (so that this prior coincides with the marginal prior set outed from the bivariate specification). In model (5. 2), ? 2 ? 2 we yield b1i and b0i j are independent, and that ? 0 follows the like prior as ? , that is, Ga(2, 1. 140). We arrogate a monotonic prior on the intercept, and fool that the rate ratios, exp(? j ), j = 1, . . . , 5, lie among 0. 1 and 10 with chance 0. 95 which gives, use the approach expound in ingredient 4. 1, a normal prior with mean 0 and segmentation 1. 172 . put off 1 gives PQL and INLA summaries for models (5. 15. 3). thither are some differences amidst the PQL and Bayesian an alyses, with slenderly bigger measuring rod asides low the last mentioned, which probably reflects that with m = 59 clusters, a flyspeck true statement is broken when victimisation asymptotic consequence. on that point are some differences in the point estimates which is at least part due to the nonflat priors utilisethe priors turn in comparatively considerable variances, but here the selective information are not so galore(postnominal) so there is sensibility to the prior. reassuringly down the stairs all 3 models inference for the baseline-treatment interaction of interest is virtually y resembling and suggests no stiff treatment effect. We may compare models victimisation log p(y ) for 3 models, we amaze values of ? 674. 8, ? 638. 9, and ? 665. 5, so that the second model is potently preferred. 5. Smoothing of turn in age bracket effects in an age-age bracket model We collapse selective information from Breslow and twenty-four hours (1975) on deprec iator pubic louse order in Iceland. allow Y jk be the number of breast crabby person of cases in age assort j (2024,. . . , 8084) and carry age bracket k (18401849,. . . ,19401949) with j = 1, . . . , J = 13 and k = 1, . . . , K = 11. pursuance Breslow and Clayton (1993), we wear thin Y jk ? jk ? ind Poisson(? jk ) with log ? jk = log n jk + ? j + ? k + vk + u k (5. 5) and where n jk is the person-years denominator, exp(? j ), j = 1, . . . , J , represent refractory effects for age sexual relation encounters, exp(? is the congeneric try associated with a one group adjoin in age bracket group, vk ? iid 406 Y. F ONG AND OTHERS 2 N (0, ? v ) represent unregulated random effects associated with age group k, with smooth cohort scathe u k pursuit a second-order random-effects model with Eu k u i i k = 2u k? 1 ? u k? 2 and Var(u k u i 2 i k) = ? u . This latter(prenominal) model is to allow the evaluate to alter swimmingly with cohort. An equivalent image of this model is, for 2 k K ? 1, 1 Eu k u l l = k = (4u k? 1 + 4u k+1 ? u k? 2 ? u k+2 ), 6 Var(u k u l l = k) = 2 ? . 6 Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University subroutine library on April 20, 2013 The rank of Q in the (4. 1) internal representation of this model is K ? 2 reflecting that both the boilersuit level and the general trim down are aliased (hence the carriage of ? in (5. 5)). The term exp(vk ) reflects the un structured remainder proportional peril and, pastime the argument in prick 4. 2, we specify that this quantity should lie in 0. 5, 2. 0 with probability 0. 95, with a marginal log Cauchy ? 2 distribution, to sire the da Gamma prior ? v ? Ga(0. 5, 0. 00149).The term exp(u k ) reflects the smooth component of the end sexual intercourse risk, and the specification of a 2 prior for the associated variance component ? u is more difficult, assumption its conditional interpretation. utilize the algorithm expound in part 4 . 2, we break downd simulations of u for different choices of da Gamma ? 2 hyperparameters and opinionated on the choice ? u ? Ga(0. 5, 0. 001) recruit 2 shows 10 realizations from the prior. The rationale here is to experiment realizations to see if they adapt to our prior expectations and in particular screening the required amount of smoothing. every(prenominal) but one of the realizations start smoothly crosswise the 11 cohorts, as is desirable. imputable to the tail of the da Gamma distribution, we leave everlastingly cook some constitutional realizations. The INLA results, summarized in pictorial form, are presented in suppose 2(b), on base likelihood fits in which the giving pitch cohort effect is incarnate as a linear term and as a factor. We see that the smoothing model provides a smooth fit in espouse cohort, as we would rely. 5. 3 B-Spline nonparametric atavism We demonstrate the use of INLA for nonparametric smoothing victimisation OSullivan splines, which are based on a B-spline basis.We expand victimization info from Bachrach and others (1999) that concerns longitudinal measurements of spinal anaesthesia anaesthesia tog up mineral density (SBMD) on 230 distaff subjects aged(a) amongst 8 and 27, and of 1 of 4 cultural groups Asian, minatory, Hispanic, and purity. permit yi j denote the SBMD measure for subject i at occasion j, for i = 1, . . . , 230 and j = 1, . . . , n i with n i being in the midst of 1 and 4. epitome 3 shows these entropy, with the color in lines indicating measurements on the aforementioned(prenominal) muliebrity. We assume the model K Yi j = x i ? 1 + agei j ? 2 + k=1 z i jk b1k + b2i + ij, where x i is a 1 ? vector containing an indicator for the sociality of one-on-one i, with ? 1 the associated 4 ? 1 vector of rigid effects, z i jk is the kth basis associated with age, with associated parameter b1k ? 2 2 N (0, ? 1 ), and b2i ? N (0, ? 2 ) are woman-specific random effects, finally, i j ? iid N (0, ? 2 ). All random terms are simulated independent. eyeshade that the spline model is fictional common to all ethnic groups and all women, though it would be straightforward to allow a different spline for each ethnicity. piece of music this model in the form y = x ? + z 1b1 + z 2b 2 + = C ? + . Bayesian GLMMs 407Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University depository library on April 20, 2013 Fig. 2. (a) decade realizations (on the comparative risk scale) from the random effects second-order random walk model in which the prior on the random-effects precision is Ga(0. 5,0. 001), (b) summaries of fitted models the square(a) line corresponds to a log-linear model in birth cohort, the circles to birth cohort as a factor, and + to the Bayesian smoothing model. we use the manner depict in branch 4. 3 to examine the trenchant number of parameters implied by the ? 2 ? 2 priors ? 1 ? Ga(a1 , a2 ) and ? 2 ? Ga(a3 , a4 ).To fit the m odel, we first use the R code provided in sceptre and Ormerod (2008) to take a shit the basis functions, which are then stimulant to the INLA program. caterpillar track the REML reading genuine of the model, we obtain 2 ? = 0. 033 which we use to evaluate the stiff degrees of freedoms associated with priors for ? 1 and 2 . We assume the usual indelicate prior, ? (? 2 ) ? 1/? 2 for ? 2 . later some experimentation, we settled ? 2 408 Y. F ONG AND OTHERS Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University subroutine library on April 20, 2013 Fig. 3. SBMD versus age by ethnicity. measures on the same woman are linked with elderly lines.The solid trim back corresponds to the fitted spline and the dash lines to the psyche fits. ?2 2 on the prior ? 1 ? Ga(0. 5, 5 ? 10? 6 ). For ? 2 , we wished to pull in a 90% breakup for b2i of 0. 3 which, ? 2 with 1 degree of freedom for the marginal distribution, leads to ? 2 ? Ga(0. 5, 0. 00113). symbol 4 shows the priors for ? 1 and ? 2 , on with the implied in force(p) degrees of freedom under(a) the untrue priors. For the spline component, the 90% prior musical interval for the stiff degrees of freedom is 2. 4,10. dodge 2 compares estimates from REML and INLA implementations of the model, and we see close symmetry betwixt the 2. routine 4 too shows the caudal medians for ? 1 and ? 2 and for the 2 in force(p) degrees of freedom. For the spline and random effects these correspond to 8 and 214, respectively. The latter image shows that there is significant disagreement between the 230 women here. This is confirm in Figure 3 where we fall out large tumid differences between the profiles. This sybaritic also shows the fitted spline, which appears to simulate the reduce in the entropy well. 5. 4 Timings For the 3 models in the longitudinal entropy example, INLA takes 1 to 2 s to run, victimisation a single CPU.To get estimates with similar precision with MCMC, we ran JAGS for blow 000 iterations, which took 4 to 6 min. For the model in the temporal smoothing example, INLA takes 45 s to run, victimisation 1 CPU. split up of the INLA procedure can be penalise in a parallel manner. If there are 2 CPUs for sale, as is the case with like a shots prevailing INTEL pith 2 couplet processors, INLA only takes 27 s to run. It is not presently possible to implement this model in JAGS. We ran the MCMC benefit reinforced into the INLA software for 3. 6 trillion iterations, to obtain estimates of comparable accuracy, which took 15 h.For the model in the B-spline nonparametric relapse example, INLA took 5 s to run, use a single CPU. We ran the MCMC service program built into the INLA software for 2. 5 one thousand million iterations to obtain estimates of comparable accuracy, the synopsis taking 40 h. Bayesian GLMMs 409 Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University program library on April 20, 2013 Fig. 4. antecedent summaries (a) ? 1 , the standard deviation of the spline coefficients, (b) hard-hitting degrees of freedom associated with the prior for the spline coefficients, (c) efficient degrees of freedom versus ? , (d) ? 2 , the standard deviation of the between- mortal random effects, (e) utile degrees of freedom associated with the individual random effects, and (f) effective degrees of freedom versus ? 2 . The upended race lines on panels (a), (b), (d), and (e) correspond to the posterior medians. get across 2. REML and INLA summaries for spinal cram entropy. barricade corresponds to Asian group changeable block off bleak Hispanic White be on ? 1 ? 2 ? REML 0. 560 0. 029 0. 106 0. 021 0. 013 0. 022 0. 026 0. 022 0. 021 0. 002 0. 018 0. 109 0. 033 INLA 0. 563 0. 031 0. 106 0. 021 0. 13 0. 022 0. 026 0. 022 0. 021 0. 002 0. 024 0. 006 0. 109 0. 006 0. 033 0. 002 lower For the entries marked with a standard errors were un purchasable. 410 Y. F ONG AND OTHERS 6. D ISCUSSION In this writing, we have demo the use of the INLA computational method for GLMMs. We have found that the approximation strategy utilize by INLA is hi-fi in general, but less accurate for binomial data with clear denominators. The supplementary significant available at Biostatistics online contains an massive simulation study, replicating that presented in Breslow and Clayton (1993).There are some suggestions in the discussion of mourn and others (2009) on how to wee-wee an rectify Gaussian approximation that does not use the mode and the breaking ball at the mode. It is probable that these suggestions will improve the results for binomial data with small denominators. There is an pressing need for diagnosis tools to swag when INLA is inaccurate. Conceptually, computation for nonlinear compound effects models (Davidian and Giltinan, 1995 Pinheiro and Bates, 2000) can also be handled by INLA but this cleverness is not currently availabl e. The website www. r-inla. rg contains all the data and R scripts to perform the analyses and simulations inform in the paper. The latest put out of software to implement INLA can also be found at this site. Recently, Breslow (2005) revisited PQL and think that, PQL clam up performs remarkably well in comparison with more elaborate procedures in galore(postnominal) practical situations. We believe that INLA provides an attractive pick to PQL for GLMMs, and we hope that this paper stimulates the greater use of Bayesian methods for this class. Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University depository library on April 20, 2013S UPPLEMENTARY square appurtenant material is available at http//biostatistics. oxfordjournals. org. recognition negate of amuse none declared. F UNDING field of study Institutes of health (R01 CA095994) to J. W. Statistics for basis (sfi. nr. no) to H. R. R EFERENCES BACHRACH , L. K. , H ASTIE , T. , WANG , M. C. , NARASIMHAN , B. AND M arcus senilis , R. (1999). organize mineral scholarship in brawny Asian, Hispanic, Black and Caucasic youth. A longitudinal study. The diary of clinical Endocrinology and metamorphosis 84, 47024712. B ESAG , J. , G REEN , P. J. , H IGDON , D. AND M ENGERSEN , K. 1995). Bayesian computation and stochastic systems (with discussion). statistical Science 10, 366. B RESLOW, N. E. (2005). Whither PQL? In Lin, D. and Heagerty, P. J. (editors), transactions of the hour Seattle Symposium. unfermented York Springer, pp. 122. B RESLOW, N. E. AND C LAYTON , D. G. (1993). uncut inference in extrapolate linear sundry(a) models. ledger of the American statistical railroad tie 88, 925. B RESLOW, N. E. AND DAY, N. E. (1975). collateral standardisation and multiplicative models for rates, with wing to the age valuation account of crabmeat relative incidence and relative oftenness data.journal of continuing indispositions 28, 289301. C LAYTON , D. G. (199 6). conclude linear interracial models. In Gilks, W. R. , Richardson, S. and Spiegelhalter, D. J. (editors), Markov chain monte Carlo in Practice. capital of the United Kingdom Chapman and dormitory, pp. 275301. Bayesian GLMMs 411 C RAINICEANU , C. M. , D IGGLE , P. J. AND ROWLINGSON , B. (2008). Bayesian compendium for penalized spline statistical retroflection utilise winBUGS. journal of the American statistical sleeper 102, 2137. C RAINICEANU , C. M. , RUPPERT, D. AND billy club , M. P. (2005). Bayesian analysis for penalized spline reverting using winBUGS. diary of statistical software program 14.DAVIDIAN , M. AND G ILTINAN , D. M. (1995). nonlinear Models for reiterate Measurement Data. capital of the United Kingdom Chapman and Hall. D I C ICCIO , T. J. , K tin , R. E. , R AFTERY, A. AND W after partERMAN , L. (1997). deliberation Bayes factors by corporate trust simulation and asymptotic approximations. daybook of the American statistical affiliation 92, 903 915. Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University program library on April 20, 2013 D IGGLE , P. , H EAGERTY, P. , L IANG , K. -Y. Oxford Oxford University Press. AND Z EGER , S. (2002). abstract of longitudinal Data, second edition. FAHRMEIR , L. , K NEIB , T.AND L ANG , S. (2004). Penalized structured additive retroflexion for space-time data a Bayesian perspective. Statistica Sinica 14, 715745. G AMERMAN , D. (1997). ingest from the posterior distribution in conclude linear coalesce models. Statistics and figuring 7, 5768. G ELMAN , A. (2006). foregoing distributions for variance parameters in ranked models. Bayesian analysis 1, 515534. H ASTIE , T. J. AND T IBSHIRANI , R. J. (1990). reason out analog Models. capital of the United Kingdom Chapman and Hall. H OBERT, J. P. AND C ASELLA , G. (1996). The effect of outlawed priors on Gibbs sampling in ranked linear abstruse models. ledger of the American statistical crosstie 91, 1461 1473. K buns , R. E. AND S TEFFEY, D. (1989). hazard Bayesian inference in conditionally independent hierarchic models (parametric falsifiable Bayes models). ledger of the American statistical companionship 84, 717726. K ELSALL , J. E. AND WAKEFIELD , J. C. (1999). word of Bayesian models for spacially correlated ailment and flick data by N. Best, I. Waller, A. Thomas, E. Conlon and R. Arnold. In Bernardo, J. M. , Berger, J. O. , Dawid, A. P. and Smith, A. F. M. (editors), one-sixth Valencia internationalist clashing on Bayesian Statistics. capital of the United Kingdom Oxford University Press.M C C ULLAGH , P. AND N older , J. A. (1989). generalised elongate Models, second edition. capital of the United Kingdom Chapman and Hall. M C C ULLOCH , C. E. , S EARLE , S. R. AND N EUHAUS , J. M. (2008). generalize, running(a), and flux Models, second edition. sassy York illusion Wiley and Sons. M ENG , X. AND W ONG , W. (1996). Simulating ratios of normalizing constants via a round-eyed identity. statistical Sinica 6, 831860. NATARAJAN , R. AND K ASS , R. E. (2000). advert Bayesian methods for generalized linear mixed models. ledger of the American statistical tie-up 95, 227237. N old , J. AND W EDDERBURN , R. (1972). conclude linear models.journal of the olympian statistical Society, serial A 135, 370384. P INHEIRO , J. C. AND BATES , D. M. (2000). motley-Effects Models in S and S-plus. parvenue York Springer. P LUMMER , M. (2008). Penalized button functions for Bayesian model comparison. Biostatistics 9, 523539. P LUMMER , M. (2009). Jags version 1. 0. 3 manual. skilful Report. regret , H. AND H ELD , L. (2005). Gaussian Markov haphazard field Thoery and Application. Boca Raton Chapman and Hall/CRC. feel , H. , M ARTINO , S. AND C HOPIN , N. (2009). near Bayesian inference for latent Gaussian models using co-ordinated nested laplace approximations (with discussion).journal of the princely statistical Society, series B 71, 3193 92. 412 RUPPERT, D. R. , verge , M. P. University Press. AND Y. F ONG AND OTHERS C ARROLL , R. J. (2003). Semiparametric Regression. saucily York Cambridge S KENE , A. M. AND WAKEFIELD , J. C. (1990). hierarchal models for multi-centre binary response studies. Statistics in practice of medicine 9, 919929. S PIEGELHALTER , D. , B EST, N. , C ARLIN , B. AND van DER L INDE , A. (1998). Bayesian measures of model complexity and fit (with discussion). Journal of the purplish statistical Society, serial publication B 64, 583639. S PIEGELHALTER , D. J. , T HOMAS , A.AND B EST, N. G. (1998). WinBUGS user Manual. recital 1. 1. 1. Cambridge. T dormitory room , P. F. AND VAIL , S. C. (1990). virtually covariance models for longitudinal count data with overdispersion. biometrics 46, 657671. V ERBEKE , G. V ERBEKE , G. AND AND Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University subroutine library on April 20, 2013 M OLENBERGHS , G. (2000). Linear Mixed Mode ls for longitudinal Data. bracing York Springer. M OLENBERGHS , G. (2005). Models for clear-cut longitudinal Data. peeled York Springer. WAKEFIELD , J. C. (2007). Disease function and spatial regress with count data.Biostatistics 8, 158183. WAKEFIELD , J. C. (2009). Multi-level modelling, the bionomical fallacy, and cross study designs. global Journal of Epidemiology 38, 330336. scepter , M. P. AND O RMEROD , J. T. (2008). On semiparametric regression with OSullivan penalised splines. Australian and impertinently Zealand Journal of Statistics 50, 179198. Z EGER , S. L. AND K ARIM , M. R. (1991). Generalized linear models with random effects a Gibbs sampling approach. Journal of the American statistical connection 86, 7986. Received kinfolk 4, 2009 rewrite November 4, 2009 trustworthy for publication November 6, 2009