in All availableThis forumThis topic
 Automatic Differentiation
Author Message
[n00b@localhost]
Apprentice

Joined: 30 Aug 2004
Posts: 266
Location: London, UK

 Posted: Sun Aug 25, 2013 11:46 pm    Post subject: Automatic Differentiation I am writing a small example program using automatic differentiation via operator overloading. The program requires the 1st and 2nd order derivatives of the log of the probability density functions of the uniform, normal, lognormal and gamma distributions. I have unit tests that compare my log PDF function with the log of the PDF from another library I am assuming to be correct. I then take the source for the log PDF function and run it through an online derivative calculator to give me the formula for the 1st and 2nd order derivatives of the log PDFs. This gives me the basis for unit testing my log PDF function using my automatic derivative type. All tests comparing my log PDF with the log of the PDF from the standard library pass. Using the automatic derivative type the tests for the uniform, normal and gamma distribution pass but the 2nd derivative of the lognormal distribution fails. The log PDF of the normal distribution is: logpdf(x, mu, sigma) = -0.5 * log(2 * M_PI) - log(sigma) - (x - mu)*(x - mu)/(2 * sigma * sigma) while the log PDF for the lognormal distribution is: logpdf(x, mu, sigma) = -0.5 * log(2 * M_PI) - log(sigma) - log(x) - (log(x) - mu)*(log(x) - mu)/(2 * sigma * sigma) The only additional operator needed to evaluate the derivatives of the lognormal PDF as well as the normal PDF is an overloaded log() operator. I calculated the 1st and 2nd order derivatives shown below but I'm not sure they're correct. Could someone with a better understanding of calculus please check them? h(x) = log(g(x)) (use log rule and chain rule) h'(x) = g'(x) / g(x) (use quotient rule) h''(x) = (g''(x)g(x) - g'(x)g'(x)) / (g(x)g(x)) I also have a feeling my 2nd derivative divide operator may not be correct. I attempted to derive the 2nd derivative of the quotient rule myself but this wasn't giving correct answers for the unit tests. I looked up the 2nd derivative of the quotient rule on Wikipedia and it gives the answers I expect but doesn't show the derivation. Could someone show me how this is derived? Thanks!
Akkara
Administrator

Joined: 28 Mar 2006
Posts: 6393
Location: &akkara

Posted: Sun Aug 25, 2013 11:59 pm    Post subject:

 Quote: I attempted to derive the 2nd derivative of the quotient rule myself [...but couldn't...] Could someone show me how this is derived?

Write as product: D[n(x)/d(x)] == D[n(x) * (1/d(x))]

Use product rule: == n(x) * D[1/d(x)] + (1/d(x)) * D[n(x)]

Use power rule and chain rule for D[1/d(x)]: == -1/((d(x))^2) * D[d(x)]

Substitute for D[1/d(x)]: D[n(x)/d(x)] == n(x) * (-1/((d(x))^2) * D[d(x)]) + (1/d(x)) * D[n(x)]

Use ' notation: n(x) * (-1/((d(x))^2) * d'(x) + (1/d(x)) * n'(x)

Factor out a 1/d(x)^2 == (n(x) * (-1) * d'(x) + d(x) * n'(x)) / (d(x))^2

Rewrite: (d(x) * n'(x) - n(x) * d'(x)) / (d(x))^2
[n00b@localhost]
Apprentice

Joined: 30 Aug 2004
Posts: 266
Location: London, UK

 Posted: Mon Aug 26, 2013 9:56 am    Post subject: Is that the 2nd order derivative of the quotient rule? Why is it different to the one on Wikipedia? Also, what about the 2nd order log rule? Is that correct?
Akkara
Administrator

Joined: 28 Mar 2006
Posts: 6393
Location: &akkara

 Posted: Mon Aug 26, 2013 12:14 pm    Post subject: Oops! That's the regular 1st order. Didn't notice you wanted 2nd. To get it, run it thru again, similar procedure, longer and more complicated expression. It'll be messy. But you'll get your answer. The 2nd order log rule, same thing. Write two "D" operators and peel the onion: D[ D[ ln(x) ] ] == D[ 1/x ] == -1/x^2 If you meant 2nd order of ln(f(x)), then it's chain rule and log rule. Twice. And one application of the quotient rule since the 1st derivative has a 1/f(x) in it.
 Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First
 All times are GMT Page 1 of 1

 Jump to: Select a forum Assistance----------------News & AnnouncementsFrequently Asked QuestionsInstalling GentooMultimediaDesktop EnvironmentsNetworking & SecurityKernel & HardwarePortage & ProgrammingGamers & PlayersOther Things GentooUnsupported Software Discussion & Documentation----------------Documentation, Tips & TricksGentoo ChatGentoo Forums FeedbackOff the WallDuplicate Threads International Gentoo Users----------------中文 (Chinese)DutchFinnishFrenchDeutsches Forum (German)  Diskussionsforum  Deutsche DokumentationGreekForum italiano (Italian)  Forum di discussione italiano  Risorse italiane (documentazione e tools)Polskie forum (Polish)  Instalacja i sprzęt  Polish OTWPortuguese  Documentação, Ferramentas e DicasRussianScandinavianSpanishOther Languages Architectures & Platforms----------------Gentoo on AMD64Gentoo on ARMGentoo on PPCGentoo on SparcGentoo on Alternative ArchitecturesGentoo for Mac OS X (Portage for Mac OS X)
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

 Copyright 2001-2018 Gentoo Foundation, Inc. Designed by Kyle Manna © 2003; Style derived from original subSilver theme. | Hosting by Gossamer Threads Inc. © | Powered by phpBB 2.0.23-gentoo-p11 © 2001, 2002 phpBB Group