#| 

  IQ over years in Racket/Gamble 

  For some reason (*) I got interested in the following problem
  which may - or may not - have some background in real life.

  A person at age A (say at 19) got an IQ score of IQ (say 126).
  What is the probability that A has the same IQ score at a later 
  age (say 68).

  Or: A did a test at age 19 that gave a Stanine score 
  (https://en.wikipedia.org/wiki/Stanine ) of 9 (of 9),
  which represents > 126. What is the probability that
  A at age 68 has a score at least 127, and what are the statistics
  of that score.


  Here we assume - by some calculation based on typical values from 
  long-term IQ stability studies (e.g., Scottish Mental Survey, Dunedin Study),
  that the IQ score declines per year according to "normal-drif" of 
  normal(-0.175,1.7) (which was calculated in collaboration with ChatGPT 4o).
  This means there is actually some possibility of actual raising the IQ 
  scores per year.

  Below are some different scenarios.

  First, the result with only this normal-dift (use-total-change #f) and 
  a fix IQ score at age 19 of 126.

variable : last-year-iq
111.74388584163005: 0.00010000000000000938
100.50161253573263: 0.00010000000000000938
118.81603734944788: 0.00010000000000000938
134.78756036446543: 0.00010000000000000938
123.9017767906441: 0.00010000000000000938
...
113.54156191193381: 0.00010000000000000938
109.72733675361474: 0.00010000000000000938
121.91289530430694: 0.00010000000000000938
140.94588234742193: 0.00010000000000000938
101.51113482259984: 0.00010000000000000938
mean: 117.46375997309865
HPD interval (0.5): 109.44023082213705..125.33087375435574
HPD interval (0.75): 102.58751988281114..129.59812555266058
HPD interval (0.84): 101.25011926599578..133.87909198563995
HPD interval (0.95): 95.63510326857154..140.8923528521379
HPD interval (0.99): 87.4779455898522..147.00424580074073
HPD interval (0.999): 77.76095561376495..154.8894628882916

variable : last year == start-iq
#f: 0.8949999999999578
#t: 0.10500000000000732
mean: 0.10500000000000732

variable : last-year > start-iq
#f: 0.7650999999999721
#t: 0.2349000000000263
mean: 0.2349000000000263

variable : last-year >= 132
#f: 0.8967999999999576
#t: 0.10320000000000727
mean: 0.10320000000000727

variable : min iq
98.91880875564517: 0.00010000000000000938
100.50161253573263: 0.00010000000000000938
101.14834462057858: 0.00010000000000000938
100.60061627726083: 0.00010000000000000938
102.94718810851506: 0.00010000000000000938
...
101.26321086902603: 0.00010000000000000938
99.65729987847551: 0.00010000000000000938
91.5644875855826: 0.00010000000000000938
103.52821281960003: 0.00010000000000000938
105.54764684425754: 0.00010000000000000938
mean: 98.81716562538998
HPD interval (0.5): 97.12710682804403..104.13160802005173
HPD interval (0.75): 93.89532734214028..106.04060926576622
HPD interval (0.84): 91.89912272251922..107.02622351654212
HPD interval (0.95): 87.9057871374556..109.20731670127128
HPD interval (0.99): 82.6855340023095..110.95229891516342
HPD interval (0.999): 75.98129403117375..113.33851154428586

variable : mean iq
124.06642589635996: 0.00010000000000000938
119.3792446294079: 0.00010000000000000938
121.10573132920959: 0.00010000000000000938
121.35316445692126: 0.00010000000000000938
120.3657777781639: 0.00010000000000000938
...
120.05231073193404: 0.00010000000000000938
121.2496197300287: 0.00010000000000000938
120.93448108342503: 0.00010000000000000938
119.95519365337378: 0.00010000000000000938
121.17885985184634: 0.00010000000000000938
mean: 121.79617487943884
HPD interval (0.5): 120.99441012433952..122.60167813565042
HPD interval (0.75): 120.49993959969608..123.2343873306845
HPD interval (0.84): 120.24016914349433..123.56811254091012
HPD interval (0.95): 119.5553254889283..124.22185858447556
HPD interval (0.99): 118.8292316590145..124.89409857438943
HPD interval (0.999): 117.91925476559014..125.94660405864488

variable : max iq
141.12420923340008: 0.00010000000000000938
137.12876784008392: 0.00010000000000000938
144.54466412168608: 0.00010000000000000938
143.93125091024083: 0.00010000000000000938
141.63206502062113: 0.00010000000000000938
...
146.8227563535882: 0.00010000000000000938
153.50726213295883: 0.00010000000000000938
135.52121416207126: 0.00010000000000000938
138.9243320800094: 0.00010000000000000938
134.98110779062733: 0.00010000000000000938
mean: 140.51649367085113
HPD interval (0.5): 136.209049124956..142.00029844446956
HPD interval (0.75): 134.51610814465968..144.48005254787265
HPD interval (0.84): 133.59021158200792..145.8296250611723
HPD interval (0.95): 132.39592011688885..150.03952714821065
HPD interval (0.99): 130.9789406216636..155.19907466324491
HPD interval (0.999): 129.29723271684512..162.05556354376182


  But it tends to be more complex than that since life occasionally 
  happens. So let's add two events:
  - probability that a negative event happen
    Such as cancer diagnosis, chemotherapy or some other sickness/events
    of self or of relatives. Or loss of job etc.
    Let's make this a probability of 0.02 that such negative event happen,
    and that the score decline for that year is (max 0 (normal 2 1))

  - probability that a positive event happen
    For example getting a new IQ boosting interest or some other happy things 
    that boosts the IQ score.
    Let's make this a probability of 0.01 that such a positive event happen,
    and that the score raises for that year is (max 0 (normal 2 1))
    
  Interestingly, the mean value of score at 68 is then 116.38 (vs 117.46 for just 
  the normal-drift version) which is perhaps a little surprising result (assuming that all
  assumptions that is made are reasonable).
 
variable : last-year-iq
98.33409276333869: 0.0009999999999999994
120.40644204101497: 0.0009999999999999994
125.41393736696293: 0.0009999999999999994
109.09168639732198: 0.0009999999999999994
120.99901803919258: 0.0009999999999999994
...
93.86864441781299: 0.0009999999999999994
96.71021336078809: 0.0009999999999999994
97.68901277204336: 0.0009999999999999994
136.78005172086557: 0.0009999999999999994
123.32913901698275: 0.0009999999999999994
mean: 116.3824009408219
HPD interval (0.5): 109.51752372020213..124.95300926590957
HPD interval (0.75): 102.28952213561831..130.29992548482244
HPD interval (0.84): 99.66658924011207..134.11253038650653
HPD interval (0.95): 92.07716547968498..141.74894355450584
HPD interval (0.99): 84.10173239244001..146.39047316570034
HPD interval (0.999): 83.11332825966221..157.31818230991755

variable : last year == start-iq
#f: 0.9020000000000007
#t: 0.09800000000000006
mean: 0.09800000000000006

variable : last-year > start-iq
#f: 0.8090000000000006
#t: 0.19100000000000014
mean: 0.19100000000000014

variable : last-year >= 132
#f: 0.9070000000000007
#t: 0.09300000000000005
mean: 0.09300000000000005

variable : min iq
100.69405232017839: 0.0009999999999999994
98.71544356452998: 0.0009999999999999994
105.23928881320695: 0.0009999999999999994
87.6252316594026: 0.0009999999999999994
93.19985231003297: 0.0009999999999999994
...
96.71021336078809: 0.0009999999999999994
97.47068473031733: 0.0009999999999999994
88.0419100922028: 0.0009999999999999994
97.49981642918613: 0.0009999999999999994
100.73198218978567: 0.0009999999999999994
mean: 97.52708619024158
HPD interval (0.5): 95.4831644533238..102.59428987712509
HPD interval (0.75): 91.60433858564814..104.24365431283334
HPD interval (0.84): 89.66964177991403..105.16055557378945
HPD interval (0.95): 85.94710760491228..107.47430946930099
HPD interval (0.99): 80.04315207151197..109.00694350092681
HPD interval (0.999): 74.02523921124052..113.85124076426901

variable : mean iq
120.13844554313314: 0.0009999999999999994
122.23036951346153: 0.0009999999999999994
119.85937548241503: 0.0009999999999999994
121.80414009112579: 0.0009999999999999994
122.6725591002927: 0.0009999999999999994
...
119.69339940925231: 0.0009999999999999994
121.34134780966043: 0.0009999999999999994
120.11523963341698: 0.0009999999999999994
121.1137619107923: 0.0009999999999999994
120.44635212551482: 0.0009999999999999994
mean: 121.26873448047523
HPD interval (0.5): 120.61633437451063..122.26580872680879
HPD interval (0.75): 119.87178486197337..122.73251144531413
HPD interval (0.84): 119.50614388834978..122.91998073385314
HPD interval (0.95): 119.05639990008301..123.7184609987559
HPD interval (0.99): 118.1535341855641..124.55716249315743
HPD interval (0.999): 117.24225652389941..124.95871645406794

variable : median iq
123.65400308305436: 0.0009999999999999994
122.79882405682105: 0.0009999999999999994
121.6401025835172: 0.0009999999999999994
124.50395550453634: 0.0009999999999999994
120.99901803919258: 0.0009999999999999994
...
120.4487433846775: 0.0009999999999999994
124.98801917556281: 0.0009999999999999994
123.410566110972: 0.0009999999999999994
122.2300020124593: 0.0009999999999999994
120.99697643307418: 0.0009999999999999994
mean: 122.41385022754747
HPD interval (0.5): 121.80296054465461..123.61272114893414
HPD interval (0.75): 120.62211897049403..123.80715433458444
HPD interval (0.84): 120.56183503739538..124.42690994491282
HPD interval (0.95): 119.81241002523605..125.21417211215203
HPD interval (0.99): 118.6069252398479..125.591822005197
HPD interval (0.999): 117.53157279411866..126

variable : max iq
137.08263967204124: 0.0009999999999999994
134.05688908983987: 0.0009999999999999994
151.95397824518304: 0.0009999999999999994
141.4581363018161: 0.0009999999999999994
138.47381626080573: 0.0009999999999999994
...
134.37070812824487: 0.0009999999999999994
139.6449389109829: 0.0009999999999999994
136.44487115519644: 0.0009999999999999994
139.322307248204: 0.0009999999999999994
136.78005172086557: 0.0009999999999999994
mean: 140.26236390199503
HPD interval (0.5): 136.03041360403319..141.60131401894043
HPD interval (0.75): 134.54813750673225..144.39613105418772
HPD interval (0.84): 133.53012491121868..145.8305190006272
HPD interval (0.95): 132.47594533157442..150.2840087776638
HPD interval (0.99): 130.54125913167806..155.23881223305705
HPD interval (0.999): 128.8986204568194..167.84886486173917

variable : mean-diffs
-0.021462316713824297: 0.0009999999999999994
-0.662264326510873: 0.0009999999999999994
0.3192248403933918: 0.0009999999999999994
-0.16952364899487696: 0.0009999999999999994
-0.2460407050463115: 0.0009999999999999994
...
-0.125756599847248: 0.0009999999999999994
-0.029781741861350557: 0.0009999999999999994
-0.05700620101485191: 0.0009999999999999994
-0.14532596222412467: 0.0009999999999999994
-0.2527390854480804: 0.0009999999999999994
mean: -0.200366647066205
HPD interval (0.5): -0.34338492249578884..-0.02181230696021726
HPD interval (0.75): -0.49396828884128513..0.0895817809338008
HPD interval (0.84): -0.5486127241643318..0.16901104971888614
HPD interval (0.95): -0.7067257191732296..0.3281029907188717
HPD interval (0.99): -0.8728805751574997..0.4248015242854238
HPD interval (0.999): -0.8934723279237039..0.6524621314566156

  As noted above a Stanine score of 9/9 actually means that the score is > 126,
  so let's adjust for that by making start-iq (at age 19) a random variable
  with the conditional probability
    (normal 100 15) > 126

 Then the mean value of start iq is 132.19. 
 The mean score at age 68 would then be 123.07 (HPD at 0.84: 106.73..142.05).

 Below is the output of the full model including the total change. 

variable : start iq
137.20404869223762: 0.0009999999999999994
130.66606019215263: 0.0009999999999999994
134.6356610721391: 0.0009999999999999994
126.65607313718517: 0.0009999999999999994
127.46624564650769: 0.0009999999999999994
...
127.0242648180282: 0.0009999999999999994
129.8846569104556: 0.0009999999999999994
130.82402040026804: 0.0009999999999999994
128.96127824379178: 0.0009999999999999994
134.21707106247501: 0.0009999999999999994
mean: 132.18555462368187
HPD interval (0.5): 126.0783415229412..130.7172654211975
HPD interval (0.75): 126.00797871561116..134.72403707424542
HPD interval (0.84): 126.00295677434333..136.82949672818603
HPD interval (0.95): 126.00295677434333..142.0788096011719
HPD interval (0.99): 126.00295677434333..148.0848193165378
HPD interval (0.999): 126.00295677434333..155.1162509694568

variable : last-year-iq
129.06126423796525: 0.0009999999999999994
120.0429792640847: 0.0009999999999999994
147.54762312043138: 0.0009999999999999994
115.32995630183596: 0.0009999999999999994
109.15281693268399: 0.0009999999999999994
...
136.67667717155285: 0.0009999999999999994
115.8603567128148: 0.0009999999999999994
144.87660501255425: 0.0009999999999999994
128.2576084461535: 0.0009999999999999994
133.93917914298677: 0.0009999999999999994
mean: 123.07161400799365
HPD interval (0.5): 113.53282427784644..129.76730460172658
HPD interval (0.75): 110.04616321827451..138.0887763282543
HPD interval (0.84): 106.72291882032988..142.0528196789997
HPD interval (0.95): 98.52781151164969..147.99329600867077
HPD interval (0.99): 89.76463273233382..155.5474028005989
HPD interval (0.999): 78.49502157136945..166.2420460971493

variable : last year == start-iq
#f: 0.9000000000000007
#t: 0.10000000000000006
mean: 0.10000000000000006

variable : last-year > start-iq
#f: 0.7700000000000006
#t: 0.23000000000000018
mean: 0.23000000000000018

variable : last-year >= 132
#f: 0.7520000000000006
#t: 0.2480000000000002
mean: 0.2480000000000002

variable : min iq
97.41643716533649: 0.0009999999999999994
98.85326316733443: 0.0009999999999999994
96.51760601352971: 0.0009999999999999994
96.07314539461228: 0.0009999999999999994
82.77888233915537: 0.0009999999999999994
...
109.4289235746051: 0.0009999999999999994
99.4519486284054: 0.0009999999999999994
106.10208537025737: 0.0009999999999999994
99.8892352233763: 0.0009999999999999994
117.03452050469738: 0.0009999999999999994
mean: 104.01566955699961
HPD interval (0.5): 98.98680137826584..108.70353374795747
HPD interval (0.75): 95.57808268582909..112.19983465849357
HPD interval (0.84): 93.53565160420595..114.09212261831757
HPD interval (0.95): 90.4260330524443..120.32631099300426
HPD interval (0.99): 80.68357738177653..123.81236421358997
HPD interval (0.999): 77.00351726359285..133.8454648530093

variable : mean iq
131.25689566296614: 0.0009999999999999994
125.45621932079622: 0.0009999999999999994
125.62700981973389: 0.0009999999999999994
130.43331738783365: 0.0009999999999999994
124.5317125010463: 0.0009999999999999994
...
124.3930380429914: 0.0009999999999999994
121.20121513263234: 0.0009999999999999994
124.331621539698: 0.0009999999999999994
136.5304144500486: 0.0009999999999999994
127.2709132311884: 0.0009999999999999994
mean: 127.55186648078713
HPD interval (0.5): 121.85630007603554..127.47938876033952
HPD interval (0.75): 120.56137837885825..130.94869868607915
HPD interval (0.84): 119.95881611195354..132.78225997823068
HPD interval (0.95): 119.84481885372102..138.05617126886102
HPD interval (0.99): 118.70822466159018..143.29664932438314
HPD interval (0.999): 118.259130205859..152.21583567763307

variable : median iq
131.25763629727172: 0.0009999999999999994
124.08873361305187: 0.0009999999999999994
126.075941848096: 0.0009999999999999994
127.14997837783211: 0.0009999999999999994
124.29343934206516: 0.0009999999999999994
...
121.328824621934: 0.0009999999999999994
126.9881239006401: 0.0009999999999999994
122.73635974394327: 0.0009999999999999994
122.22888640237895: 0.0009999999999999994
125.41629307849605: 0.0009999999999999994
mean: 128.68456562161052
HPD interval (0.5): 122.32627896380433..128.1127531510519
HPD interval (0.75): 121.43655889424267..131.87030591732506
HPD interval (0.84): 121.43655889424267..134.15074408657657
HPD interval (0.95): 120.53330488443545..138.95596429983664
HPD interval (0.99): 119.18157052263282..144.74717701731188
HPD interval (0.999): 119.18157052263282..153.5435199516953

variable : max iq
153.55006546595612: 0.0009999999999999994
141.01009897817772: 0.0009999999999999994
146.57954851811087: 0.0009999999999999994
136.25518989153835: 0.0009999999999999994
169.48712659049735: 0.0009999999999999994
...
140.46398349996497: 0.0009999999999999994
147.7926296996059: 0.0009999999999999994
136.02282565838186: 0.0009999999999999994
157.7278934863969: 0.0009999999999999994
146.45723089084933: 0.0009999999999999994
mean: 146.4315720336767
HPD interval (0.5): 139.25960199236627..148.03255954932942
HPD interval (0.75): 137.6400012892779..153.21998582083046
HPD interval (0.84): 136.2140248731721..155.1722121375034
HPD interval (0.95): 133.17625356390542..159.42735431353924
HPD interval (0.99): 131.24419373995525..166.31118988025403
HPD interval (0.999): 131.04494531603447..178.91281053416853

variable : mean-diffs
-0.20585782221258242: 0.0009999999999999994
-0.9592769632114816: 0.0009999999999999994
-0.34199824887137115: 0.0009999999999999994
-0.09736310443873257: 0.0009999999999999994
-0.1386999421656805: 0.0009999999999999994
...
-0.17713899958664303: 0.0009999999999999994
-0.06000507446834469: 0.0009999999999999994
-0.39376261452401007: 0.0009999999999999994
-0.14917245062436515: 0.0009999999999999994
-0.06709158997588638: 0.0009999999999999994
mean: -0.1898737628268417
HPD interval (0.5): -0.3126460081931661..-0.009611729753967957
HPD interval (0.75): -0.48106223978490803..0.07318554588466182
HPD interval (0.84): -0.47154269945795324..0.19404804650447302
HPD interval (0.95): -0.623106965252837..0.3546842258053218
HPD interval (0.99): -0.8404344941177376..0.43661447811079235
HPD interval (0.999): -1.0571763104993408..0.7585189246136798


  (*) One part of the interest in this problem is that I realized 
  (or rather contemplated about) that the IQ scores are age normed so 
  a person's score at age 19 is not really comparable to the score 
  at later year, say about 50 years later . The age score are only 
  comparable for the same age (age group). And, the IQ scores 
  - before age norming - tends to decline over the year, but
  the average for each age (group) after age norming is - per definition - 
  100.
  

  This program was created by Hakan Kjellerstrand, hakank@gmail.com
  See also my Racket page: http://www.hakank.org/racket/

|#

#lang gamble

; (require gamble/viz)
(require racket)
(require "gamble_utils.rkt")
; (require "gamble_distributions.rkt")


(define (model)
  (; enumerate
   ; rejection-sampler
   importance-sampler
   ; mh-sampler

   (define start-age 19)
   (define end-age 68)

   ; Let's first assume that the IQ score
   ; was "exactly" 126
   ; (define start-iq 126)
   ; The Stanine score of 9/9 actually mean > 126
   
   (define start-iq (normal 100 15))
   (observe/fail (> start-iq 126))
   
   
   (define use-total-change #t)
   ; (define use-total-change #f)   
   
   ;; (define (yearly-change)
   ;;   (if (sample (bernoulli 0.03))
   ;;       (* -1 (max 0 (sample (normal 4 3))))
   ;;       (sample (normal -0.175 1.7)))) ; baseline drift

   (define (normal-drift)
     (normal -0.175 1.7))

   (define (negative-event-impact)
     (if (flip 0.02)  ; 2% chance of positive event
         (max 0 (normal 2 1)) ; typical impact: +1 to +4
         0))
   
   (define (positive-event-impact)
     (if (flip 0.01)  ; 2% chance of positive event
         (max 0 (normal 2 1)) ; typical impact: +1 to +4
         0))

   (define (total-change)
     (+ (normal-drift) (- (negative-event-impact)) (positive-event-impact)))
   
   ; IQ at age end
   (define (iq end)
     (define (f age score)
       (if (>= age end)
           score
           ; (let ([inc (normal -0.175 1.7)])
           (let ([inc (if use-total-change (total-change) (normal-drift))])
             (f (add1 age) (+ score inc))))
       )
     (f start-age start-iq)
     )

   (define iqs (for/list ([age (range start-age end-age)]) (iq age)))
   ; (show2 "iqs" iqs)
   ; (show2 "diffs" (differences iqs))
   
   (define last-year-iq (last iqs))

   ; (observe/fail (> last-year-iq 132))
   
   (define min-iq (min-list iqs))
   (define mean-iq (avg iqs))
   (define median-iq (median iqs))         
   (define max-iq (max-list iqs))
   (define mean-diffs (avg (differences iqs)))
   
   (define p-126 (<= (abs (- last-year-iq start-iq)) 2))
   (define p-gt-126 (> last-year-iq start-iq))
   (define p-ge-132 (> last-year-iq 132))      
   
   (list ; iqs
         start-iq
         last-year-iq
         p-126
         p-gt-126
         p-ge-132
         min-iq
         mean-iq
         median-iq
         max-iq
         mean-diffs
         )

   )
)

(show-marginals (model)
                (list  ; "iqs"
                       "start iq"
                       "last-year-iq"
                       "last year == start-iq"
                       "last-year > start-iq"
                       "last-year >= 132"
                       "min iq"
                       "mean iq"
                       "median iq"
                       "max iq"
                       "mean-diffs"
                       
                     )
                    #:num-samples 1000
                    #:truncate-output 5
                    ; #:skip-marginals? #t
                    ; #:show-stats? #t
                    ; #:credible-interval 0.84
                    #:hpd-interval (list 0.5 0.75 0.84 0.95 0.99 0.999)
                    ; #:show-histogram? #t
                    ; #:show-percentiles? #t
                    ; #:burn 0
                    ; #:thin 0
                    )