.lH .. 5.... 2%.: . .o. . . 2.3.3.3....r0: .2 1?]!‘4942 : . . c . . ... \ .. ... ... .. ...: to. L 41 _ .. . “a. 231?.” .0.th (...-“.1...‘ v c. -. .34! a u.J..(fi«.fii.‘..-i.. . L... .13.! .31.... a... f! 3..., ...m. o.. ,Hil... .. .. I .... V v. . c.5373... ..."... or. ... . . . . .v I ll. .. . .3 .....4. . ...-.12! .57.. 0...... ...“ .1 'Jflflnfiuuvc 0 on...» , _ .. 1?... .10.... who-HM... ..t. «I. .L......f..1ll. -. .- 0' I. .o O . g. . . . . . . vat-16$... .. ti”... o, .0 1. I‘d-‘4‘? ...v .... 21‘?- o. ..t. .3" 6:19.... 99? . ...... 11...?! q. . a "7'. Pt. 0. .0 a O... V . t 10-....31. .. c. ‘9'...- . . ...7. .o.' .... .o . 31.. $0-! . .I "Q‘O... .... q u. a. 1. . . ... . o. . Q... .o ... ‘93“...3399 ... ..1. . a0 . {...}... 1.... t J ....t in“ : . .... 3.10....- 0:... c2498... L . ... Unit . 49.38. 3.) .0, o. v . . o... .....1... ...... .....VI. 2 ...... . . .... ... .. 9.. Utixo’ bar cog-...)...u- o. [.72. ...... .. .. . n I... ...-.....c; «- “v.0 . C . . .c .... . o. .L; ...... . n p . ...-54.3.”: 1 . .... 2.... .0.-. 7.6.3....Y ....c o....!.!. u \. 9.. .0. .. ... 60 3. ...... r. .... . 2o .0 17...... . . fl... . I . .. . . "if... . , . f ...... ._ p.435 £01.. ..a..0 . ,. .... . . ......v: nlSc....!..c...!'J.. .0 . ..1. ....a.§ ._ :33 ... , . , . I‘Oail ... 1.114.. “no: fat-... . .0 I... 7,. I! ....qu 5. H}: ... h . V 2'53. . . . .. a .... ......U.J.J.tb. .....xsfifl. J..:.Hl.m ...".n... ..u 5.0.2:? Lem... .V . . . ... . 4.. . ...»...us1rozfinuct ...-I2“: . . .l . . . .u. ,. . .o . .....o. 5|.- 1 . . ... . o. o, C .‘d: F \WOU‘Lum 6.1.5 .4"... .0 Qt! . ... Dus"l-. . 0.5%.. ..vtov . u . . ... . .0. V. . I... 0.... n.‘ . .Y......n I 4‘!!! v. 1.... ... . , A ...J .5...’ o ..T‘ a 3... o o 0‘ g i... ... . . _ .... 0.. . .... .... . . ...-5h“... .19... ’evo.t$ . . é 30:13.6: .... .416. . (...... ...)v...:Vb.. .1; ... . ... a 0.... ...... .... .. . 3.19.... .... ...... t ... ...-... 1...?!» .b ... . . , ..HJ. J v. 0 . . 010. “38 .01. $v.v:.'..o.. . fi .2. .....34 .... cl: :1. . . ..vocov .. ...... .. ...... ...”..r. ...-... I ... . .v.lo:. . .z . . ...... . . ... ......o.f.1. .....‘J a ... 1...... . . . . tr . c .. . . ;7\.‘r0|. ’p 13‘4“qu ftp-......Q '00; :3- J 5... 10¢ . o b . . .- noguurr‘..wt..11’... It. ...:‘UI..E’I1 . o ...l .00! a a .1... .a .I,’d.“. ....‘Orh. ..‘ .. C... ...-c x .11‘! .‘\.. o ,0.5 ... . , . Jwt‘H _| m; . .... . ... .r 9!.» .... 3‘03; ‘ l. . ... ‘3. r»..- 3. tt‘..lo:....o.. ... a... 9 : ......- .!....o. . .‘I . .. $0.... .....3....‘.‘.. u .0. ... . v a. .83.. .... ...f on. .... ... ... ... 391...... .o a. . . o . . . . . ......iwfiflrl u fikooa . V. v.8. .. . ... Il.o..k.¢o.(...l.lc .o ...-u .. ... ..Qlo. 29.. . .. .i: -... (I. ..v .Il..ld ...o. . ... . ... Z . .. .I . ... o..§t.. 4:. . ...! ....3..:, . .t » .... ... S e c. It» ..I t,.. . ”pal... ...) .....3. 1.6... u o... .1: .... .. .... . 691.1...27 a... ....u....... .. . .... . I ......7 .. t, 3...... .. . . . ... 3... ...... ... . 9313.1 . ... ..., x .. . . . , , . . 3.... ...UI ~ «(.1 ....t’ln. ...-‘5 o pouroiho. .. ..f.o...u.tIa.. . . . . .. . . - ... ....ots: . m. . ... .. .- ...‘u. ...! .x.. o. ....n... ...... .. $.39! .... .3 ....- .‘soofl......>. ... e .. . ,. . ,. ....” w v. 3 L...) ... u 1). .tflzzlvl... to..(..vt3.9. 4..)3‘2I83.» ......4. ... . . .... 3“.o.!..«.r§ .... .. .u. ...I ‘8 ,:.II.... ......T, .... .l...’ .. .... , . . .I ... .w .u of“. . $0.on ... ..f... C. . . , . w? . 3.1.47.5, .. . 1...... .. . in ...s. . .5323 4 11...... 1...... 1.....J 1 . ...... ...... . . . . 0,. no ...} .. ... . 1. V 20...... .. . . .. . . .... 33.4.3. . . ...-I... ...-.857...’ :1)... o t r .. . . w .V... N ...... 4.4.1.6... ’ E. . .1 . .... . ... c . ”...... 3| 4...... ...w‘? o .. . O ‘nrnocluloo. ....=P2-p._..o ...... .... ‘3‘ l .0... “coo... V ...A o. 23.2:- 031 \...~‘O‘L . ....r .1. ~ .K. 7 ¢ .l.\. T ..3. {I 1., . :3 r 34...... .6 r u"... o. r In {5.9.8. .....x.t...l .. . Surfvu...oi ... 10.6. D ‘00....i. ... {...-0 c .....3 a . .. .C..: 01 ‘Cu... In... .1 o u'l‘lv ..."...1; . ... . . .. o .m?!(. (11.. . J. 11.8....C..¢...L {Winn-o... M... 1... . _v. ..q s . 9....0... 9.5.3.1. ... . . 21..., ...l. o .... .. ... . 19.. x. .. ........, .w... . . ... . .354.) e d“. I? "A? ......V . .r. ‘9 ..9 ... . ... ..P..:........O:...I. 4-... . ...... ... ..393!..:......90.Vt J 3.2.... . ......., . .. i an. .. ... n . ... 9.4 .b. .1. ... .3: 41 V .12.. _ , ....i.........................:........ .. .... .. unspfil... .-....z.......v... .............. ... ...- ...... ,...V..n.......,,. .V. .... “vault-wt’i V . . .uutozwé drum-3.6 "0‘ “....uoouulnuo... .1. r0... 1 . nJ:.-.‘.nvuo.sfla.‘bu~mucfifitouo ..'.:.‘.'|..$o.uv..v {ANHNHU . .. , .. . . 3 .0 . s. ... . . . . . .... a a V. .. ...: I ..3 v . V 1531...)...‘2 ”Y. 3...le ta.firpud1\dn.hu.guodmflrxutz l . (v.8...f I... I... .C‘ .. .... . 3...... . .. . . c.0330 o .. ...?! 1-3.. ...1... ......t\ .t. ...... I: :1... ......t: .... .30... . 1... .1. . .... . .Qatsfl! . . 3. \oixl. ....- 33.21.31}: I .... '00....9: ... E . . ... . . 7.. ... . . . q 3...! : . .. ... 1 I... , pr 33“? t. .0 OQIi‘r.ofo.:... 0 ...-'0... . . 5". w... . . ... ...r..:'..8 . .....l. Ito... V . 15.31”... 5" 2.1.0. }.ol.«":'.+ .6... ... .. ...... ..... r. ... . . . . 0“. t d. ...-......1 f ...?! .. 33...! ...... ..vn... . . .n. . 0.. . .. ... . '0... 0.1 ......‘2. .... z ..u $1.0...I: ......h.l...q 5r... 0. ...-o .0. . ... 2... .. .~ 3.... . . . ...... . ... v . ... . “ r. .n oncv‘o)‘ 1-. r 0.0 .. It. ' otnn'o ...... ... :9...’ .c .3“. 3 5...... . .. .51 C 0' .0 Q. 99:...1 ... V . ... D .. :.’lqoooA .V..... .0. ..~ .3. 33; . 15.1! t ..v ‘1’. ,Sotioio... y ... 0.”. v.10 ... .... ...... 0.....8311 or V. . .s... ... .pu.d .... . :3 .. .lioLo'. o .0... ...-....1. . s.~a....a....a. .v .... I... I... ~ $210“... .02.“...5. & .‘ .... 3.3.3:... ......v. , 23%.“... PJ... I . . . . V . .. . c . . . . .. . . .l. . _ . . "..."...Li? ......o .33“... nail. 31...." “anew." .... Jungiaw: ...... 19:341.": ..uv;:....n. ....fl.... 3” .41“. flashy-$.11... :11», z... ..I.i_c:1...- . o."ru-:.". QQ” ..- f .9. .u'. .....h ’o.~t§‘ c... .. . ...). .t 0.. ... ... t . in... . . I... ..V ..3 ....9 ... .o I. ...I..V a... ..l. :......1£..J.fl..n.....lc\ )‘II .0 0...... ...)... .0. 8!... ...V O x I ... .3 o .10.... . . .vo .- ..lo..... .. . . ......e’ ...!v :10 u...£.| 9... . .....14.‘ il.... ...0. I. C ...-3|..." 8.. . . fi ... Il..l...b . .H .. ...O. c. an... .o I . ... ....tt... .... 1...”..Juuuq..uuuu...z. . ... .....vfrli. u............l ... V... . .. . . ...... .2". .....ns}...n: . '0' v . s . . Q ‘ . . . .. . ... , . . _ . .. :7... CI‘L.3. ‘53....333 0’ c. ‘1 ‘0...’.. 3 “PT "21". . . . .9». § V . I. ....J..Mq.n i ...."wol ...I ' 0...“.6'Na. . . a Q . v . . . . . C n . .. . . . . . ..bu. J...run...xw...u:!..~¢".. .. . . .‘95D..Q..o 1...... .....2‘... ., In... 5.). ...“. y8¢bal.s....$l~o.x Mm“ ...-Mn..no . a . . . . . ._ o . . . a o. . . , o . . v . . 5...... .... O. .E.‘ . . . ..V . . V . "an. ...o n o u .. Flu-o... ... H"...o..a."t.nb..1 vuuflfl.",...lo.fi..flrh.. no“... «‘3 “had-89.“. ... .10.... . , . . . . . . . .V o .0 ...‘p. o. _ .... ..$ ......S .5 .. ..‘J . C P: ... c . 71:26... . .. . . . . ... A: . . vat. . v. 3.. .... 1‘... O. 02...? ... c .90... .‘l ...of‘ .....l! . ...: ......o: . c0! 1.. IV; 61.... .. ll’..Ho\... to 4..‘...:~b’..“.9. I. .13 ....x.... .. ....T . ......ll . .-.. a... o. 3... .49... .. . ... ...W 3;. V .43... a... i . ... .... . ......o..s:!....... . ... ,0 C36 3 6319111 . H”. .....:..c.. .fl 12.16.. At...v.. ......l ”.03.!“ 5:..." .72.). . . o . 5. ...... dutch... "3 “5...... r ... u. . \ . . ...: ......L.o.l|. a 30.0'. o o ”.3": 9.. ......vamnfiwn. .. .... I ...melu...rs luv...” . . ...“... .mwwnw1a.m.§s.$w .. . . I .....f. ...)!L. a i... 3...: V .....I i .. 30)....‘9K. 3......1." . . .. .... ..V. 3...... ......c v... 1.2.1". ..., $7. 31': 9...!Iilr ....m. ... 9... o... . 1.2... 9., .. .. In” 30.... . or 4...... ...l...s.-...V.s. .. 4...... ...-.31.. . ... ..I. .... . .o J \6 .~ . I. 1:}! I . ..1.!..-..l|. .1 ....o... . .0310...” .0“? 3. o. ......v..-‘ 5,... .l1 lib... . .._ ...-.... .....qfa‘tll. l .. ..u‘ .. (on-v. .. $2.33.. . . . ¢.#HW.I‘0.‘.. .....s. ...-.3... 1.5%...“ 3.3.35.1..3. O...-..-;.a.'..!.!l.-o.. on: toil-oval 5. iicu. ) .‘ Apt} . .... . ...... ...,0. 1‘9 ...“ Jln . . . . .W: _ 3.. «I q o‘. . . 71.1.31“. ...«0 ..1 ._ A ... . ...wor... ~,.ioi1u.,3~..ufl.. .. ...-.1.‘ 21...... . It... 2:... w. . .-.... .... .0 .1 .r I‘ ...O. . I-vvvoaocoo. ...? . to V 0‘ ,.c.‘.|..6.... ..I. _o :\n .....q‘ C. ‘-v I O. . 4‘}. .Vyf. I 00‘ 0.... I ..I.Q..o..lo'._ ....lv¢au\.. I..— .of... ,3”. . .... ......I! u. 193...”... . ... ...... o... ...: 1...; v . .o ..t A. :11. ..l ... .. .. .. . .. L oi ... ... . 8...... a ’0... n. . ...-.v I r o .0 b. 3-01. l0...l ...-.01... _. 17...... .. 3.. . Ii... .‘3 u . 1.2.3.1....1r. 55...... . ...IO. ... v. . ... 9..., .890... ...1. ... ,72. .. . .. ...—9.3. .. ... ... . . ...,1,o:6.. u D ...... ...... .9. _. . ...—.76.. '1... . ......93!‘ .... .0..(v.~..!.... .. Q... v“ $.33.- ..o... ...“..- I . I, 0‘... .C. 3...: $.VI..: C . {.31 .. I . n ..o . . . . .l I .. ...! ’.1.3o.,. .\ o“; 23.8““..v3 .CS 0.... o 2. OBI; .... ... ... ...... ....-. ......"b . ..Ui.,b-Oc‘o.!. . . .0. I.....o.o ... r. ....o a 10.30 334 .Mta.”0fl“.o ' ' Q s ,— Pass 31...}. A... . UK. ......o . . o... ...a. : . . . .. ...-.9. 9.. . , .. .. . . . «3 I... .. ....C. ...-D. 3 ...t 9. u. M... b“ ot‘VI‘Ci.1...w ‘61,. (3.“...0 V. . . .. . «...-v.1: .. .\. I. . . . . ..O‘: . 7.6.1.3..-. .ch|.1..¢! it... I. '3‘. o ..B of.” ‘l.\<00. 9. ..¥. '.Q..‘ ...... ...... . .. . .. . . ...A... -. .... .0 5.. . . . . ..A ....fifi‘o. .'......u.“.. ‘38.. ..., 1 ya... :E’rfli“ . . ...... ...-v..‘r.g..o...‘_.. . -..... . . ... ..V..- .... ..s. .339 o. . . ‘11.). .... S ... ...,oto- . . I . $.«vfi .1. u. l‘ . Shift... ...3‘ ......i. .... .. . ... . .Voplt . . .... $10.20... .0. .... .. . . . ._ .... .. ... .....J. . .5334? a .I. . . ‘5 Q ... ...)... .Is..' ...-....nn. U. . 3...! I}. . . . . ‘r ... ... V!) . . filo-4:13.602... .001 .. .. . '3'. ... .30.. 0.10.143. tn. 0" “h. I f. . ... ....tn‘o13VF.c¢.>L‘.¢, .. . Von ... . . . . .. 9. .il ...... 0.... ... .. 0 v. n‘ « .v3.- .Q‘o...uV‘ t 3 ‘1‘. .p v. ......QIIV.... .8 of... 3...... 3...}. O. I: . ., I... . o ... ,V .. ... .\ v0... ....1. .|.I. cl. ... .. .... ... . ...! 3:.» . \Il... 1.. .. o ... I . . ...... . ’{ta‘v‘lt 03' . .. 2‘ O. ..I.... .. . . . . ... , . . . .0. . . .. . V ... ...... ... . ......1. . , 3.. .1 . . o 1.. ... 9621.....1 . .03“... (I... . .. I... . .... ‘I.‘..‘I¢pl .1: .... .. on .. . I. l...- .X..... ’1 .0‘.. .v ....- .....Q. . .‘L’t. I... .1..Q.- o 0.... . .‘g”‘ 6 3‘ o..!7.0~..§6.o ..Ovo..‘¢. Is... . o . ‘0 . ...X.,t..o.‘..u~ on \ v. .. I .n h". ...v ... OI... .8 . .. .... .I n?! .V. . . 0.0.! I. I ....O: . o .Iiuaol..s. .... .... .1...............3§ 39......01 . . . . .. .. .... l: . .3 .1: o . .. 2.813.. .bxol. ...o ..o )1.».. .... . ...... 0.... . . ...: ....V . . . . ... V . .. , ... .Lor .l... 3 ... .... ~1..aJ...-..30! ... to .. .. .2339. . . V . ...: . .. . . .r|gy.i.vvvt. ..6.v41-J.A‘ . . . . , . . V . ...-3...! 1...... 0&3. .... :1. c. n. V . . L . . .. ..a .1. ....‘§E’—.. . . r . . ....Il< . . , . :9. ...V... . . .- ‘ ..Cos ...-l \ . 0 .... . . . . ,5. .01... ..IIUS‘ . V . .20....b1 .. .\..‘C . s a..1..o.d.1l¥i _. V o. .... J .. .- 1.0.1.. . .. o: .0. .1 0...»...3 ......Ns31 :0... ......Oo..‘. .13.}... 43¢. . ... . .. ._ , . .. so: 1......X: ....hfiui L... 53: . . ....t....l.... u... o. ...t. 0...... . . J t \“l ....5‘zov‘3. ...... c I 3‘: . . t... . . .... .... 1. . , . . ....« ......9~.~..n..3!t.o v. ...o’t ‘3 . . .... ... ...... ......3...:...«— «5.32:9... . b“ . . . . . ... . {anviviu ....n. to... .... 3 . . \O. Juno. v. ...1 . . . ... . I ...-... 0., .. . .. a... . .. ... . .05 . TX -0 .. 71.1,. ‘ o ... . . .3. .. .. .‘ £~.lv‘n.lv.¢-“' 53‘ 0a... Emb‘firti“ 3:...5' .....tll....aa.. .! 3.30%; 75.19 t -2‘01. '1”. a. .... .u C ‘fr. 3. V O. . .... §. .. I... it... .V‘l .... ....S...OA . 3 9 i‘b%!.l ’ 0. tag r... .00?" . ...: ...ouo. ....h .... o . . \vvo... Athittfwol J. ...l..v-“..I«.S‘t .L’ .“ it. 0....-. .1. . ... ...: . .. .... .0 ....m.“ l o. .0 '3‘. I, v... a... .fi: . 30“. V . .‘3 .. ' 9ovg' .4" ‘v‘ .I J.....¢o.., ... . .. 1 . a". l .‘ 0.. u. . ~4‘4. . . . . .. é a “ ‘gg . .. ..m {w}. I .I 3.1 ...! ‘ ‘I $.ng 1 ...‘v.l kw 0-. .‘ 0.. o‘c‘ o.“ .19.... ‘il... .. . ......oo. ... . . .0 v. ,V CI. 1 U‘....’Obu .... ...... ( .... o 0..., .o " o'u"...“¢. . ..c .. I‘vlob V... . .3. ....o .... T . .. ..Qt..~.v!lkoiv0.c, ..n . ....V ... o. a y ., .‘ .ll. a .y t 01 . . .. 1 3 ... .3... . .. ...! n. .9. .‘- . . 9.3.. 13...... - . ...- .... . ....I‘ .00 ‘38.. ...“...{c i334v f . in-) BC .8. .....V .9. . 35.3.10... . .. Q. ... . ...v . . . 2.03. 1’... :31... . o. ... I ...... I! . . . .0 .... . . ...-5“}... l.)\!...! “a...“ .l . N. .. . . . . 1 T “.31.. . . . . . ...Du‘qqop‘“: ~I. .3 ’Iy‘. 1“...- c‘.. . . I3... 01......v. . . A .. ... v.;., l ,..”..v0.0!1..0.‘2§. ...:ra. . .. .P.¢.t......,. . . o o... “.....v’ot... _. ......‘l...€ 10,1310... . § _ .~.. .01.. . .I...’.J Co.....Blisa.§..1ab‘.‘c£.15q-I.Q? . V .9 3... 0-7..- ..v!‘..’u.1. ..1' ... l!‘%’n9 .030... ..T....~I...|... {\3... 3...... ....v! ..I. v ... $(\¢.S\..> 25.0.4...- ‘b ‘|"l! .. .4. YA 39.0.3.1: '3 ..." . .. '. I r 959...! .32.... .DJ . *'. ~.. I .. .un u.- .. it... 0 Q o'llatolllcc. .. .Ao.‘..011... .v . ..ah‘ o“‘. £33-. 1... u ‘S .'a... . ...-...» 4 271:... . ‘ O ...; ll. .Q «v.4 .r ..V .... ..I .., . . . .....‘fa ..o.. 0...- J > ... . ...; . . v A." .-.: .... u o a; a»... . V... . . t - ...X: o a . .0. .. 9.. . v. . _ . . . . 00.. . .0. .« ...... ...-I! . ..- ...; .V... .I. .....V....n¢.o l .v..o-..o.u u . .7... 4.-.}... .50.. . . . .. . .. v-.I| . . . In: .u ~.Ool.... . ..... . .. . V . . .6. . A 0.9! u. . 9v .. ....-IQISQ. 0.... 0.: ... .....- it. . . . . . ,. .. .. .. .\. ...... ...(V‘..t‘ .6... ....L t .‘ ... ...: of .V . .... 5.1,. 0 I. . a . ....V .3 .. ......)- . . . . . ..c . . . .. ..--...... ...-.... . . . o , A. . . . .... 6.3L. 3...... .40... ,1... . . . ... 9.000- .. 0.... . A .. ...t ... «0.:‘1. L. .3.- 43 ...lo ... .4 . v. u. . 339.7... .1"... ... ‘ .35.. 3.5.... . ‘.v.... 3.5.. . .5 . vihv‘o1. ...: . . , . . . ... 4.... . ...Iv§.5.'o . ,Y ... J. 1.... ... .- I. .. . . .. .. . . . . ..0‘ I0 9.. . . . . . ......o........‘... . . . . . ‘3. :.J. ....i.’ Q l o: co. ..na, _. 9.7)....1 . a: o .. . , “9N‘.3..1. “Isa"!‘u d O 2! ... 3| 5 , . ... o .n.‘ an. I s- . f ‘. I. ‘olwwvrnfil I 3 ...-r .950 ...n‘. .0" law-3 ..I Ahab": . .Ofihwnaq.€ ~ OOMvQ. D...‘ ‘ .0. >0. 2.31,..00 . .... 710...? a 'fi\ ..-. tul- . ,. g. .V..1.-.... fr {’3 ‘3 ...‘Wa...“ gw‘caq 3.0. O‘ I‘m. J I E44932... 2.5 J." $ ...: . .. .....l‘ trot. ‘1 ..l.‘.'. i V . .a . .. .14. . . 0:. I... v: D. II' T D. . , I: .....‘v v.8: ‘I... . ‘. ...... <1 . .‘o . \h.~ .Q‘. o .5. 0.0... — . . .. . Q! ... ... V. o. . 6.. I .00. u. ... .... ...‘of 9... I. \.i .. V's-o 4.19.}... 2f.).0..... . . .OI .1..,.Il| 0.. . fid- ..J. I. , , .....-.l|-..n... . ..., . . . .. . ......1... . . .. . .\.., , . . . . . . : . . . . . .l‘......¢.. . . , . . . ...‘(Doqt .. ...“ , n. . u . . .. .I...‘.I‘tr . . .. , . . . . . . V . ...-0‘1! 9&‘8 .. ...!!! 97“... . W913... 3.21 \‘2I’l3l..¢.£'un o. 4.. 0‘. 3.. it 3‘“ ‘30 ‘3.0.3. l I_'.Q. 3 w. . , -. I . l a . . ‘ ..Ilt O. ."o 3.' 1‘- u. 35 v. . o.q‘\:‘|l5..1t. . if 4 \ul 3!...“ (fit ...-‘9'... % o- i... 9’. I \’ W’s... .0. z.- I‘ 06.4... .. §‘u«0.vri%'OOQ . .1. u.‘ ‘zloilbaoo...’ P . I ... ... .I C , o O . . .. 7;.22 ... ... 1 . .o .. .. . . _ . . . wvA-‘o."‘ ‘7“‘7 o u u . 5 . .'. 0 'o ' J .1.\...... . . o. I .. ... ........¢ .....‘. T‘F-‘I - .... . ’.-.\R... 0.-.. ..u . I... . .. .... :05}... VI 7' . T, 7" ... ....0... a w n ...I. 1 . .0... . ...V:-.~‘..f.. 1 . 2 f . .4 . . . 5.! ‘0 0.!301‘... ’. ...! ... . ..‘V. -. .51.... o .. .....Yolo.‘ .. .u'...l.vw¢1 ' . ..I...’ 0a.: .00.... I V . . ...... I . .11).. . . h o .....v ...HI“... , ...}. i ‘I‘ on... 0313.19 I I .! .... I . . ......sioo _ V . . ..C. . . . . ... .5 . .. o . . ..l .. a. .. .. . . .. ....l .3? v! b.¢- . a... ......5...... .. .... . . ... .0. A V . 06... . a... VOO.‘O,. V ‘31-0-443930 03.0.3.9, ... ...-O. ‘ . I ... 0.... . .vN .. . x Q . . o.....!..‘ 3‘”! .... ...-u o. .335 .4 . ..l. 4.. ..1 ‘0}.‘19J‘IDQ.’V Y. 1 .. ... o \l. V “.0 .l? 9 ...}.I .. ..II .....If, if: ... .... .0. 0......) .',$IO,4\OQ is. . . ‘1...O.Vp co. .V 90. ......0. '15... .I... 14.. -Kub.‘~l‘. ... ..l. .c a {‘1 ..$..\....’ .. 00.... . . 0130......00. a...” .30. ...-a: g . ... ...-.... o... .. a-.. to. ..o‘bVI .V... ‘95.. lo. 2. V . ... . . Q . ..-)j.o)'oul.il.0 . ....4 0‘. .VII‘ 23.3.. din. .-v ,‘u...... 'Il :.:$.’3. . ~ . .... .. O. a. .1... O .. ’1... I‘Vr..-6.!1..'ta;fio. . .. . ...- v: .... Alv.. o...t|.v..‘ 11V. Irl"..7‘l ..Iao...l:.. I. 1.0 ‘3'...”1266. . . if)... ‘ . .. . ...?uo‘.‘.:. I... . .. . . . I. u . .o '0... 3’... .....DJ. .. . . . . . 9.. . ..V: V . 7 ....... . .. .. ..a .oV. . . . ...-I ) V. n u....... ._ ... .. ....VO .. :u. .. u .‘3... ~00...., . 03.4.1. 0 we . . . . a... . . o v .I .... 00o -.. . .ui. . ..b ...o' . . . . ... ...o- . .. .. .... . . . o . ..I .. u . . . T3-.. . V .. Duo. . V .r . ......o ...... ..I . ~ .0. I... u 0.. 3 vi. l .. .. .. . . I . . . . . £1.53..- .5. . ... l . . 00...... .V. .. I... b... .. . Z... . . .... n. n. ‘ o o. . .. ...t . V . .. . ‘33.. ... v s." . ... .. ... .. . ..r. o... .. ......o. . . _ i. . .. .. . , .. . . . . O ...-7‘ 0‘ ‘ . . . . 3. ... t. l .. u v . . . I. . L 0' £301... 50 .... 0.0.00 . .n . . V I 5...... l u. .. i n, ...... -..l: ‘34... . . .. I . 0“ 5‘... ‘00! . q fifotluwfl Qt A . . .... o. o: ....O; .. ..P ‘. .0. v.- .....u b‘. . o .. .o .a ., . $.01. A . In‘ .!._1.Q0Av..al.,. ....uV"... .. u, . . .. .... . ..v? . 21.61:)... -0 3:1! . . ... ...- .../.... .3‘. ... . avg....‘..IO. . .A 0.. .. .‘U.C‘on..l.3|1 C .. 01'. 9’; . I.- ‘ . . u . . .1. .. . 5.-.... .. .. . . .o .. o. . fit. A . . 01.... ...o:. . ...... .,. . . .u‘...o.|.. a . I. ... . .. 0... . o . . . . , ... a. . . . u. u do 0.. .. . O . \. 0.. ._ u . . .. . . .. . ... . ._ g . . . O, I, . u. I 0!. ...- ... . ...L ...-.05... .. .c). . .. .. .,.7‘...... ... ‘0... ......I . . $6 .... . ... ¢ oa...0.{.,o .5. I... .s . . 9.5-. a .. ‘ .I.....o l!‘.0".§l. . . .502 9:! ... .‘. c . «1"... IIQI.- o... 1...... ...! .6 .|...v8.|-... t... .0. o .. Q. .90.. 00.}. .‘d.¢4.‘1'.. ... ...QO‘ . bl: l:.¢.'l.n0.\)ai.nol.. - .... ..vl ... ‘9 .‘ . 0.. ... .... O\V. ... n .1 .0 I. a, . . 900.‘O.. \. v ,I....... .o. s .... . ..I..." .55.... . . . . , to. \O ...: O .9 ...... .... '9 O. ""-1 .LV .1.' .. . .. . . c. o. In. V II . V. .. , 5-09.. . o 3 Q o ....v. 0 .... . ... . .h.‘.l.. ... . . ... . I . n .9. . VI . .. . .. .on.i .. . I. I .-. 009 ..u...‘. c . 0 . 5.00:...v. ....v . . .v .... . ukI . . . o. ...... . .40. . .. 0.. 'l O o . 45. . ...OL.. Obi... 0". . . .v . J....c.o.! a II, n 09* o‘.‘ 9 .6 --A“ "'90 '04... ‘I I .. t ‘6 II P Jung“... ...... .. high, “awninglfiuv . . .... ...... -.....z. 3.93%.}... averanaifim. ...... ... .63. .... . | . v: . . .1... C u i. .. . . \ .0! " V .‘NI. fl. 3.9%! J 3...? ...; w m. .nout‘tny. . . .f. sw.\ ...”?- .. t ...... ”My“... 6 .... Hatch.” .CJ"..... .3. _. . . r c 0 . . . #w .3... fig”... . .44. . t? .. ...... . . A 10.“ man-b w . ... :.|. ...O. .. .Ona Xv. . . n . ' . . . . . o u .b .... , . .I..;..Q. ., u...“J..h....-” 3.: ...: ... _ . J 0| . ..t . .. .....3mfd H. . Q. \ . I I A I n v I 1 9.. ...... -... ..4..04u..on....1 - bl. '. . . ..1...:... A no . . . . , .. .... u..ufi5.:...é¥.£m.r.r v .o . . .. . . - .I‘o. 1-“ $0 3.9%." .. :03: .0 N o: 9.. ...... A 04.. I c .- l.‘ I I'll III-IF I III! .rl I'll III ’II. .I. THFQIS to. This is to certify that the thesis entitled POSITIVE IDENTIFICAION USING COMPARISONS OF LUMBAR SPINE RADIOGRAPHS: A VALIDATION STUDY presented by JANE C. WANKMILLER has been accepted towards fulfillment of the requirements for the MS. degree in Criminal Justice . 2/7 / f Major Professor’s Signature [/I At? ——/,a7 fij/n /¢) Date MSU is an Affinnative Action/Equal Opportunity Employer LIBRARY Michigan State University PLACE IN RETURN BOX to remove this checkout from your record. TO AVOID FINES return on or before date due. MAY BE RECALLED with earlier due date if requested. DATE DUE DATE DUE DATE DUE 5/08 K:lProj/Acc&Pres/ClRC/DateDuetindd POSITIVE IDENTIFICATION USING COMPARISONS OF LUMBAR SPINE RADIOGRAPI-IS: A VALIDATION STUDY By Jane C. Wankrniller A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Forensic Science 2010 ABSTRACT POSITIVE IDENTIFICATION USING COMPARISONS OF LUMBAR SPINE RADIOGRAPHS: A VALIDATION STUDY By Jane C. Wankmiller Forensic anthropologists commonly use the comparisons of antemortem and postmortem medical or dental radiographs to positively identify unknown decedents. It is important to establish the validity of such methods due to increased scrutiny of the rules for admissibility of expert witness testimony. This study is designed to evaluate the validity of comparing of x-rays of the lumbar spine for the purposes of making a positive identification. Twenty-five participants were provided with sets of 20 radiographs that simulated antemortem x-rays of known individuals that they were asked to compare against 10 radiographs simulating postmortem x-rays of unknown individuals. The x-rays were all taken of cadavers from the Gross Anatomy Lab at Michigan State University. All 5 “unknown” individuals had corresponding matches in the set of 20 “antemortem” individuals, and the participants were evaluated on their ability to correctly identify the appropriate matches. Each participant also completed a two-part data sheet that asked for a number of personal details and their answers to the identification test. The participants achieved an overall accuracy rate of over 90%, and an overall error rate of 4.17%. This study shows that observers who have more experience are less likely to attempt a match that is negatively impacted by confounding factors, such as poor image quality, and are more successful overall than those with less experience. Results also show that correct matches are most often achieved by taking into account a combination of gross structures and minute details. Copyright by JANE C. WANKMILLER 2010 I would like to dedicate this document to my family. Your unwavering faith in me has enabled me to have faith in myself. iv ACKNOWLEDGEMENTS It will be nearly impossible for me to adequately express my gratitude to the people who have so generously helped and supported me throughout this process. Dr. Sauer, thank you for all of your advice and patience and for reminding me to keep it simple. Dr. Foran, thank you for keeping me positive and motivated and for never letting me forget that I had a thesis to complete. Dr. DeJong, thank you for all of your statistics help and for being a calming force for me in times of utter panic. Dr. Fenton, thank you for your support and for taking the time to counsel me when you saw that I needed it. Thank you to the Forensic Sciences Program, School of Criminal Justice, for providing financial support for this project. Bruce and Deb in the Anatomy Lab at MSU, thank you for helping to generate the sample of cadavers I used for the x-rays for this project. To Jacque Liles and Kristin Liles, Willed Body Program, MSU, thank you for the permission to use the cadavers from the gross anatomy lab. Thank you to Dr. Mysliwiec for your permission to duplicate my x-rays at the Ingham Spine Center. Greg Bis, thank you for all of your help and kindness—you single-handedly restored my faith in people. Thank you to my fellow graduate students at MSU, particularly Michael Koot and Colleen Milligan, and to my family and my husband who took the time to talk with me about statistics, research, and life. You all helped me to maintain my sanity, and I love you for it. A special thank you to all of the volunteers who participated in this study. I deeply appreciate your willingness to take the time out of your busy schedules to help a graduate student who, in most cases, you had never met. Without you this project would not exist, and I cannot thank you enough. TABLE OF CONTENTS LIST OF TABLES ........................................................................................................... viii LIST OF FIGURES ........................................................................................................... ix INTRODUCTION ............................................................................................................... 1 CHAPTER 1 HISTORY AND BACKGROUND OF FORENSIC RADIOGRAPHY ............................. 5 Validation of X-ray Comparison ......................................................................................... 7 CHAPTER 2 RESEARCH QUESTIONS, GOALS, AND HYPOTHESES ........................................... 10 Research Questions ............................................................................................................ 10 Goals .................................................................................................................................. 11 Hypotheses ......................................................................................................................... 11 CHAPTER 3 MATERIALS AND METHODS ....................................................................................... 13 Radiographs ....................................................................................................................... l3 Selection of Participants .................................................................................................... 17 Data Sheets ......................................................................................................................... 18 CHAPTER 4 RESULTS .......................................................................................................................... 20 Accuracy and Error Rates .................................................................................................. 21 Effect of Personal Attributes on Performance ................................................................... 26 Contingency Tables ........................................................................................................... 27 Total Sample ............................................................................................................. 28 Professionals Only .................................................................................................... 29 Students Only ........................................................................................................... 30 Tests for Significance ........................................................................................................ 30 CHAPTER 5 DISCUSSION .................................................................................................................... 33 Effect of Personal Attributes on Performance .......................................................... 33 Students .................................................................................................................... 32 Professionals ............................................................................................................. 35 Total Sample ............................................................................................................. 36 Problematic Cases .............................................................................................................. 39 Problematic Case: Unknown D ................................................................................ 40 Problematic Case: Unknown E ................................................................................. 44 Features .............................................................................................................................. 45 vi Limitations ......................................................................................................................... 51 CHAPTER 6 CONCLUSIONS ............................................................................................................... 55 APPENDIX A Cadaver Information.............. 62 APPENDIX B Simulated Antemortem Radiograph Images65 APPENDIX C Simulated Postmortem Radiograph Images86 APPENDIX D Sample Data Sheets ............................................................................................................ 92 APPENDIX E Results from Contingency Tables ...................................................................................... 95 APPENDIX F Results from Nonparametric Statistical Tests99 APPENDIX G Participant Answers to Data Sheet Questions. . .104 APPENDIX H Features Cited by Participants when Making Matches between Radiographs ................ 117 REFERENCES ................................................................................................................ 123 vii LIST OF TABLES Table 1: Independent and Dependent Variables Used in this Study .......................... 20 Table 2: Participant Performance ...................................................................................... 24 Table 3: Participant Performance, Excluding UnknownD/Known 20 ............................... 43 Table 4: Cadaver Information ...................................................................... 63 Table 5: Results from Contingency Tables ...................................................... 95 Table 6: Results from Nonparametric Statistical Tests ........................................ 99 Table 7: Participant Answers to Data Sheet Questions ....................................... 104 Table 8: Features Cited by Participants when Making Matches between Radiographs ........................................................................................ 117 viii LIST OF FIGURES Figure 1: Lumbar vertebrae were de-fleshed before they were cleaned according to the procedures outlined by Fenton et al. (2003) ...................................................................... 15 Figure 2: Vertebrae were held in position over the x-ray film using a wooden dowel in a brace and florist foam ........................................................................................................ 16 Figure 3: Unknown D (left) and Known 20 (right). This illustration shows the difference in orientation between the two images and the poor image quality of Known 20 ............. 41 Figure 4: Known 3 (left), Unknown E (center), and Known 15 (right). With the three images side-by-side, the similarities between Known 3 and Unknown E and the differences between Unknown E and Known 15 become apparent ................................... 45 Figure 5: Bar chart showing the average number of points of similarity cited by students and professionals ............................................................................................................ 46 Figure 6: Bar chart showing the number of times each type of vertebral feature was cited byparticipants.................... ........................................................................................ 47 Figure 7: A “point” may refer to the contour of a bony feature, or it may refer to any number of smaller details contained within the larger feature .......................................... 48 Figure 8: All participants answered the match between Unknown A and Known 14 correctly. Note the large disparity among the numbers of points cited by participants ........................................................................................... 49 Figure 9: The terms, syndesmophyte, osteophyte, and arthritis were used interchangeably to describe bony changes of the vertebral column (the vertebra shown is L3 from Unknown C/Known 9) ............................................................................... 50 Figure 10: Simulated Antemortem Image #1 .................................................... 65 Figure 11: Simulated Antemortem Image #2 .................................................... 66 Figure 12: Simulated Antemortem Image #3 .................................................... 67 Figure 13: Simulated Antemortem Image #4 .................................................... 68 Figure 14: Simulated Antemortem Image #5 .................................................... 69 Figure 15: Simulated Antemortem Image #6 .................................................... 70 ix Figure 16: Figure 17: Figure 18: Figure 19: Figure 20: Figure 21: Figure 22: Figure 23: Figure 24: Figure 25: Figure 26: Figure 27: Figure 28: Figure 29: Figure 30: Figure 31: Figure 32: Figure 33: Figure 34: Simulated Antemortem Image #7 .................................................... 71 Simulated Antemortem Image #8 .................................................... 72 Simulated Antemortem Image #9 .................................................... 73 Simulated Antemortem Image #10 ................................................... 74 Simulated Antemortem Image #11 ................................................... 75 Simulated Antemortem Image #12 ................................................... 76 Simulated Antemortem Image #13 ................................................... 77 Simulated Antemortem Image #14 .................................................. 78 Simulated Antemortem Image #15 .................................................. 79 Simulated Antemortem Image #16 .................................................. 80 Simulated Antemortem Image #17 .................................................. 81 Simulated Antemortem Image #18 .................................................. 82 Simulated Antemortem Image #19 .................................................. 83 Simulated Antemortem Image #20 .................................................. 84 Simulated Postmortem Images Al and A2 ......................................... 86 Simulated Postmortem Images B1 and B2 .......................................... 87 Simulated Postmortem Images C1 and C2 ......................................... 88 Simulated Postmortem Images D1 and D2 ......................................... 89 Simulated Postmortem Images El and E2 ......................................... 90 Introduction Positive Identification Two types of identification are possible for an unidentified decedent: presumptive identification and positive identification. Presumptive identifications are based on characteristics that are not considered unique to an individual, such as tattoos, scars, or physical abnormalities; or items that may be circumstantially associated with the body, such as a driver’s license, credit cards, or clothing (Dix et a1. 2000). On the other hand, according to Dix et al. (2000: 75), positive identification “entails scientifically establishing identity through the presence of unique characteristics.” Such identifications are made only in situations where the investigator is confident that the information acquired from the decedent (radiographs, fingerprints, or DNA) and any corresponding antemortem information can only have come from the same individual, “...essentially excluding every other person on earth” (Baker 2005: 2). Presumptive identifications are often one step in establishing positive identifications—they can lead to a potential name for the decedent, which can then be used to acquire additional information with the hope of eventually arriving at a positive identification. Common methods used by forensic scientists to obtain a positive identification are DNA analysis, the comparison of fingerprints or the comparison of antemortem and postmortem medical or dental radiographs. The same methods can also be used to exclude a presumptive identification if there are inconsistencies between the antemortem and postmortem records (Murphy et al. 1980; Owsley and Mann 1992; Sanders et a1. 1972; Valenzuela 1997). Positive identification of an unknown decedent is extremely important in a medicolegal investigation—family members cannot receive compensation for their loss until there is a death certificate, and the medicolegal community must know the person’s identity, cause and manner of death for any legal proceedings (Lichtenstein et al. 1988; Murphy et al. 1980; Sanders et a1. 1972). Often it is the duty of the Medical Examiner, Coroner, or Forensic Pathologist to determine the identity of a deceased individual. However, when human remains are too badly burned, decomposed, or fragmented to be identifiable, they often require the expertise of a forensic anthropologist to determine a positive identification. The method most frequently employed by forensic anthropologists to arrive at positive identification of human remains is the comparison of antemortem and postmortem medical or dental radiographs. A dmissibility of Expert Witness Testimony Throughout the last decade and a half, the forensic sciences have experienced an increased emphasis on validation and standardization of methods, techniques, and record keeping. The movement gained momentum in 1993 after the Daubert v. Merrell Dow Pharmaceuticals, Inc. Supreme Court decision, which established several guidelines for the admissibility of expert testimony, particularly with regard to scientific evidence: 1. A method must have been or must have the potential to be empirically tested; 2. A method must have established error rates; 3. A method must have been subject to peer review; and 4. A method must be generally accepted in the relevant scientific community (Brogdon 1998; Christensen 2004a). According to Rogers and Allard (2004: 203), “if the defense requests a Daubert hearing on the technique used for identification and if the technique is deemed inadequate, the identification will be inadmissible and the prosecution’s case may be seriously threatened.” The Daubert decision resulted in the Court acknowledging a “gatekeeper” role for trial judges, under Rule 702 of the Federal Rules of Evidence. Its application greatly restricted the admissibility of scientific testimony, which until this time, under the Frye v. United States (1923) ruling, had only required the general acceptance of a method among the relevant scientific community. While Daubert has become a federal standard for the admissibility of evidence, some states have decided not to adopt it at the state level in favor of using the Frye standard. The admissibility of expert witness testimony under Daubert specifically concerned scientific experts and said nothing of experts with technical expertise. It was refined somewhat by the K umho T ine Co., Ltd, et al. v. Carmichael et al. (1999) Supreme Court decision, which expanded the admissibility of expert witness testimony to include experts fi'om scientific or technical backgrounds as long as the method is relevant to the task at hand and rests upon a reliable foundation (K umho Tire Co., Ltd., et al. v. Carmichael et al. 526 US. 137; http://1aw.onecle.com/ussc/526). This push toward increased standardization and validation of forensic science techniques was more recently impelled by the release of the 2009 National Academy of Sciences (NAS) report on forensic science in the US. The NAS found that there is a large amount of variability across the forensic science disciplines in the quality of education, facilities, and the extent to which methods and record keeping have been validated and standardized; their message to the forensic science community is that there is a tremendous need to reduce, and ideally eliminate, such disparities. In the 2009 report, the NAS presented recommendations to Congress, encouraging legislators to provide support for research that would lead to standardization of methods, terminology, reporting and education, validation of forensic methods, quantifying error and accuracy rates for those methods, laboratory accreditation, and quality control (National Research Council 2009). Among the techniques being brought into question by the NAS report are such things as fingerprint comparisons and medical and dental x-ray comparisons. The report cites these techniques as being based on an expert’s interpretation of his/her observations, making them highly susceptible to inter-observer variation. Techniques such as DNA analysis are “laboratory based,” according to the NAS report, making them less susceptible to such variation. According to the NAS, the observer-based techniques across forensic science disciplines are greatly in need of standardization and validation. It is this push toward validation of techniques that rely on expert interpretations that inspired this study. The following thesis will present a research project designed to validate the practice of comparing antemortem and postmortem abdominal radiographs, focusing on features of the lumbar spine, for the purpose of securing the positive identification of an unknown decedent. Chapter 1: History and Background of Forensic Radiography Wilhelm Rbntgen discovered x-rays in 1895 during an experiment he was conducting to study cathode rays. The first forensic application of the newly discovered x-rays followed within one year of their discovery (Brogdon 1998; Evans et al. 1981). In an 1895 case in Montreal, x-ray images were used to find a bullet that had been lodged in a man’s leg. The x-ray plate was submitted as evidence in court and the shooter was convicted of attempted murder (Brogdon 1998). X—ray evidence was first accepted into a US. civil court in July of the same year. In the US. case, a man named James Smith fell from a ladder while working on someone else’s property and he saw a respected doctor for the diagnosis of a leg fracture. The doctor was unable on several occasions to diagnose a fracture, so Smith never received treatment. Smith sued the doctor for malpractice and won the case based on x-ray evidence that was submitted which showed clear evidence of a femoral neck fracture (Brogdon 1998). The skeleton, or at least fragments of it, is known to be able to withstand and survive taphonomic forces that destroy other tissues of the body, such as decomposition, burning, and carnivore activity (Kahana and Hiss 1997; Murphy et al. 1980). Additionally, many people from developed countries have had or will have x-rays taken at some point during their lives to diagnose injury or disease. Skeletal features are often visible on radiographs, even if the purpose of the x-ray was to examine sofi tissue. Because skeletal elements are also often available for x-ray postmortem, they are ideal for comparing with the goal of matching radiographs of a decedent to those of a missing person. Radiography was first suggested for use in identification of unknown decedents in an 1896 article in the Journal of the A merican Medical A ssociation (Brogdon 1998), but was not used in this manner until the 1920’s. In 1927, Culbert and Law presented the first known case of positive identification made using a comparison of antemortem and postmortem radiographs which specifically focused on aspects of the frontal sinus and mastoid air cells (Brogdon 1998; Elliott 1953; Evans et al. 1981; Jablonski and Shum 1989; Sauer et al. 1988). Since that time, forensic radiologists, forensic pathologists, and forensic anthropologists have expanded the use of comparative radiography to identify whether remains are human or non-human, to assess personal attributes like age and sex, to locate evidence of healed and recent injury, disease, foreign objects, congenital anomalies, and to identify unknown decedents. One of the largest early applications of forensic comparative radiography was during the 1949 Noronic disaster (Brogdon 1998; Elliott 1953; Jablonski and Shum 1989), when the Noronic, an ocean liner, caught on fire at Toronto, Canada, killing 119 people. In many cases, the bodies were fragmentary and burned far beyond recognition, so the investigators took x-rays that could be compared against antemortem records they were able to acquire from 35 of the missing persons from the disaster. Of those 35 victims, 24 were positively identified by comparing antemortem and postmortem radiographs. As mentioned above, anthropologists have been publishing on the utility of x-ray comparison in positive identification since the 19205 (Brogdon 1998; Kahana and Hiss 1997; Sauer et al. 1988; Ubelaker 1984). Radiographs of a human skeleton, according to Hogge et al. (1994: 373), “...present unique skeletal anatomic information analogous to a fingerprint and provide for a reliable means of identification when comparison radiographs are available.” A similar sentiment is shared by a number of other researchers (Angyal and Dérczy 1998; Brogdon 1998; Brogdon et al. 2010; Christensen 2004b; Ciaffi et al. 2010; Comelison et al. 2002; Elliott 1953; Evans et al. 1981; Fischman 1985; Hogge et al. 1994; Jablonski and Shum 1989; Kahana, Goldin and Hiss 2002; Kahana and Hiss 1997; Kavanaugh 2002; Koot et a1. 2005; Mundorfi' et al. 2006; Murphy and Gantner 1982; Murphy et al. 1980; Owsley and Mann 1992; Owsley et al. 1993; Quatrehomme et al. 1996; Rich et al. 2002; Sanders et al. 1972; Sauer et a1. 1988; Scott et al. 2010; Telmon 2001; Ubelaker 1984; Valenzuela 1997; Weiler et al. 2000). Monozygous twin studies have also revealed that twins can be distinguished from one another based on the radiograph comparisons (Greulich 1960; Marsh 2003). Validation of X -ray Comparison The comparison of x-rays in the forensic anthropology setting relies heavily on visual interpretation of the films. Radiograph comparison is similar to fingerprint analysis, where both gross and minute details are compared and, based on the assumption that all individuals possess individualizing, unique characteristics and combinations of characteristics, an identification is either made or excluded. However, unlike fingerprint analysis, a minimum number of points required for a positive identification has yet to be established for comparing radiographs. Presumptive identifications are either confirmed or excluded based on the existence of similarities or differences between the pairs of images. In order for the identifications in these instances to be considered “positive,” corresponding features must be identified between the antemortem and postmortem films, and if there are any differences between the two, they must be explainable as age-related changes, surgical procedures, trauma, or disease (Rich et a1. 2002; Sauer et al. 1988; Telmon et al. 2001; Weiler et al. 2000). According to Kahana and Hiss (1997), the features must also be unique to the individual and must remain stable over time despite life processes. A number of published cases reports demonstrate positive identifications that were made in forensic cases by comparing antemortem and postmortem radiographs of various regions of the body (Angyal and Dérczy 1998; Brogdon et al. 2010; Elliott 1953; Quatrehomme et al. 1996; Scott et al. 2010; Telmon et al. 2001). Other reports have presented cases that involved radiographs of the spine, specifically. The Noronic disaster, mentioned above, made use of various skeletal features, and according to Elliott (1953), the spine was particularly useful in many of the identifications. Owsley et a1. (1993) present a case study where they were able to identify one of Jeffrey Dahmer’s first victims, who had been dismembered and severely fragmented and dispersed across Dahmer’s backyard, using x—rays of the head and cervical spine, in addition to dental periapical x-rays. Kahana and Hiss (1997) published on the utility of comparing trabecular patterns in the proximal humerus that are often visible in chest x-rays; however that site, in particular, was found to be problematic by Ciaffi et al. (2010) because of the difficulty in duplicating the position of the humerus from radiograph to radiograph. Mundorf’f et al. (2006), Scott et al. (2010) and Telmon et a1. (2001) also present cases for which chest radiographs were useful in making positive identifications, and all focused primarily on the cervical spine. Owsley and Mann (1992) reported on using radiographs of the abdomen and pelvis, and Hulewicz and Wilcher (2003) and Valenzuela (1997) found that features of the lumbar spine were particularly useful in determining the identity of unknown decedents. Comparing antemortem and postmortem radiographs of several regions of the body for positive identification has been validated by researchers. Christensen (2004b) quantified the differences in frontal sinus morphology using Elliptic Fourier analysis and Euclidian distances. Her study, which involved a sample size of 584 individuals, proved that skeletal variations visible in cranial radiographs can be quantified and that her subjects could be statistically distinguished from one another. The same statistical method was employed by Paolello and Cabo Pérez (2008), who were able to quantify the difference in morphology of left transverse processes of the second lumbar vertebra from over 70 individuals. Kuehn et a1. (1997), Martel et a1. (1977), and Weiler et al. (2000) examined the validity of comparing variations in the thoracic spine that were visible in chest x-rays; Koot et a1. (2005) studied the comparison of x-rays of the hand, focusing on bone morphology and trabecular patterns; and Hogge et al. (1994) looked specifically at the experience level of the examiner and to what extent it has an effect on his/her ability to positively identify or exclude an individual. Finally, Rich et al. (2002) examined the utility of comparing surgical interventions that were visible on foot and ankle radiographs, and found that their presence and position in combination with associated bony response are highly reliable for making positive identifications. All of these studies have repeatedly functioned similarly to establish x-ray comparison as a reliable means for arriving at positive identifications, to publish error and accuracy rates, and to identify some best practices for comparing radiographs. This study was designed to add to the existing literature by validating and presenting error and accuracy rates for making identifications based on radiographs of the lumbar spine. Chapter 2: Research Questions, Goals and Hypotheses This study was conducted to establish error rates and accuracy rates, using a reproducible test scenario, for the practice of comparing medical radiographic images of the lumbar spine to determine a positive identification. Admissibility of an expert witness’ testimony is crucial, and ideally the information discovered through the process of this study will serve to reaffirrn this type of evidence as valid and within the boundaries of the legal frameworks established by the Frye, Daubert, and K umho Tire Supreme Court rulings. Research Questions 1. Is the comparison of x-rays of the lumbar spine a valid means for arriving at a positive identification for unidentified human remains? 2. What is the overall error rate for individuals who use the comparison of x-rays of lumbar vertebrae to arrive at a positive identification? 3. What are the rates of false positives and false negatives for this type of identification? 4. To what extent do personal variables—such as level of education, years practicing forensic anthropology, and the amount of radiograph comparison case experience—affect the performance of the test subjects in making matches between radiographs? 10 Goals 1.To validate and establish accuracy and error rates for making identifications based on comparisons of lumbar spine radiographs. 2.To discern to what extent the performance of individuals on the x—ray comparison test is affected by personal attributes such as level of education, years practicing forensic anthropology, and the amount of previous case experience they have. 3.To evaluate which are the most reliable and most unreliable anatomical features to use when matching radiographs of the lumbar spine. 4.To offer recommendations to forensic anthropologists as to the most effective means for comparing radiographs of the spine in order to increase their chances of correctly identifying or excluding a decedent or correctly determining that a decision cannot be made based on the available information. Hypotheses Several hypotheses were developed to allow for meaningful statistical analysis of the data that were collected from the study participants. The hypotheses are as follows: H Al: The accuracy rate among all of the participants in the study for positive identifications using x-ray images of the lumbar spine will be 100%. H A2: The performance of professional forensic anthropologists will be significantly different from that of students. 11 H A3: Participants with different levels of education (PhD, MA/MS, BA/BS) will have significantly different performance on the x-ray matching test. H A4: Results achieved by participants with a greater amount of experience—years in the field, number of radiograph comparison cases, number of cases involving comparisons of x-rays of the spine, specifically—will be significantly different from those achieved by participants with less experience. H A5: Participants who work within academia will be significantly different from participants who work outside of academia. H A6: The number of anatomical features cited by the participants with greater education and experience will be significantly different from the number of features cited by participants with less education and experience. 12 Chapter 3: Materials and Methods The materials for this study included two sets of radiographs—one simulating the antemortem condition of known individuals and the other simulating the postmortem (skeletonized) condition of unknown individuals—as well as a two-part data sheet/brief survey that the participants were asked to complete. R adiogrqohs All x-rays for this project were taken from a sample of cadavers that were used with permission from the Michigan State University Willed Body Program, Department of Radiology, Division of Anatomy. The cadavers used for this study were all donated to the Gross Anatomy Laboratory at Michigan State University through the Willed Body Program. Because the specimens are crucial for the education of medical students enrolled in the College of Osteopathic Medicine and the College of Human Medicine at MSU, it was only possible to dissect the spines from bodies that were scheduled for cremation at the end of the term during which the x-rays were taken, Spring 2007. It was not possible to know in advance (before taking the x-rays) which specimens would be most suitable for the simulated antemortem and postmortem radiographs that would constitute the main materials for this study. Bodies scheduled for cremation were x-rayed first. Once suitable simulated antemortem images were chosen, the 5 from which the spines would be dissected out were selected, and then it was possible to continue x-raying the other bodies that would serve as foils in the comparison test. The constraint of only being permitted to remove the spines from a small number of cadavers is a limiting factor 13 for this study. Bodies that were relatively consistent in size were chosen so that the size of the vertebrae would not be an obvious distinguishing feature for the study participants, forcing them to focus on the more nuanced bony details. It also had to be possible to physically maneuver the bodies in order to be able to place the x-ray films beneath them, so the majority of the bodies chosen were of small to medium builds. Size, therefore, was another limitation of the sample that could be used for the antemortem, and subsequently, postmortem, x-rays for this project. Initially, 13 cadavers were available for dissection, 10 of which satisfied the body size criterion. Another 19 cadavers that were not scheduled for cremation also satisfied the body size criterion, making the total number of bodies that could potentially be used in this study 29. All 29 were x-rayed, and based on the quality of the resulting images, only 19 were useable for the study itself. As a result, two of the 20 simulated antemortem images (Known 2 and Known 19) are of the same individual, neither of which were matches for any of the simulated postmortem images. For details regarding specific cadavers used for this study, see Appendix A. The x-rays for this study were taken using a General Electric Amx2 Portable X- Ray machine, which is housed in the Michigan State University Forensic Anthropology Lab. Three duplicate sets of x-rays were created, using an XMA 3600 Duplicator, so that identical sets of study materials could be sent simultaneously to three separate locations, thereby maximizing the number of possible participants and minimizing the time it would take to collect responses from all of the participants. The simulated antemortem radiographs were taken according to standard clinical guidelines for KUB (Kidneys, Ureters, Bladder) Anterior-Posterior abdominal x-rays (Bontrager 2004; Brogdon 1998; 14 Evans et al. 1981). With the body lying supine, legs extended, and the arms at the sides, the beam was aligned in the mid-sagittal plane and was oriented perpendicularly to a point centered at the level of the iliac crests. Care was taken to ensure that the iliac crests were level and that there was minimal rotation to the bodies. The x-ray settings varied depending upon the completeness and the size of the bodies—the kVp setting ranged from 65 to 85, and the mAs setting ranged from 40 to 64. The wide range in kVp and mAs settings is due to varying degrees of dissection of the bodies, the increased density of embalrned, in some cases long-preserved internal organs, and the collection of embalrning fluid in the abdomens. The simulated postmortem sample was generated by removing the lumbar vertebrae from a subgroup of the individuals for which simulated antemortem radiographs had already been taken. The vertebrae were dissected fiom the bodies, macerated, and cleaned according to the procedures outlined by Fenton et al. (2003). They were then were allowed to dry for at least 24 hours before they were x-rayed a second time. Figure I .' Lumbar vertebrae were de-fleshed before they were cleaned according to the procedures outlined by Fenton et al. (2003) 15 In any forensic case where comparisons of radiographs are involved, the positioning of the elements and the x-ray beam itself are crucial in order to maximize the potential for a meaningful comparison (Ciaffi et al. 2010; Jablonski and Shum 1989; Kahana et a1. 2002; Koot et al. 2005; Kuehn et a1. 2002; Martel et a1. 1977; Mundorff et al. 2006; Quatrehomme et a1. 1993; Sauer et al. 1988; Weiler et a1. 2000). With this in mind, special attention was paid to the orientation of the vertebrae, which were matched as closely as possible to the orientation of the vertebrae of the same individuals from the antemortem films. The beam was consistently perpendicular to the spines being radiographed. To maintain consistency, a wooden brace was used to hold a dowel, which ran through the vertebral foramina to keep the vertebrae vertically aligned, and florist foam was used to brace the spinous processes so the vertebrae would not shift side-to- side during the x-ray process (see Figure 3). The x-ray settings for the simulated postmortem radiographs were much more consistent; the kVp setting ranged from 55 to 65 and the mAs setting ranged from 20 to 25. Figure 2: V ertebrae were held in position over the x-ray film using a wooden dowel in a brace and florist foam 16 Selection of Participants The selection of participants for this project began with an e-mail that was sent to all of the Diplomates of the Physical Anthropology section of the American Academy of Forensic Sciences. The Diplomates were chosen to be the core of the sample because they have all earned their PhDs, and because there is a minimum level of knowledge and experience that must be satisfied in order to earn that distinction. Because one of the objectives of this study was to evaluate the extent to which an individual’s levels of experience and education affect their performance, a sample of graduate students was also selected to take the test for comparative purposes. Therefore, all of the participants in this study are either professional forensic anthropologists or forensic anthropology graduate students who were drawn from a nonrandom sample of volunteers around the United States. Over a period of two years, from November 2007 through November 2009, the study materials were sent out to participants around the United States. Initially, 40 peOple volunteered to participate in the study and the expectation was that each participant would have the study materials in his/her possession for no more than 2-3 weeks; however a number of participants kept the materials for months at a time, due to a variety of personal and professional extenuating circumstances. The average time the packages spent at any one location was approximately 2 months. Because of the extended time period for data collection, it was impossible to send the materials to all of the anthropologists and graduate students who initially volunteered while maintaining a reasonable timeline for the completion of the project. In all, 25 of the volunteers were able to complete the x-ray matching test and return their data sheets to Michigan State 17 University. Each of the study participants was presented with a set of twenty radiographs (numbered 1-20) that simulated the antemortem images of known individuals to compare against ten radiographs (lettered Al/A2-B1/EZ) that simulated the postmortem images of five unknown individuals. Two different exposures of each of the five “postmortem” spines were included in order to maximize the participants’ ability to compare various details. The participants were not provided with information regarding the age, sex, or ancestry of the individuals in the x-rays being compared. See Appendix B and Appendix C for images of the radiographs that were presented to the study participants. Data Sheets This project complies with the requirements and the approval of the Michigan State University Institutional Review Board. Each participant was asked to complete a two-part data sheet, which was coded using a 3-digit, randomly generated number. These numbers were strictly for the researcher’s use during the data collection and analysis phase of this research; they are no longer linked in any way to the participants themselves because the anthropologists and graduate students who volunteered for this study are from a small, nonrandom group of colleagues, making it imperative to maintain confidentiality of their identities. They were asked to return the data sheets to the researcher in a self-addressed-stamped-envelope along with their signed consent forms. When the envelopes arrived at Michigan State University, the letters were opened and the data sheets were separated from the consent forms. The consent forms were then stored in a folder inside of a locked filing cabinet and were kept separate from the data sheets. 18 Part 1 of the data sheet asks for a number of personal details, such as level of education, years practicing forensic anthropology, number of cases involving identifications made by comparing radiographs, and the number of cases specifically involving radiographs of the spine. Part 2 of the data sheet includes five sections, asking which of the twenty “known” radiographs correspond to the five lettered “unknown” images. Beneath each of these questions, the participants were asked to indicate how many individual points of similarity they were able to identify between the radiographs and “which specific anatomical features” they used to make the identification. A final open-ended statement at the bottom of the page invited the participants to add any additional information they wished to include. See Appendix D for sample data sheets. 19 Chapter 4: Results This study involved a number of independent and dependent variables that were recorded from the data sheets each participant completed. The following table (Table 1) outlines the variables that were compared with one another in the two phases of analysis: contingency tables and in the nonparametric tests. Table I : Independent and Dependent Variables Used in this S tuay Independent Variables Total Sample Highest Degree (BA/BS, MA/MS, PhD) Number of radiograph cases PhD or Not PhD Any radiograph cases involving the spine Student or professional Number of radiograph cases involving the spine Students Professionals Highest Degree (BA/BS, MA/MS) Highest degree (BA/BS, MA/MS, PhD) Years in graduate school PhD or not PhD Type of graduate program Years practicing professionally Expected degree Work primarily within or outside of academia ABD or not ABD“ Number of radiograph cases Number of radiograph cases Any radiograph comparison cases involving Any radiograph comparison cases the spine involving the spine Number of radiograph cases involving the Number of radiograph cases involving the spine spine Dependent Variables Score out of 5 Potential outcomes include: 5/5, 4/5 (non-identification), 4/5 (misidentification), 35 (non- identifications), 3/5 (misidentifications), 3/5 (1 non-identification, 1 misidentification) Error/No Error In this category, scores are divided according to whether the participant answered with a misidentification or with either a correct answer or a non-answer. Number of Points This category represents the number of specific anatomical features each participant cited when making matches. It was evaluated as part of this study to see whether a relationship exists between the independent variables and the number of points used to make a match. 20 A ccuracy and Error Rates The primary goal of this study was to validate the comparison of radiographs of the lumbar spine for the purpose of identifying and unknown decedent. This study is based on a repeatable test for assessing the performance of practitioners in making identifications based on the comparison of antemortem and postmortem radiographs of the lumbar spine. For the purpose of this test, abdominal A-P x-rays were taken, according to clinical guidelines for diagnostic procedures, of 19 cadavers in varying degrees of abdominal dissection. These radiographs simulate the antemortem condition of 20 known individuals. The spines of 5 of the cadavers of which simulated antemortem x-rays had been taken were removed, de-fleshed, rearticulated, and x-rayed a second time to simulate the skeletonized postmortem condition of 5 unknown individuals. Because all of the “unknown” individuals were derived from the sample of “known” individuals, all 5 had the potential to be correctly matched. Therefore, a correct answer for a given identification means that the participant correctly matched the simulated antemortem and postmortem radiographs that were taken of the same individual. There are two classes of incorrect answers: 1)Misidentifications, where the participants identified one or more of the “unknown” individuals as the incorrect “known” individual; and 2) Non-identifications, where the participants failed to find any corresponding “known” individual for one or more of the “unknowns.” Only the misidentifications are considered in this study to be errors. Non-identifications are neither considered to be correct answers, nor errors. There was no potential for the participants to identify true negatives or false negatives as part of this study because all of the “unknown” individuals had matches in the sample of “known” individuals. The term, 21 “identification,” will be used throughout the remainder of this document to refer to matches between radiographs. As was discussed in Chapter 1, for an identification to be a “positive identification” in the truest sense of the term, more information would have had to have been provided to the participants—this will be addressed again in Chapter 5. Sixteen practicing professional forensic anthropologists and 9 graduate students participated in this study. Professional anthropologists have earned either an MA/MS or a PhD and are either currently working as forensic anthropologists or have worked as such at some point during their careers. They range in professional experience fi'om 1-5 years in the field to more than 20, and in radiograph comparison case experience from 0 to more than 100 cases. The 9 graduate students have earned either BA/BS or MA/MS degrees and are all currently enrolled in forensic anthropology or physical anthropology Masters or Doctorate programs. They range in years of graduate school from less than 1 to more than 6, and in radiograph comparison case experience from 0 to more than 20 cases. Table 2 (page 24) presents a summary of the performance of the graduate students and the professional anthropologists on the x-ray matching test. Of the 16 professional anthropologists, 3 have earned a MA/MS as their highest degree, and 13 have earned their PhD. Of the 3 professionals with their MA/MS, I scored 100% on the x-ray matching test, 1 misidentified Unknown D, and 1 misidentified Unknown E. Eleven of the 13 PhDs scored 100% on the matching test; 1 PhD misidentified Unknown D and l misidentified Unknown E; the PhD who misidentified Unknown E also chose not to identify Unknown D; and 1 PhD chose not to identify Unknown D or Unknown E. Because the sample of professional anthropologists who have earned an MA/MS 22 as their highest degree is so small, for the purposes of presenting the results of this study, professionals will be taken together as one group. In all, the 16 professional anthropologists were asked to make an aggregate of 80 identifications. Seventy-three of those identifications were made correctly, there were 4 instances of misidentification and there were 3 instances where the professionals declined to make matches. Therefore, the accuracy rate for professional anthropologists is 91.25% [calculatedz total correct/(total Mis-IDs + total Non-IDs + total correct)] and the error rate for professional anthropologists is 5% [calculated: total incorrect/(total correct + total Mis-IDs + total Non-IDs)]. 23 Table 2: Participant Performance # Correct False Positives No Subject (True Positives) # Incorrect (Misi den tifica tions) Identification Accuracy Total Possible = 5 Attempted Graduate Students 1 5 0 0 0 1.00 2 3 2 2 0 0.60 3 5 0 0 0 1.00 9 5 0 0 0 1.00 10 5 0 0 0 1.00 11 4 l 0 l 0.80 19 4 l 0 l 0.80 20 5 O 0 0 1.00 25 5 O 0 0 1.00 Total: 41 4 2 2 0.91 41/45 (91.1%) 4/45 (8.9%) 2/45 (4.4%) 2/45 (4.4%) Professionals 4 5 0 0 0 1.00 5 5 0 0 0 1.00 6 4 1 l 0 0.67 7 5 0 0 0 1.00 8 4 1 l 0 0.80 12 5 0 0 O 1.00 13 5 0 0 0 1.00 14 5 O 0 0 1.00 15 5 0 0 0 1.00 16 3 2 0 2 0.60 17 3 2 l l 0.60 18 4 l 1 0 0.80 21 5 0 0 0 1.00 22 5 0 0 0 1.00 23 5 0 0 0 1.00 24 5 0 0 0 1.00 Total: 73 7 2 3 0.91 73/80 (91.25%) 7/8018.8%) 4/80 (5%) 3/80 (3.8%) Group Total: 114 11 4 5 0.91 114/125 (91.2%) 11/125 (8.8%) 4/125 (3.2%) 5/125 (4%) 24 Of the 9 graduate students who participated in this study, 6 scored 100% on the x- ray test, I graduate student misidentified Unknown C and Unknown D, and 2 graduate students chose not to identify Unknown D. The 9 graduate students were asked to make an aggregate of 45 identifications. Forty-one of those identifications (91.11%) were made correctly; there were 2 instances of misidentification (4.44%); and there were 2 instances where the graduate students declined to make one identification each (4.44%). The accuracy rate for graduate students, therefore, is 91.11% [calculatedz total correct/(total correct + total Mis-IDs + total Non-IDs)]. The error rate among graduate students is 4.44% [calculatedz total incorrect/(total correct + total Mis-IDs + total Non- IDs)]. One graduate student is responsible for both of the misidentifications associated with graduate students; that student noted on his/her data sheet that he/she had been attending graduate school for less than one year at the time. In all, there were 25 participants in this study who were asked to make a total of 125 identifications (25 participants x 5 identifications each = 125 total identifications). Of the 125 total possible identifications, 114 (91 .2%) were made correctly, there were 5 (4%) misidentifications, and there were 6 (4.8%) instances where participants declined to make a match based on the available information. The participants in this study have an overall accuracy rate of 91.2% [calculatedz total correct/(total correct + total mis-IDs + total non-IDs)]. The overall error rate is 4% [calculated: total incorrect/(total correct + total mis-IDs + total non-IDs]. The discrepancy between the accuracy rate and the error rate lies in the fact that only the misidentifications are considered to be true errors. When only attempted matches are taken into consideration for these calculations, in other words, when non-identifications are removed from the calculations, the overall accuracy 25 rate increases to 95.8% [calculated: total correct/(total correct + total mis-IDs)]. If this calculation is performed for professionals and graduate students separately, in other words when only attempted matches are taken into consideration for each group, the accuracy rate among professionals increases to 94.81% and among graduate students to 95.34%. The significance of the observed differences presented above is discussed in the following section. Effect of Personal Attributes on Performance The second goal of this study was to assess whether personal attributes, such as level of education and amount of experience, affected the participants’ performance in making matches between the simulated antemortem and simulated postmortem images. This section presents the results of the statistical tests that yielded significant results concerning the relationships between the participants’ personal attributes and their performance. As was discussed in the previous chapter, two separate statistical tests were carried out using the data that were collected during the course of this study. First, contingency tables were used as descriptive statistics to explore whether any of the independent and dependent variables were acting together. Second, once it was established that some of the variables were not acting independently, the data were subjected to independent samples nonparametric tests—either the Kruskal-Wallis Independent Samples test or the Mann-Whitney U Independent Samples test, depending upon the type of data and the number of possibilities for each variable being compared. The independent variables (see Table 1, page 20) included personal attributes such as level of education, years practicing, and the amount of radiograph experience the 26 participants have had. The dependent variables include the participants’ scores on the matching test, whether they answered all of the identification questions correctly, misidentified, or chose not to identify one or more of the simulated unknown individuals, and the number of specific anatomical features they used to make the identifications (see Table 1, page 20). Contingency Tables f The first step in the data analysis was to carry out contingency tables where each fl of the independent variables was tested to see if there were relationships between them and each of the dependent variables. The vast majority of the contingency tables failed to indicate the existence of a relationship between the variables; however relationships were found to exist between five of the independent variables and several of the dependent variables. The following section presents the results of the contingency tables, which were used to evaluate the existence of relationships between the variables, and nonparametric tests which were carried out to determine if the aforementioned relationships, where found, are statistically significant. Because of the small sample size, all of the contingency tables produced using the data that have been collected for this study contained a number of cells with expected values of less than five or zero. This is known to artificially inflate the value of chi-square, therefore increasing the probability of finding significant differences. This must be considered when interpreting the results of the contingency tables. (See Appendix E for significance values associated with the contingency table phase of data analysis.) 27 Total Sample Out of all the contingency tables that were carried out to evaluate the presence of relationships between the independent and dependent variables—taking the entire sample into consideration (n = 25)—only two yielded significant results. The contingency tables indicate the following variables do not act independently: Student or Professional and A verage Number of Points, and Number of Radiograph Cases and Score out of 5. Only 22 valid cases exist for the variable, A verage Number of Points, because one professional did not answer the question about how many points he/she identified when making matches, and two indicated that they used “multiple” or “many” points of similarity when making matches—answers that could not be quantified for this type of analysis. The contingency table involving Student or Professional and A verage Number of Points yielded a significance value of p = 0.005. All nine students cited an average of less than 10 points of similarity between radiographs. Four of the professionals cited less than 10 points, 7 cited an average of 11-20 points, and 2 cited an average of over 20 points when making their matches. It is clear that within the parameters of this study, students cited fewer average points of similarity than their professional counterparts when making matches between radiographs. The second significant relationship indicated by a contingency table, taking the entire sample into consideration, is between Number of Radiograph Cases and S core out of 5. This contingency table yielded a significance of p = 0.039. Three of the participants have had zero case experience involving radiograph comparison. Of those three, one scored 5/5, one scored 4/5 (non- identification), and one scored 3/5 (two misidentifications). Eight of the participants have had experience with 1-10 radiograph comparison cases, five of whom scored 5/5 28 W“— and three of whom scored 4/5 (misidentification) on the matching test. The remaining 14 participants have worked on more than 10 cases involving radiograph comparisons, 11 of whom scored 5/5, I scored 4/5 (non-identification), I scored 3/5 (2 non-identifications), and 1 scored 3/5 (1 misidentification/ 1 non-identification). The results from this comparison may be affected by the small sample size. The number of participants in this study who have worked on more than 10 radiograph comparison cases is higher than the number of participants in the other two categories combined. It is not surprising, therefore, that a greater number of those participants also scored higher on the x-ray matching test. This relationship is further discussed in the following chapter. Professionals Only Significant results were produced by two of the contingency tables that examined variables relevant to only the professionals: Years Practicing Professionally and Score out of 5, and Number of Radiograph Cases and Score out of 5. In both of these cases, the small sample size (n = 16) likely affected the outcome of the contingency tables. When the relationship between Years Practicing Professionally and Score out of 5 was evaluated, the contingency table produced a significance value of p = .039. The number of professionals who have been practicing for greater than 20 years is more than double the number of professionals who have worked for 10 or fewer years. It is not surprising, therefore, that the number of professionals who have been practicing for many years and scored 5/5 is larger than the number of professionals who have been practicing for fewer years—there are simply more of them. When the relationship between Number of Radiograph Cases and Score out of 5 29 was evaluated, the contingency table produced a significance value of p = .039. A similar problem exists with this relationship as exists with the relationship discussed immediately above, in that the number of professionals who have worked on more than 10 radiograph comparison cases more than doubles the number of professionals who have worked on fewer than 10 radiograph comparison cases. It is possible that while these two variables appear to be acting together, it may be a function of the small sample size. Students Only While relationships were detected by contingency tables when the entire sample and professionals are considered, none of the contingency tables that were carried out involving only students indicated that any of the variables were acting together. The lack of significance is likely related to the small number of students (n = 9) among the participants in this study. Tests for Significance Subsequent to the contingency table phase of analysis, each of the independent variables was then calculated against each of the dependent variables to assess whether any relationships between the variables are statistically significant. This process was carried out for each of the variables in part to verify the results of the contingency tables—to make sure that relationships do not exist where the contingency tables did not find relationships, and to evaluate the significance of the relationships that were detected. (See Appendix F for significance values associated with the non-parametric tests phase of this analysis.) 30 The relationships between three sets of variables were found to be statistically significant. When only professionals were taken into consideration (n = 16), Years Practicing Professionally was found to be related to Score out of 5 and Number of Radiograph Cases was found to be related to Score out of 5. Student or Professional was found to be related to Average Number of Points, when the total sample (n = 22, three of the participants did not provide information regarding the number of points they cited) was taken into consideration. When only students were taken into consideration, no statistically significant relationships were detected among the variables. When the entire sample (n = 22) was taken into consideration, only the relationship between the independent variable Student or Professional and the dependent variable Number of Points was found to be significant. For the analysis in this case, the Independent Samples Kruskal-Wallis Test was chosen because it is the appropriate test to use when there are greater than two possibilities for any one of the variables. The Kruskal-Wallis test produced a significance of 0.030. Professionals, who have a greater level of education and have been actively working as forensic anthropologists for more years than any of the students, cited significantly more points on average than any of the students. There are several different possible interpretations for this relationship, which will be discussed in the following chapter. When only professionals (n = 16) are taken into consideration, two relationships between variables appear to be statistically significant. Years Practicing Professionally appears to have a statistically significant relationship with Score out of 5 and Number of Radiogrqoh Cases also appears to have a statistically significant relationship with Score out of 5. Both cases indicate that participants with more experience performed better on 31 the x-ray matching test. The Independent Samples Kruskal-Wallis Test was used in both cases, and in both cases the significance value was 0.049. The results, particularly from these two tests, allows us to reject the Null Hypothesis that assumes that the participants’ performance would be consistent regardless of their level of experience and it indicates that with greater levels of experience, the participants are expected to perform better on the x-ray matching test. This will be discussed further in the following chapter. 32 Chapter 5: Discussion Throughout the data analysis phase of this project, several points became clear, and they will be discussed in detail in the following section. First, it was found that the amount of experience the observers have appears to be directly related to their performance on the x-ray matching test. Second, many of the other personal attributes that were expected to be significantly related to performance were not. Third, there appears to be a direct relationship between the observers’ experience and the number of specific anatomical features they cited when making identifications. All three points will be discussed in greater detail below. Effect of Personal Attributes on Performance Students It is not surprising that neither the contingency tables nor the nonparametric tests indicated significant relationships between the variables that were tested when only students were taken into consideration. This is most likely due to the small sample size (n = 9). Variables like, A BD or not A BD (ABD stands for “All But Dissertation,” and refers to students in doctoral programs who have satisfied all of their degree requirements except for the dissertation) and Type of Program were found to have no bearing on the performance results of the participants. Only one student was ABD at the time the data were collected, and that student did not out-perforrn students who were not ABD, so there is no way to deduce a relationship between that variable and their performance. Six of the 9 students indicated on their data sheets that they are enrolled in Forensic Anthropology programs; 4 of those 6 are enrolled in Master’s programs; the other 2 are 33 E77:‘_"_'% enrolled in PhD programs. Of the 3 students who are not enrolled in exclusively Forensic Anthropology programs, 1 indicated that he/she is enrolled in a Forensic Anthropology/Physical Anthropology program, 1 indicated that he/she is enrolled in a Forensic Anthropology/Bioarchaeology program, and 1 indicated that he/she is enrolled in a Physical Anthropology program. All of the students are attending universities with active forensic labs, and therefore probably have similar access to and participation with forensic cases. It is for these reasons that Type of Program is believed to have no influence on the students’ performance with regard to the x-ray matching test. Though there were no significant relationships detected, there are some notable differences in performance among the student group. Six students scored 5/5—3 with a highest degree of MA/MS and 3 with a highest degree of BA/BS. Two students scored 4/5 (non-identification), l with a highest degree of MA/MS and 1 with a highest degree of BA/BS. One student, with a highest degree of BA/BS, scored 3/5 (misidentifications). Of the two students who declined to make one match each, 1 had been in graduate school for at least 4 years and one had been in graduate school for less than 2 years at the time of data collection. Among the students, only two errors were committed, and both were committed by the same person. The student who scored 3/5 (misidentifications) is the only person out of the entire sample of participants (11 = 25) who misidentified two of the Unknowns, constituting the highest number of mistakes made by any single participant. The graduate student who misidentified two of the simulated Unknowns had been in graduate school for less than one year at the time of data collection, which may suggest that this person’s lack of experience, both with regard to coursework and to case experience, may have led to his/her poor performance. 34 The two students declined to make one of the matches each both chose not to identify Unknown D. One of the two indicated that he/she could not find a match for Unknown D, and the other indicated that he/she was not satisfied with the number of points of similarity he/she was able to find between Unknown D and any of the Known radiographs (see Appendix G for the answers participants provided for the questions asked on the data sheets). A non-identification is neither a correct answer nor an error because it is a conservative approach to the problem and those who answered this way would not come away having misidentified a decedent. Professionals In response to conclusions of previous research (Hogge et al. 1994; Koot et al. 2005), this study explored whether PhDs perform better on the test than MA/MS professionals. Only 3 of the professionals who participated in this study have a highest degree of a MA/MS; the other 13 have earned PhDs, and there was no statistically significant relationship between whether the participants have earned a PhD or a MA/MS and their performance. Two of the 3 MA/MS professionals scored 4/5 on the matching test, both of whom misidentified one individual each. Two of the 13 PhDs misidentified one individual each. The sample of MA/MS is far too small to draw any meaningful conclusions from a comparison between MA/MS and PhDs. As expected, the data analysis for this project revealed a significant relationship between Years Practicing Professionally and Number of Radiograph Cases when those variables were evaluated with respect to Score out of 5. This study produced results that support the conclusions of previous researchers whose studies focused on other regions of 35 the body and found that observer experience is an important variable. It was somewhat surprising to find that the number of radiograph cases involving the spine was not significantly related to the participants’ performance. Thirteen of the 16 professionals included in this study have had experience comparing x-rays of the spine for the purpose of positively identifying an unknown decedent, so it is possible that the sample size is too small to draw a meaningful conclusion regarding this variable. It is not possible to know, however, whether the number of professionals lacking experience with radiograph cases involving the spine would increase given a larger sample size. Total Sample Several trends are notable in the data analysis phase of this project. Though, proportionally, students appear to perform similarly to professionals on the x-ray matching test, making the statistical tests produce insignificant results when the relationships between variables were evaluated, there are some nuances that require further discussion. Though it was expected that students would not perform as well as professionals on the x-ray matching test, it was found that the difference in performance between professionals and students was not significantly significant, so dividing the sample into student/professionals may not be a meaningful distinction. It is possible that this particular comparison is affected by the disproportionately greater number of professionals in the sample, and with a larger number of student participants the result may have been different. It is possible, however, that the lack of a significant difference between students and professionals could be the result the nature of the student sample 36 included in this evaluation. The students may be predisposed to perform well on such a test because they are all enrolled in graduate school programs focused on forensic anthropology and physical anthropology, therefore they are already a self-selected group of people who have expressed interest in human identification and skeletal anatomy. It is possible that their interest in human identification and skeletal anatomy may enhance their ability to do well on the x-ray test. The majority of them performed very well on the x-ray matching test, even though most of the students had been in graduate school for no more than 3 or 4 years. The results may have been different given a larger sample size; however it is not possible to know. Significant relationships were found to exist regarding only a handful of the variables: Highest Degree, Student or Professional, and Number of R adiogrqoh Cases. The only significant relationship that existed between the variable Student or Professional and any of the dependent variables involved the A verage Number of Points they cited when making matches between the radiographs. All 9 of the students cited an average of 10 or fewer points of similarity between radiographs, while 9 of the professionals cited an average of over 10 points, and only 4 averaged 10 or fewer. Only 13 of the 16 professionals are included in this analysis; the remaining 3 did not provide information on their data sheets regarding the number of points of similarity they were able to identify between the radiographs. A significant relationship was found to exist between Highest Degree and the number of points the participants cited as well. In the sample, there are 5 participants with a highest degree of BA or BS, 7 with a highest degree of MA or MS, and 13 with a highest degree of PhD. The lack of evidence for a significant relationship between 37 Highest Degree and the dependent variables is probably because proportionally, the three groups performed similarly on the x-ray matching test. All 5 BA/BS participants cited an average of 10 points or fewer when they made matches; 4/7 MA/MS participants cited an average of 1-10 points and the other three cited an average of 11-20 points. Four of the 10 PhDs who answered this question on the data sheet cited an average of 1-10 points, 4 cited an average of 11-20 points, and 2 cited an average of over 20 points. PhDs are the only group that included any participants who cited an average of over 20 points, while the BA/BS are the only group who cited exclusively 1-10 points, which makes it appear as though the participants with more education are citing more specific points of similarity between radiographs than those with lower levels of education. There is some amount of discrepancy among participants with regard to the interpretation of what was meant by “specific anatomical features” as it was indicated on the data sheet, which may have an effect on the results presented above. This will be discussed in further detail later in this chapter. When Number of Radiograph Cases was tested against Score out of 5, a significant result was revealed. Three of the 25 participants have had zero experience with cases involving the comparison of radiographs. Eight of the participants have worked on 1-10 cases involving radiograph comparison, and 14 have worked on greater than 10 radiograph comparison cases. Of the 3 participants with zero case experience, I scored 5/5, I scored 4/5 (non-identification), and I scored 3/5 (misidentifications). Five of the 8 participants who have worked on 1-10 radiograph comparison cases scored 5/5 and the other three scored 4/5 (misidentification). Eleven of the participants who have had greater than 10 cases involving radiograph comparison scored 5/5, 2 scored 4/5 (non- 38 identification), I scored 3/5 (non-identifications), 3 scored 4/5 (misidentification), and I scored 3/5 (1 misidentification and 1 non-identification). Two of the 4 misidentifications made by participants were made by a single student who had been in graduate school for less than one year at the time of data collection and who has zero case experience, which may lend even more credence to the idea that performance will increase with increased experience. Of the remaining three, two were made by professionals with a highest degree of MA/MS and the other was made by a PhD, with a qualifying statement. The participants in this study were provided with no information regarding the age, sex, or ancestry of the cadavers used for the radiographs. This was intentional because this study was designed to test the participants’ ability to compare details visible on the x-ray films and it was not expected to have an effect on the participants’ ability to correctly match the simulated antemortem and postmortem images of the same individuals (Koot et al. 2005; Weiler et al. 2000). Several of the participants indicated on their data sheets that they would need additional information to consider any of their matches to be positive identifications, which is an important clarification to make. While for the purposes of this study, the matches are being considered “identifications,” they could not possibly be positive identifications unless all of the biological information associated with the missing person and the decedent also matched. Problematic Cases The majority of the errors and non-matches that were committed by the participants in this study involved Unknown D or Unknown E. Unknown D was misidentified 3 times, not identified 4 times, and several of the participants noted at the 39 bottom of their data sheets that they would not consider their correct match of Known 20 to Unknown D a positive identification. Unknown E was not matched 1 time and was mismatched by 2 participants—one of whom was a PhD who has been practicing for greater than 20 years and has extensive casework experience, and qualified his/her incorrect answer by saying that it was “not something I would consider a positive identification without further study.” These two cases are revisited in more detail below. Problematic Case: Unknown D Because Unknown D proved to be problematic for so many of the participants, the images were revisited to evaluate what the problem might have been. There are three immediately obvious factors that likely resulted in the participants’ misidentification of Unknown D or unwillingness to match Unknown D to Known 20, and which likely speak to problems inherent in any x-ray comparison. First, the skeletal structures in the simulated antemortem image were obscured due to the fact that the cadaver’s abdomen was intact at the time of x-ray. The dense organs and collection of embalrning fluid in the abdomen led to a radiographic “antemortem” image that appears to be underexposed. This is unfortunately one of the limitations of a study, like this one, that involves cadaver tissue. The potential for the visual obstruction of skeletal features by internal organs and soft tissue is often a factor when comparing antemortem and postmortem radiographs because many clinical x-rays are taken for diagnostic purposes concerning soft tissues. Second, the orientation of the vertebrae in the simulated postmortem image is not exactly matched to that of the simulated antemortem image. Though the vertebrae are aligned as closely as the researcher could make them, they are not quite perfect, which 40 may have hindered participants from making a match. A third potential complication involved in this identification is that the cadaver used for Unknown D/Known 20 had a congenital defect that resulted in the individual only having 4 lumbar vertebrae. The cadaver’s 4th and 5th lumbar vertebrae were fused as one vertebra with bifid transverse processes. This was not clear on the simulated antemortem image and the existence of only 4 lumbar vertebrae in the image was not explained to the participants, who may have been focusing on non-corresponding details as a result (see Figure 3). Figure 3: Unknown D (left) and Known 20 (right). This illustration shows the difference in orientation between the two images and the poor image quality of Known 20. 41 Due to the problematic nature of Known 20, the accuracy rates were reexamined to evaluate the performance of the study participants in cases where the images are conducive to making matches. When Unknown D/Known 20 is removed from consideration, the number of misidentifications among the total sample of participants drops from 4% (5/125) to 3% (3/100), the number of cases where participants declined to make a match drops from 4.8% (6/125) to 1% (l/ 100), and the accuracy rate of the total sample increases from 91.2% to 96%. Among individual groups the difference is even more dramatic. The overall accuracy rate among professional forensic anthropologists increases from 91.3% (73/80) to 95.3% (61/64); instances of misidentification decrease from 5% (4/80) to 3.1% (2/64) and cases where participants declined to make matches decrease from 3.8% (3/80) to 1.6% (1/64) . Accuracy among PhDs increases from 92.3% (60/65, including misidentifications and non-matches as incorrect answers) to 96.2% (50/52), and to 98% (50/51 attempted matches) when the non-matches are removed from consideration. Among graduate students, accuracy increases from 91.1% (including misidentifications and non-matches as incorrect answers) to 97.2% (only one misidentification and no non-matches). See Table 3, on the following page. 42 Table 3: Participant Performance, Excluding Unknown D/Known 20 # Correct False Positives No Subject (True Positives) # Incorrect (Misi den tiflca tions) Identification Accuracy Total Possible = 5 Attempted Graduate Students 1 4 0 0 0 1.00 2 3 l 2 0 0.75 3 4 0 0 0 1.00 9 4 0 0 0 1.00 10 4 O 0 0 1.00 1 1 4 O 0 0 l .00 19 4 0 0 0 1.00 20 4 0 0 0 1.00 25 4 0 0 0 1.00 Total: 35 1 2 0 0.972 35/36 (97.2%) 1/36 (2.8%) 1/36 (2.8%) 0/36 (0%) Professionals 4 4 0 0 0 1.00 5 4 0 0 0 1.00 6 4 0 0 0 0.67 7 4 0 0 0 1.00 8 3 l l 0 0.80 12 4 O 0 0 1.00 13 4 0 O 0 1.00 14 4 0 0 0 1.00 15 4 0 0 0 1.00 16 3 1 0 1 0.60 17 3 l I O 0.60 18 4 0 0 0 0.80 21 4 0 0 0 1.00 22 4 0 0 0 1.00 23 4 0 O 0 1.00 24 4 0 0 0 1.00 Total: 61 3 2 l 0.91 61/64 (95.3%) 3/64 (4.7%) 2/64 (3.1%) 1/64 (1.6%) Group Total: 96 4 3 l 0.96 96/100 (96%) 4/100 (if/Q 3/100 (3%) 1/100 (1%) 43 Problematic Case: Unknown E The extent of osteophyte development was the most often cited criterion for making a match between Unknown E and Known 3 (having been cited a total of 28 times by 24 of the 25 participants). Both of the participants who mismatched Unknown E suggested that its “antemortem” match was Known 15, and both of them cited osteophyte development among their primary criteria. It is probable that the two participants who mismatched Unknown E with Known 15 focused primarily on the extensive osteophyte development between L2 and L3 along the right side of the vertebral column. Similar osteophyte development is visible in the images of both Known 3 and Known 15. When other features are taken into consideration, however, it becomes apparent quickly that Known 15 cannot be a match for Unknown E. Figure 4 (page 45) presents images of Known 3, Unknown E, and Known 15. The large white circles in all 3 images indicate the similar osteophyte development along the right side of the vertebral column. Other similarities between Known 3 and Unknown E are also circled in white, while black circles highlight some examples of differences between Known 15 and Unknown E. Though the osteophyte development along the right side of the vertebral column may be similar, there are marked differences between other features, for example: spinous process contours, expressions of osteophyte development, contours of the vertebrae, and the intervertebral spaces. This case was not missed nearly as frequently as Unknown D/Known 20, and there do not appear to be any problems with the quality of the images or the orientation of the vertebrae, so the participants’ performance was not reevaluated as it was for 44 Unknown D/Known 20. Figure 4: Known 3 (left), Unknown E (center), and Known 15 (right). With the three images side-by-side, the similarities between Known 3 and Unknown E and the differences between Unknown E and Known 15 become avparent. Features The number of points cited per identification ranged from 3 to 15. All students cited an average of less than 10 points of similarity when comparing the x-ray sets, while the overwhelming majority of professionals cited an average of greater than 10 points per identification. This may indicate that the professionals have more knowledge of terminology or they may be looking for more subtle details than the students, who generally have less experience (see Figure 5, on the following page). 45 Figure 5 .' Bar chart showing the average number of points of sim ilarity cited by students and professionals O Professionals I Students NNUJ M O y—a OUIO l j \\\\\\\\\\ \\\\\\\\\\\\\\\\\\ \\\\\\\\\ \\ I... , I I I1 IEIE“ I I2I2I,I 12 3 4 5 6 7 8 910111213141617202122232425 Participants (n=25"') / l Average Number of Points Cited by Partrcrpants 3 *Participants #15, #18, and #19 are missing from this graph. Participant #15 did not provide an answer for how many points he/she cited for each identification; Participants #18 and #19 answered “Multiple” and “Many,” respectively, and their answers could not be quantified. The most frequently cited criterion used for making matches was the Spinous Process (size, shape, orientation), which was cited a total of 134 times by the 25 participants. The centrum (size, contour) and transverse processes (contour, trabeculae) followed closely, having been cited 117 and 96 times, respectively. The fourth most frequently cited features were age related changes, such as osteophytic lipping, syndesmophyte development, and arthritis, terms that are often used interchangeably. These features were cited a total of 79 times by the 25 participants (See Figure 6, on the following page). 46 Figure 6: Bar chart showing the number of times each type of vertebral feature was cited by participants Average Number of Times Features were Cited by Participants Specific Anatomical Features Correct answers were achieved using anywhere from 2 to over 40 points of similarity. In cases where the participants responded that no identification was possible, no points of similarity were ever noted. For incorrect answers where misidentifications resulted, anywhere from 3 to 11 points of similarity were noted. This brings up three important issues. First, a number of participants interpreted some of the terminology in the open-ended questions on the data sheets differently. Participants were asked, “How many points of similarity did you find when making this match?” and “Which specific anatomical features did you find most useful in making this identification?” Several 47 participants noted that there was some confusion with regard to what was meant by “point” of similarity (see Appendix G for the participants’ answers to the questions on the data sheets). For example, “point of similarity” could either refer generally to the left transverse process of L2, or the same term could refer to any number of specific areas of radiolucency/density or specific locations along the contour of the same transverse process, thereby greatly increasing the number of “points” (see Figure 7, below). This is likely the reason for the large discrepancies between the number of points used by participants in making the matches between the Known and Unknown radiographs. Figure 7 .' A ‘point”may refer to the contour of a bony feature, or it may refer to any number of smaller details contained within the larger feature. 48 The graph below provides an example of one of the sample cases in this study that was matched correctly by all 24 participants. You can see that while everyone arrived at the same conclusion, the number of points cited ranged from 4 to almost 40. Figure 8: All participants answered the match between Unknown A and Known 14 correctly. Note the large disparity among the numbers of points cited by participants. Professionals I Students U. m \\\\\\\\\\\\\\\\\ \\\\\\\\\\\\\\\\\\\‘ \\\\\\\\\\\\\\\\‘ = L\\\\\\\\\\\\\\ u\\\\\\\\\\\‘ ll-gg '/ I . I. I. 12 3 4 5 6 7 8 91011121314161720212 Participants (n=25*) a\\\\\\\\\\\\\\\\\\\\\\\\‘ a\\\\\\\\\\\‘ \\\\\\\\ \\\\\\\\\\\‘ s\\\\\\\\\\‘ N-- Number of Points Cited for 8 Correct Match between Unknown A and Known 14 N O 23 24 25 *Participants #15, #18, and #19 are missing from this graph. Participant #15 did not provide an answer for how many points he/she cited for each identification; Participants #18 and #19 answered “Multiple” and “Many,” respectively, and their answers could not be quantified. It is problematic that misidentifications occur at all, especially when they are based on up to 11 points of similarity. In the cases where this occurred, it appears as though the observers are paying attention to the minute details or to potential non- individualizing age—related characteristics, without considering larger or more discriminating features. For example, when Unknown E was misidentified, it appears as though the participants were focusing on osteophyte development, while ignoring overall shape and proportions of the vertebral column (see Figure 4 on page 45). 49 A third problem that is notable regarding the “points” of identification is a lack of standardization in terminology among the study participants. The terms, “arthritis,” “syndesmophyte,” and “osteophyte” were repeatedly used by different researchers to describe the same features. For example, the feature pictured in the illustration below (Figure 9) was described by one participant as “arthritic activity,” by a second participant as “syndesmophytes,” and by a third participant as “osteophytes.” These terms refer to very different skeletal conditions that can have very different etiologies; however they are used interchangeably by participants in this study. It will be important in the future to standardize not only methods but terminology as well. Figure 9: The terms, syndesmophyte, osteophyte, and arthritis were used interchangeably to describe bony changes of the vertebral column (the vertebra shown is L3 from Unknown C/Known 9) 50 Limitations There are some limitations to this study that may have an impact on the success and failure of matches and on the strength of the conclusions that are able to be drawn from this research. First, because of the difficulty in generating a sample of antemortem radiographs of the abdomen for which a subsequent postmortem image can be made, this study made use of cadavers to simulate the ante- and postmortem conditions. Image quality varies substantially across the simulated antemortem set of radiographs, which is mainly due to the condition of the bodies at the time of x-ray (see Appendix A). Several of the bodies were missing the majority of their internal organs, which enabled the x-ray beam to penetrate the skeletal structures with relative ease. The bodies with more intact abdominal cavities posed somewhat of a problem because of the increased density of the tissues due to postmortem changes and the embalming process. Second, the average age at death for the cadavers that were used for this study is 77 years. Though this does not appear to have hindered anyone’s ability to make matches between the radiographs, it is a factor that must be considered, especially when discussing which criteria were most likely to be used in a correct match. The age-related changes that compose the fourth most frequently cited features, would not be present to such an extent in a younger population. Because all of the x-rays were taken of deceased individuals, there is no potential for skeletal changes between the time the “antemortem” and “postmortem” images were taken. According to Sauer et al. (1988), positive identifications can still be made after up to 2.5 decades have passed between the antemortem and postmortem x-rays. In other words, skeletal features can remain very constant for a considerable length of time, so the fact that there is no time depth in this 51 study does not necessarily diminish the quality of the conclusions. There were also constraints placed on the quality of the information collected for this study because of the way the data sheets were designed. There are two items that could be changed in the future if this study is repeated or if a similar study is conducted which may strengthen the conclusions that can be made. First, the participants should have been asked how much time it took them to make each identification. Three of the participants, all of whom answered all 5 of the identification questions correctly, revealed through personal communication that they spent 30 minutes, 2 hours, and over 10 hours making the identifications. Because all 3 of them scored 5/5, it is not believed at this time that the time people spend making the identifications necessarily has any bearing on their ability to correctly match x-ray pairs, but it would have been a more thorough study if this information had been systematically collected. The other methodological change that should be made would be to include at least one simulated postmortem image for which there is no corresponding simulated antemortem image. The accuracy and error rates presented in this study are somewhat weakened by the absence of potential for true negatives (correctly indicating that a particular image cannot be matched) and false negatives (incorrectly indicating that a particular image cannot be matched). The way this study was constructed, it left no potential for statistical analysis of specificity and sensitivity values, which would have added greatly to any statements made regarding the accuracy and error rates achieved by the participants. Another important limitation to this study is that the participant pool was small and was selected from a nonrandom self-selected group of volunteers. In order to obtain a better idea of whether this group of participants is representative of forensic 52 anthropologists in general, it would be necessary to have a much larger sample. It is unfortunate that not all of the individuals who had initially volunteered to participate were able to contribute to this study. Finally, this study does not test the participants’ ability to make positive identifications; it is an evaluation of their ability to correctly match radiographic images to one another. In order to accurately determine a positive identification of an unknown decedent, an observer would need critical information regarding the biological profile— age, sex, ancestry, and stature—of both the decedent and the individual to whom the antemortem records correspond. Information concerning the circumstances of death—for I example, location of the body, any identification present, time since death or disappearance—are also helpful in narrowing down the list of missing persons who could potentially be identified as the decedent. The single PhD who mismatched Unknown E and qualified his/her statement that he/she would not consider it a positive identification without further study was alluding to the fact that the study did not simulate an actual scenario because participants in this study were not provided with any of the information mentioned above. None of the matches that were made during the course of this study would be considered positive identifications in an actual forensic case. This brings up an interesting issue. It is possible that participants may have felt compelled to provide an answer to each question because they were aware that there was a correct answer possible for each of the radiograph matches. As a result, some of the participants may have made judgement calls, knowing there was a correct answer, that they would not have made in a testing situation that was more representative of the conditions of an actual forensic case. This may have led to an inflated number of 53 incorrect matches and a deflated number of non-identifications, and may account for many of the errors committed by the participants. While this study did not replicate an actual forensic case scenario, which may weaken any statements made regarding positive identification specifically, it is the process of matching radiographic images to one another to positively identify an unknown decedent that is in need of validation. The research design for this project is appropriate for such an evaluation. 54 Chapter 6: Conclusions The primary goal of this study is to validate the utility in comparing lumbar spine radiographs as part of the positive identification process. Professionals were expected to perform generally well on the x-ray comparison test that was set up for this study, as they are the ones who would be the most likely to be asked to testify in court. They were the main objects of the study; the graduate students were included mainly as a comparative sample. Therefore, the overall high accuracy rate among professionals was not surprising. Ideally, none of the professionals would have answered incorrectly, but would have erred on the side of caution and chosen not to make a match where they were not confident that it was correct. In no instance should a positive identification be made where the investigator is unsure of his/her conclusion. This is an issue that should not be taken lightly by the scientific community. Conservative science is good science, and it is problematic that any misidentifications occurred at all in the course of this study. That said, this study has shown that comparing radiographs of the lumbar spine for identifying unknown decedents is generally a robust technique, with an overall accuracy rate of over 90%. When only professionals evaluated and only attempted matches are considered (removing the non-identifications from the analysis), the accuracy rate increases to 98%. The ability for observers to correctly match radiographs of the same individual is greatly affected by the quality of the images with which they are provided, and like the conclusions presented by much of the earlier research concerning radiograph comparison, orientation of the vertebrae and angle of the x-ray beam are critical for this type of analysis. The detrimental effect of a problematic image became apparent in this study when the problematic case (Unknown D/Known 20) was eliminated so that the materials 55 included only comparisons made using high-quality antemortem images that show a large number of bony anatomical features. When the problematic case was eliminated fi'om consideration when evaluating the participants’ results, the overall accuracy rate increases substantially from 91% to 96%. In this study it was important to separate out the actual incorrect answers (the misidentifications) from the non-identifications. Though the relationship is not statistically significant, trends observed during the course of this study indicate that observers who have more experience are less likely to attempt a match that is negatively impacted by confounding factors, such as poor image quality. They are more likely to err on the side of caution and decline to make a match, rather than present a conclusion of which they are not certain. As previous validation studies (Hogge et al. 1994; Koot et al. 2005) concerning radiographs of other regions of the body have demonstrated, and as was expected in the present study, it does appear that individuals with more training or experience with examining radiographs were more successful overall than those with little-to-no experience. Those with more experience also cited more specific points of similarity when making matches, likely because they have more knowledge of which features are most individualizing and because they may generally have more knowledge of anatomical features and terminology than those with less experience. This study shows that significant relationships exist between the participants’ performance and their level of education, years of experience, and number of radiograph cases, which is consistent with the findings of previous researchers. Among the most experienced participants (11 = 8)—those who have earned their PhDs, have been practicing for more than 20 years and 56 have had experience with more than 30 radiograph cases—six answered all of the matching questions correctly, one scored 35 (both non-identifications), and one scored 3/5 (1 non-identification, 1 misidentification). To revisit a point made in Chapter 5, the participant who made one misidentification is the only participant who provided an incorrect answer but qualified it, saying that he/she would not consider it an identification without further study. It is that qualifier, left out by less experienced observers who answered at least one question incorrectly, which distinguishes this experienced participant from those with less experience. It was also noted in Chapter 5 that when the entire sample of study participants is taken into account, the only significant relationship between any of the independent and dependent variables exists between Number of Radiograph Cases and Score out of 5, indicating that the participants with more experience comparing radiographs outperformed those with less radiograph comparison experience. The anatomical features most commonly relied upon to correctly match the antemortem and postmortem images were the spinous processes, transverse processes, and centrum shape/contour. In combination, these features are highly reliable. Less reliable are the patterns of radiolucencies and radiodensities (the appearance of which is highly susceptible to the orientation of the vertebrae and of the x-ray beam), and the osteophytes/syndesmophyte development. The five misidentifications that were made in this study all cited such age-related changes among the primary criteria used to make the identifications. This study has shown that it is important to be mindful of gross structures as well as smaller details when comparing radiographs of the spine, and to take into consideration as many features as possible when deciding on whether the decedent can be 57 positively identified or excluded. Unfortunately, this study also reaffirms the NAS report’s condemnation of the forensic sciences for their lack of standardization. It is important for forensic anthropologists to continue to work toward a standardization in terminology and methods. Results from this study show that terms such as “osteophytes,” “syndesmophytes,” and “arthritis” were used by different participants to explain the same structures, which is problematic because the terms themselves are not interchangeable. If forensic anthropology is to continue improving standardization in methods, techniques, and record keeping, maintaining consistent terminology will be critical. Despite the problem with terminology, the majority of the participants—students and professionals alike—consistently relied on a handful of vertebral features to arrive at correct matches between radiographs of the same individual. This is an indication that although there is a lack of consistency in terminology, there is a generally accepted practice for comparing radiographs among forensic anthropologists. The disparities between the number of specific anatomical features cited by the participants is something to be considered, however. The professional anthropologists who participated in this study cited more points than students, and it appears as though the difference (See Appendix G) is mainly because professionals recorded more detailed summaries of their observations on their data sheets. This does not mean that students did not examine the same features; they may have simply not known how to write as precisely, using specific terminology, as their professional counterparts. If standardization is to be achieved for the practice of comparing medical and dental radiographs for positive identification, it will be necessary to outline best practices for recording such information. 58 This study has shown that the lumbar spine is a reliable region of the body from which positive identifications can be made using comparisons of antemortem and postmortem radiographs if the comparison is carried out by a well-trained, experienced observer. It has also identified some problems with the standardization of terminology and methods used by forensic anthropologists when making such identifications. For that reason, it may serve as a basis from which best practices and standards can be developed for the field of forensic anthropology, specifically with regard to comparative radiography. 59 APPENDICES 60 APPENDIX A Cadaver Information 61 Cadaver Information The chart on the following page is a list of the cadavers that were used for the radiographs in this study. The cadaver ID numbers assigned by the MSU Willed Body Program, their causes of death, and their date of death have not been included in order to protect the identities of the individuals who donated their bodies to the Willed Body Program. The fifth column in the chart describes the type of dissection for each body in the sample. This column is relevant because the depth of dissection and the amount of soft tissue and organs that are intact or removed have an impact on radiograph quality. Bodies that have been dissected to the Intermediate or Deep levels are missing many, if not all, of their organs and are more likely to produce clear radiographic images of the bony features of the spine. Those that have been dissected only superficially (in many cases, meaning that only the skin and subcutaneous fat have been removed) or to the viscera still contain organs and are the bodies that tend to yield obscured or cloudy simulated antemortem radiographic images. The abbreviations used in the chart are as follows: LAM Limbs Anterior Muscles TAM Thorax Anterior Muscles LAV Limbs Anterior Viscera TAD Thorax Anterior Deep LPM Limbs Posterior Muscles TPI Thorax Posterior LPV Limbs Posterior Viscera ’ Intermediate TAI Thorax Anterior TPS Thorax Posterior Intermediate Superficial 62 oz 2 233$ oz 8% 3Q. coon .2 mm. mm 02 2 233$ oz 8% Q<.S<._. econ "— 3. 2 oz .2 233$ oz 8% $22. meow m mm 2 02 oz oZ 8% $21— 38 2 ca m 2 oz 3 233$ 02 8% 22. coon .2 me 3 oz o 233$ oz 8% 0.42. voom "— K 2 Q 233223 cm 233$ 8% 8% 3Q. gem m cm 2 oz 2 233$ 02 8% 92. econ 2 mm 2 oz oz 02 8% 22. 38 2 mm 2 oz m. 233$ 8% 8% mm... meow ”— mm o mEBooeoEE >Eo2ooeofifl I288 202 lea: 82 oz 8% >n3 econ E on w oz oz oz 83. 87.2 2.: 38 2 mm 2 29320an mars 2930er tome—2:31.88 82 9:26 vowafieeleom: SZ 8% 8% ><4 meow "— an 0 oz 2 233$ 02 8% 324 Sam 2 E m oz 2 as N 333$ oz 8% 2.2.2 meow 2 mm v m 23992: m 233$ 8% 8% >43 econ 2 on m m 23992: S 233$ 8% 8% > m a ou< 32a: 28 coma—2.x 92259on 29292235 29639 ..2... 232$ 63 APPENDIX B Simulated Antemortem Radiograph Images . 64 Figure 10: Simulated Antemortem Image #1 1’3" 1 . i? 3“": A L o ' kVp 75 @ mAs 48 Not a match for any of the simulated postmortem images 65 Figure I I .' Simulated Antemortem Image #2 kVp 75 @ mAs 40 Not a match for any of the simulated postmortem images *This is an x-ray from the same cadaver as Simulated Antemortem Image #19 66 Figure 12: Simulated Antemortem Image #3 r' 0 .l at , '21s. '* u" ... a... kVp 75 @ mAs 20 “Antemortem Match” for Unknown E 67 Figure 13: Simulated Antemortem Image #4 kVp 75 @ mAs 64 Not a match for any of the simulated postmortem images 68 Figure 14 .' Simulated Antemortem Image #5 ' ‘ ~P kVp 70 @ mAs 20 Not a match for any of the simulated postmortem images 69 Figure 15: Simulated Antemortem Image #6 P kVp 70 @ mAs 20 Not a match for any of the simulated postmortem images 70 Figure 16: Simulated Antemortem Image #7 p: ~va .3 I” r kVp 75 @ mAs 32 Not a match for any of the simulated postmortem images 71 Figure I 7: Simulated Antemortem Image #8 3*‘3" ,‘F “kw? 1“" r 33“ at» 1.“ ‘~;‘ has» A“ 3V3“~ . - «we ...“ ”ixiikg. .fifi“ as? 2...”: ' Q». 3‘ i F 3‘ bQ ..fib‘fl‘E“ 1¥§~ gummy». V3‘33. Q ‘5 T seQier ~IQQee 'sesrw kVp 75 @ mAs 80 Not a match for any of the simulated postmortem images 72 Figure 18: Simulated Antemortem Image #9 kVp 75 @ mAs 32 “Antemortem Match” for Unknown C 73 Figure 19: Simulated Antemortem Image #10 a .5 l ‘ .w’a“ .4315; A“... kVp 75 @ mAs 20 “Antemortem Match” for Unknown B 74 Figure 20: Simulated Antemortem Image #11 y ' )' . i 3 . ' fi 8 0’ . ’. . . e ._ .' t kVp 7O @ mAs 64 Not a match for any of the simulated postmortem images 75 Figure 21: Simulated Antemortem Image #12 1'7 . .8' kVp 75 @ mAs 24 Not a match for any of the simulated postmortem images 76 Figure 22.“ Simulated Antemortem Image #13 kVp 70 @ mAs 24 Not a match for any of the simulated postmortem images 77 Figure 23 .' Simulated Antemortem Image #14 kVp 75 @ mAs 24 “Antemortem Match” for Unknown A 78 Figure 24.“ Simulated Antemortem Image #15 ~———— kVp 90 @ mAs 64 Not a match for any of the simulated postmortem images 79 Figure 25 .' Simulated Antemortem Image #16 ' ,3 kVp 75 @ mAs 20 Not a match for any of the simulated postmortem images 80 Figure 26: Simulated Antemortem Image #17 QF' J i. kVp 75 @ mAs 64 Not a match for any of the simulated postmortem images 81 Figure 27: Simulated Antemortem Image #18 . .' , W kVp 70 @ mAs 24 Not a match for any of the simulated postmortem images 82 Figure 28: Simulated Antemortem Image #19 kVp 75 @ mAs 48 Not a match for any of the simulated postmortem images *This is an x-ray from the same cadaver as Simulated Antemortem Image #2 83 Figure 29: Simulated Antemortem Image #20 kVp 75 @ mAs 4O “Antemortem Match” for Unknown D 84 APPENDIX C Simulated Postmortem Radiograph Images 85 3 SEEM .8.“ .3834 Eotofiumom: v 3:. © mm mi E 239$ c8 2sz 838.589. m 25 © 8 mi N< 33 Ex “SEVEN Emtogmom nodfiafiww 5m. 333k 86 S 232$ 28 :53: 88.55389. m Ea © mm 93 E 232$ 28 2822 EctoEHmom: m 38 © 8 93 mm 2222 NM 8322: 23322563 3232223 .. R. 2.22me 87 a 23o2$ 28 .2822 838.589. 4 2:. © mm a: a 233$ St. 2822 58.52589. m 35 © 8 mi ND V22 ND 23223 Ewtogaom 22.852223. ..Nm. 22233 88 QN 23o2$ .83 :28va Eaton—«mom: v w<8 © mm 35 cm 23o2$ 28 .3822 8855309. m 26 © 8 mi NQ E22 ~Q «.2323 2222222qu 22832223. ..MM. 22233 89 m 233$ 28 2632 88.82589. m 25 © mm 3: m 233$ 28 23.22 88.82589. m 3.5 © 8 mi NW V22 NM mmm§£ Eutogmom ESSEG ..Vm. 32me 9O APPENDIX D Sample Data Sheets 91 Positive Identification through the Comparison of Abdominal X-rays, focusing on the Bony Features of the Lumbar Spine: A Validation Study DATA SHEET PART 1 PERSONAL INFORMATION Please circle the answer that best describes you, or fill in the blanks where it is appropriate. 1. What is the highest degree you hold? a. BA/BS b. MA/MS c. Ph.D. 2. Are you still a graduate student? a. Yes b. No a. If yes, what is your expected degree? a. MA/MS b. PhD. b. How many years have you been in graduate school? c. Are you ABD? a. Yes b. No d. Which of the following best describes the type of program in which you are enrolled? a. Forensic Anthropology b. Biological Anthropology c. Human Biology (1. Physical Anthropology e. Bioarchaeology 1'. Other: 3. How many years have you been practicing Forensic Anthropology? (Please answer this question only if you have finished your graduate education and are working in the field) a. 1-5 b. 6-10 c. ”-15 (I. 16-20 e. greater than 20 years 4. Approximately how many cases have you worked on that involved comparing antemortem and postmortem radiographs? a. 0 b. 1-5 c. 6-10 d. 10-20 e. 20-30 f. greater than 30 cases 5. Were any of those comparisons made between radiographs of the spine? 11. Yes b. No If yes, approximately how many? 6. Do you work primarily: :1. Within Academia b. Outside of Academia 7. Which characteristics do you generally tend to focus on when comparing radiographs of the human body? 92 Positive Identification through the Comparison of Abdominal X-rays, focusing on the Bony Features of the Lumbar Spine: A Validation Study DATA SHEET PART 2 IDENTIFICATIONS The postmortem radiograph marked “A” corresponds to antemortem radiograph #: How many points of similarity did you identify? Which specific anatomical features were most important to making this match? The postmortem radiograph marked “B” corresponds to antemortem radiograph #: How many points of similarity did you identify? Which specific anatomical features were most important to making this match? The postmortem radiograph marked “C” corresponds to antemortem radiograph #: How many points of similarity did you identify? Which specific anatomical features were most important to making this match? The postmortem radiograph marked “D” corresponds to antemortem radiograph #: How many points of similarity did you identify? Which specific anatomical features were most important to making this match? The postmortem radiograph marked “E” corresponds to antemortem radiograph #: How many points of similarity did you identify? Which specific anatomical features were most important to making this match? Please feel free to list any additional comments in the space provided below or on the reverse side of this form. 93 APPENDIX E Results from Contingency Tables 94 Table 5: Results from Contingency Tables Independent Variable Dependent Variable Cg:enfggfen:ty ISignificance NEEEEIPIESLZZB Highest degree Score out of 5 .550 .369 etain Null Hypothesis (Total Sample, n=25) Score out of 4 .233 .838 etain Null Hypothesis Error/Non-error (5) .056 .962 etain Null Hypothesis Error/Non-error (4) .149 .753 etain Null Hypothesis Number of features .519 .231 etain Null Hypothesis Student/Professional Score out of 5 .496 .148 etain Null Hypothesis (Total Sample, n=25) Score out of 4 .155 .736 etain Null Hypothesis Error/Non-error (5) .100 .617 etain Null Hypothesis Error/Non-error (4) .021 .918 etain Null Hypothesis Number of features .574 .013 eject Null Hypothesis PhD/Not PhD Score out of 5 .435 .323 etain Null Hypothesis (Total Sample, n=25) Score out of 4 .226 .51 l etain Null Hypothesis Error/Non-error (5) .017 .930 etain Null Hypothesis Error/Non-error (4) .137 .490 etain Null Hypothesis Number of features .400 .241 etain Null Hypothesis Number of radiograph Score out of 5 .658 .039 eject Null Hypothesis bases Score out of 4 .292 .676 etain Null Hypothesis (Total Sample, n=25) Error/Non-error (5) .270 .373 etain Null Hypothesis Error/Non-error (4) .246 .448 etain Null Hypothesis Number of features .492 .318 etain Null Hypothesis Any radiograph cases Score out of 5 .455 .218 etain Null Hypothesis involving the spine Score out of 4 .265 .437 etain Null Hypothesis (Total Sample, n=25) Error/Non-error (5) .154 .464 etain Null Hypothesis Error/Non-error (4) .252 .221 etain Null Hypothesis Number of features .334 .497 etain Null Hypothesis Number of radiograph Score out of 5 .575 .331 etain Null Hypothesis cases involving the spine Score out of 4 .484 .135 etain Null Hypothesis (Total Sample, n=25) Error/Non-error (5) .244 .483 etain Null Hypothesis Error/Non-error (4) .303 .313 etain Null Hypothesis Number of features .444 .556 etain Null Hypothesis Highest degree Score out of 5 .302 .638 etain Null Hypothesis (Students, n=9) Score out of 4 .302 .343 etain Null Hypothesis Error/Non-error (5) .302 .343 etain Null Hypothesis Error/Non-error (4) .302 .343 etain Null Hypothesis Number of features .302 .343 etain Null Hypothesis Years in graduate school Score out of 5 .302 .638 etain Null Hypothesis (Students, n=9) Score out of 4 .302 .343 etain Null Hypothesis Error/Non-error (5) .302 .343 etain Null Hypothesis Error/Non-error (4) .302 .343 etain Null Hypothesis Number of features .302 .343 etain Null Hypothesis 95 Table 5: Results from Contingency Tables (Continued) Independent Variable Dependent Variable (38:31:35]? Significance] N135: Efiifigs xpected degree Score out of 5 .302 .638 lRetain Null Hypothesis (Students, n=9) Score out of 4 .302 .343 Retain Null Hypothesis Error/Non-error (5) .302 .343 Retain Null Hypothesis Error/Non-error (4) .302 .343 IRetain Null Hypothesis Number of features .302 .343 Retain Null Hypothesis Type of program Score out of 5 .447 .895 etain Null Hypothesis (Students, n=9) Score out of 4 .243 .905 etain Null Hypothesis Error/Non-error (5) .243 .905 etain Null Hypothesis Error/Non-error (4) .243 .905 etain Null Hypothesis Number of features .500 .392 etain Null Hypothesis VIBD/Not ABD Score out of 5 .213 .827 etain Null Hypothesis (Students, n=9) Score out of 4 .141 .686 etain Null Hypothesis Error/Non-error (5) .141 .686 etain Null Hypothesis Error/Non-error (4) .141 .686 etain Null Hypothesis Number of features .281 .408 etain Null Hypothesis Number of radiograph Score out of 5 .555 .406 etain Null Hypothesis cases Score out of 4 .447 .325 etain Null Hypothesis (Students, n=9) Error/Non-error (5) .447 .325 etain Null Hypothesis Error/Non-error (4) .447 .325 etain Null Hypothesis Number of features .500 .223 etain Null Hypothesis Any radiograph cases Score out of 5 .386 .455 fietain Null Hypothesis involving the spine Score out of 4 .368 .236 etain Null Hypothesis (Students, n=9) Error/Non-error (5) .368 .236 Retain Null Hypothesis Error/Non-error (4) .368 .236 [Retain Null Hypothesis Number of features .302 .439 Retain Null Hypothesis Number of radiograph Score out of 5 .386 .455 Retain Null Hypothesis cases involving the spine Score out of 4 .368 .236 Retain Null Hypothesis (Students, n=9) Error/Non-error (5) .368 .236 Retain Null Hypothesis Error/Non-error (4) .368 .236 Retain Null Hypothesis Number of features .156 .635 Retain Null Hypothesis Highest degree Score out of 5 .511 .130 etain Null Hypothesis (Professionals, n=l6) Score out of 4 .302 .447 etain Null Hypothesis Error/Non-error (5) .177 .473 etain Null Hypothesis Error/Non-error (4) .290 .226 etain Null Hypothesis Number of features .452 .342 etain Null Hypothesis Years practicing Score out of 5 .608 .404 Retain Null Hypothesis professionally Score out of 4 .453 .658 Retain Null Hypothesis (Professionals, n=l6) Error/Non-error (5) .368 .474 Retain Null Hypothesis Error/Non-error (4) .414 .347 Retain Null Hypothesis Number of features .592 .634 Retain Null Hypothesis 96 Table 5: Results from Contingency Tables (Continued) Independent Variable Dependent Variable Cgsffggfenncty Significance! N11111: Hgibeecstis Work primarily within Score out of 5 .463 .225 Retain Null Hypothesis or outside of academia Score out of 4 .245 .599 Retain Null Hypothesis (Professionals, n=l6) Error/Non-error (5) .092 .712 Retain Null Hypothesis Error/Non-error (4) .213 .383 Retain Null Hypothesis Number of features .407 .462 Retain Null Hypothesis Number of radiograph Score out of 5 .586 .039 tReject Null Hypothesis cases Score out of 4 .216 .676 Retain Null Hypothesis (Professionals, n=l6) Error/Non-error (5) .345 .142 Retain Null Hypothesis Error/Non-error (4) .151 .541 Retain Null Hypothesis Number of features .492 .318 Retain Null Hypothesis Any radiograph cases Score out of 5 .511 .130 Retain Null Hypothesis involving the spine Score out of 4 .302 .447 Retain Null Hypothesis (Professionals, n=l6) Error/Non-error (5) .177 .473 Retain Null Hypothesis Error/Non-error (4) .290 .226 Retain Null Hypothesis Number of features .334 .497 Retain Null Hypothesis Number of radiograph Score out of 5 .602 .241 Retain Null Hypothesis cases involving the spine Score out of 4 .469 .414 Retain Null Hypothesis (Professionals, n=l6) Error/Non-error (5) .316 .459 [Retain Null Hypothesis Error/Non-error (4) .316 .459 Retain Null Hypothesis Number of features .444 .556 etain Null Hypothesis 97 APPENDIX F Results from Nonparametric Statistical Tests 98 2852:: :2 5302 now. 223-2222 ”ease 20 .2252 23522.23 :22 22292 92. D 202223-2222 Avv 2020-202292m 2mm": .292—Wm .285 28502»: :2 Z 22232 own. D 232223-2522 20 2820-2oZtotm 05% 23502»: :22 2223. 3 _. 2:23-23:28! 2 no So 28m 05 M25022 828 28502.22 :2 Z 2220M 3N. 2:23-36:22 n .20 25 8on 2223228 .8 202252 23502.2: :22 2223— 32. 2:23-22me2 $2228 20 232:. Z 23502.2: :2 Z 250% m2 _. D 52223-2232 33 echo-20220:”; 23502»: :2 Z 2223. now. D 32223-2222 29 Btu-202202.22 2mm": .oEEmm 280.5 23502.2: :22 2220M SN. 2:23-53:3— 2 go So 88m 22% 22 w2_>_o>E 23502.2: :2 Z 22202 ovm. mama-.332! m 20 So 88m 8.28 222232222 22¢. 28502.22 :22 2220M ONN. 2:23-22me 8.22202 .8 23252 2.3502»! :22 2220M Sm. D 202223-2222 32 2020-2022022”; 285222 2.2 2302 $2 2 20523-:swz 6 85-822522 awn: 6.28% 28: 28502.2: :2 Z 2223— N3. 2:23-333! 2 mo 25 8on 898 $3502.22 :22 22292 m2. 2:23-33:52 m 20 25 88m 2923228 .8 23822 $8522.»: :22 guano: :8. 2523.23.22.0— mBBwou no 23252 28502.2: :2 Z 22202 omo. D 52223-2222 92 202o-2oZtotm 23502»: :22 2230.2 vmo. D 202223-2222 AC 2020-2022025 23502.22 :22 22232 022. 2:23.3me v 20 :5 29% Gm": .oEEam 220.5 23502.23 :22 22292 of. 2:23-53:22 m 20 So 88m Ecofimflen—bsvam 23502.2: :22 2220M mm _. £22322me 32820.2 .20 232222 285222 ==z 552 m3. 2 3052325: € 52982322 285222 .32 2302 :2 2 20523252 5 829822522 285922 :2 2302 Man 223-2222 4 .6 .8 28m 3?: .2282 2.82 285222 .32 .2502 as. 2.23-3232 m do =5 28m 32% .8222 awflwofi 232 ....“z 883m .82. 8:85:25 «zeta.» 2.252.»: 932...; 2322.222 2225 222.222.2222 2222222222222 22.3.3282 .6 2325 99 2328232 :22 22302 omv. D 32223-2222 3283-2 mo 32252 23222;: :22 22802 won. D 32223-2222 2% 2920-2022052 23282»: :22 22332 now. D 32223-2222 AC ..oto-2oz\2o2m 232892: :2 Z 2232 now. D 32223-2222 2 2o :5 88m 8": .223—28 23282»: :22 22802 3w. 2223-23—35— m we :5 Boom Qm< 825m? 232202»: :22 22232 awn. D 32223-2222 3223-2 20 3222 Z 2852:: 2.2 5302 8v. 2 282.3-..52 § 85-822822 23282.2: :2 Z 2232 cow. D 32223-2222 AC 2020-2oZtotm 23282»: :22 22802 cav- D 32223-2222 2 20 So 2on 8H2 220229 23252»: :22 2233— cwm. 2223-32322 m .20 So Room 222on .20 39C. 2852;: ==z 5502 :m. 2233-2823. 3282 2c H22.52 232202»: :2 Z 2232 2 mm. D 32223-2222 A3 Stu-202282.“ 2852:: .32 532 Km. 2 3523-532 6 55.82285 2352:: :2 5302 5. 2 20523-2“: 2- 20 =5 28m an: .3532 23282.2: :22 2232 one. 2223-23.25— m we :5 Boom 3232 28392— 232202»: :2 Z 22.31 Km. D 32223-2222 3233-2 .20 32252 2852:: 2:2 5302 E. 2 2n-,___E§-._§2 22 285-52282 232622: :2 Z 2232 Km. D 32223-2222 23 2020-2072025 23282-2: :22 2232 _ mm. D 32223-2222 2 20 So Boom 8H2 220239 .0028 23282»: :2 Z 22.802 one. 2223-23—32! n we 25 2on 82.23% 22 23> 2.328%: :2 Z 2232 _ um. D 32223-2222 3.2832 mo 32.222 23282»: :22 22502 _ hm. D 32223-2222 32 2oto-2oz\202m 23252»: :22 22802 _ mm. D 32223-2222 AC 2020-2oz\2otm 232202»: :22 2232 _ mm. D 32223-2222 v 20 So 28m an: 222229 232202»: :22 22232 owe. 2:23-35:22 m 20 :5 Boom 3232 2323: «WMMWOEW “HZ 333m «mob 992352w2m 0.2.2.2; «no—EMEQQ 0.2.2.23» «23.293222 «232.2200» 23& 39.2232 223233582 59% 233% .6 3335 100 5852:: 52 5308 ac. £33-3wa 80380 “6 .352 2359:: :22 5501 omo. D 3523-582 SD 550-52285 $35032 =32 550m 3N. D 3523-582 AC hobo-50228.5 A2": .mficommmfloiv £85032 .22 550M Non. 283-80352 v .5 :5 050m 385230252 «6052:: :22 550% 2 so. 283.8me m Mo 50 050m 950285 m80> $358322 :32 580% gm. 2:83-28me 3580.2 20 825:2 3052;: ==2 5.50M 3N. D 3523-582 QV .550-502285 $8552»: .22 580m 23. D 3523-582 AB hobo-coghobm 3052;: :52 530m one. £83-8me v 20 :5 080m A2": 88:230-55V $8585 :52 .8?— cm... 553.355 n he :5 28m 020% 3.25: 3059:: :22 5801 mmo. D 3523-582 8582 .5 002522 2852:: ==2 550% vow. D 352 3-582 Q; 5050-:o22otm an: £20259 £85032 :22 580% 3N D 3523-582 AC Hobo-zoguotm 02% 2355;: ==2 550m 3N. D 3523-582 v 20 So 080m 05 w5>_o>5 $80 £00522»: :52 550% 3v. 283.8me m 20 So 080m 583028.. 20 “025:2 2855:: :32 550% m3. D 3523-582 8.5802 20 855.2 2352;: :22 580m #8. D 3523-582 A3 .obo-soZtotm 2352:: :52 580m 3N. D 3523-582 AC 550-:oZtobm an: £20559 £35093; ==2 530m 3N D 3523-582 v mo :5 0.50m 05% 05 w5>_o>5 2855;: :52 580% 3?. 2:83-8me n mo 30 080m 380 588228. 3< 3050a»: ==2 580m Ev. D 3523-582 8.588 Mo 8552 2352;: :22 530% Km. D 3523-582 3; 550-:022otm £85032 =52 580m RN. D 3523-582 6 850-52285 8": .acovamv £8509?— ==2 580m Em. D 352 3-582 v he :5 050m ”080 20055;: .22 5802 mmv. 283.8me m mo :5 0.50m 5882282 no .8252 MMMMMESE ““2 8:88:05 .89 8:855m 0355» .5235 9.55» .8250... «20:25:09» San-K 3.555% 030832202 28% £5.02 .6 036K 101 3852:: :22 ES»: v.3. £33-?me 8:58: :0 H3:52 ammo—tom»: :22 5.80: #mm. D 3523-582 Db hotoéoguohm GWG 6.223328 $350931: :32 580: :N D 3523-582 Anv BhoéoZtotm 05% £850qu := 2 £30: mmm. m_:a 3-13.30— v no :5 88m 2: w:_>_o>5 339 $8509?— ::2 530.: we: £33-?me m we :5 28m naflwofifi no :58: 2 £858»: :22 580: mum. £33-?me 3538 mo .3852 ”552:: :2 533. SN. : 3533-:52 E 85.5235 £350qu :3 2 ES»: nwv. D xofimgéfiwi AC hotoéoguobm 3:": $3568.65: $3533: ::2 5.80: :5. 333-13383 v :0 So 28m 05% 2: wag—oi: £353.51 :=2 :50: :2. £33-?me m mo :5 Boom momma nufimoES be £358»: ==z £30m wwo. £33-3max “228: Mo .3552 322$ =2 :53. Rm. 3 $523-:52 3 .ouzozhoam $358»: := 2 580: mm: D 3523-582 :3 ShoéOZbObm 3:": $355888 £358»: ==2 £30m mac. £33-3me v :0 So 28m 398 £852.»: :52 «8?: a3. m:—«3-.dxm=..v_ m :0 So 88m £33292 :0 ~38: 2 $8533: :=2 5301 wow. $334853! 8.33: :0 “38:2 £353»: :22 £80: mom. D 5523-582 2; 5toéoZtobm £358»: :::2 580: can. D >0523é52 AD 5:952:35 3:“: .mfiaoflmomoiv mac—tom»: :32 530: M29. £33-3me v :0 So 28m £583“ :0 023.5 5 £358»: :32 580: SN m:_a>2-_8_mEv_ m :0 So 28m 555 btafitm x53 fluuuéoflwcnmwfl ”“2 conduciflm «mob ouggnflm «Ear—5» 23:—895 «33:5 «51:32.5 $33356» mask 3.95:3: uEuEEquZ 28:53:32 .6 $335 102 APPENDIX C Participant Answers to Data Sheet Questions 103 AS bifioamov 8:5: .mommooofi ac amofifiaoaa m: we maoafim .oEm Em: ao 3 3:6 :2 05 oEo>ma3b ”AN: 3:3m8amov Em: a: Baa—Em 23 3 ac 8369a :o 0%.? 8:56 .mommooofi m:oafim Ame—030a E3 25: 33023.: ”n: a: momooofi 3 .35 m: mo .8333 me A: 338%8 "m: :0 mafia: 8353.: 388a 9805.3}: oamfi ”output? mommooofi omtmfiooamo Am: Amm— onoamov mo oasfi m 05 £85» maoafim mo :3 3 bfimooamov mummooofi m:oafim Av: 338%0 .80 £833.: oamnm $83383 Am: gflooamov mommooofi 305% ”mu—A‘— ao mafia: mommooofi .moaaosbm 0:38: 853:: bo> 8:033 ao ammo—0:908 canaooamo 305% me buaoE >3 :3 8:833 m3: Nqivqma ”mommooofi maoafim A: :3 m: Aim: asap? A: 93 m: 8 2:056 32:89.3: ac 338%3 mafia: ”N4 ao been bimooamov 3:5: 3:38am»: 8:6: 33:00am": 8:6: :88 D 383.: hmofifiaoaa aaom canaooumo ac hmofifiaoaz ac amgonafiE 338:: MO oaanm 3:83 ac can—E . m: mmooofi 305% 3 b5: ac oamam ”3 mo oamfi ”Em3aa 30: mo oaanm .3 m: mafia: 3 b5: :0 omanm .8:an .mmooofi mmooofi 305% me mm: b5: ao ”my: b5: no oamnm m:oafim was 3 m: .30: m oamfi .mq mmooofi nag—m ”N: 888a ”m1. :3 3 :0 been as 092% an: ac oamnm ”N:— .v: m:oafim ac oamfi 305% :4 bmomaofifl Sven mmooofi 355% .3 ac 8388a 3333 oZ mo_ma3 .0925 ”N: ao mafia: mmooofi m:oafim ao mafia: My: ao 2ma3\oaa:m 305% .«o oaasm 8:50 we 3:80 62:83: «have .8388a .38.: 330:3 853:0 8:30 8333: .8388a maoafim .mommooofi m:oafim mommooofi maoafim $03.: shave .mommooofi _ aflofioa .N: :0 mafia: 6: ac mommooofi .waaoo mo 8:50 330:3 gave 03033.: 3263 o Z .mommooofi m:oafim omifiooamo 955% 03:5 .mowmooofi w:oafim .mommooofi maoafim .mommooofi maoafim 558:0 .25: 5 338:0 mm: X: 2: Na Z: «gonnam 28.5.sz 33h SQQ S .23:an aaafifiataa‘ K mafia 104 ...03: 090 :0: 2:00 _ 35 moaaofi 30:205.. mmoocfi 0333.3: Em: a0 05% 30:03: 53:05 aanm -> ”30:03:. 35 EB3a 3 matca 350285 e =8 283 as: 256 023 :0: ca 03% 33:0: a0 30a: :33 33 0a:30 mmoccfi 03.853: W: .0: ”00:00.83a 03:05 :0: 3a: 3: .0 ”3:30 .30: Ba: 33033 :0: 3 33: N: .a. 03% >50: mfiwafixo 303 xEm: 5:3 :03 3 v: .m Sac: ac 23 3 813 .235 S 3:83 :2 32% 22% 50:0: 3:: m: ammo—0:303 3 H ”mmoocfi .0533 05 m: 3 303 03:05 0m3_ 35 :5 30:00— mmcocfi 05053:: 0_ma3 05 a: a: 0:: 3 A023 oammcaacv _|3_a m: 50: 03353.: 53 :0: 3::an 35 3583a 0035 Em: a0 5050: ac :33 @0745 05:30 2:35 50: 30:03: a0 0305lbmma05 3 :00 03:0: 5:: cEm: .c ”303 03:05 53:0: 3 .0 am: 3 a0 :03. 33 03.7. :30: 0300: :33 :0: mommoocfi .300a :33 3500 0:500 03% 50: bmmaoa a0 m32< Em: a0 a3:0>0 03333: ac 30a: Em: aoaa: m: .m ”3:335 333— .3530: a: 38:53 ac 3033 303a 03:05 3:33:05 .m ”macbm 3-3 ama3ah GEN: Aaowcaxoacaaav .m ”a0533maaa00 :2 ac 33:05 33053 3300 .0330a :8 3 0:3 3: :03: :0: :33 ..a:: N: .m ”3335 n: a: .ma:03% 033535 3305 ac 30a3Em 055:0 :03 33:33; :0: 03: 30330030 3383: a0 0%.? 5:3 mamm35 05 333 33a. .3 50: Av bac Slaaotcaacaaaa 50—05050: :2 to: m: .N ”03:3 333533 50: .mommoocaa 95 39,30 :3 03 0305 00:3 3: 5203 00 :03: ac 3-3 .N ”a: 30000fi 033053.: a 03: :0: 2:03 03:05:35 b32303: mmaE .3 m: a0 33000fi 03>» m 4 a0 03 30E:0|ac:0a:m 3 .: .acm:3a500 _ .33: cm 3: Q E: lat? :— aaowofi 35 AEmE .3 :0 03:03:35 Em: 330: 33 a: a0 023 v: :0: :0 £5 a: :03 a: :0 033098 Ea. a: 30:30:03 >a< 38330030 0m34 :0 5:30:55 :0: a0 003:3 3E 30330030 05 ._ :00: 83> m :< 00:32.0 .353. h 3:330 2: X: 2: NA: 2: 0003.5 «33:53:09» machamau 302E. SQQ 0» .2033: aaukauatam K 0335 105 3 amass 3 mmcacEEcE 8338: 3 .NA m.— mmoocaa v4 am:ca5 NA 335% and £4 .3 ammo—ESE: Eafimcaaa 3?. m4 Em:c.a5 3 3:833 H4 3338: 3:25am .3 3223.58 .3383 3333.3: 3 ac maatcm 325 :33: 33338: 335% :3 v: 3338: um..— 64 .NA 3385 33333.: ”m1— hmcacEEcE ca c: QcacEEcE 3:0: 333335. ”N: ac mama—E..— 3m3>ma3.aa. 33333: 33333:... ”3 :3 .NA .3338: mmocca: moaca 335.83 ”3 Em:ca5 ”m4 :3 ”3 5:3 aam3E atom ”v4 macaEm ”m4 :3 gcaaam 5333 ac maam3a>a Na aam3aa bucm ma Em3E 53cm 3 maam3E 33m :3 m:— ac maaaafl N.— aam3E atom 3 53 3 33333389 macaamm ac 3353c: ac:3a5 x3335: ac:3aa:. :3 3:233 53 3:33 mac—3 333853.. mac—3 b33333 C358 xEm: m.— .Em: 33:33:33 3 3:35 Em: 3 .Em: :3 ac 3333 a3 m: 3333: b33338 3 a3_ m6 3333338: ”3353: 333— 3 33353336 33323336533 ac 33:3: 333— 33353: cm?~ ac 333:3 ”3333: 3:85 m: :3 3:33 .Abamaoacaafl 5am: :3 5:3 3:333 53 m4 ac 33:3: mac—3 33:38:33 ac 3:: 5:5 @3— 3 Em: m.— 3ma3ac53: 3:85 :3 ac 333 ”3333338: v4 ac 3393338 £2 3 b33333 332 $22335 ”3333338: 8:393 ”3338: 33333.: a“: 3:333 3333338: accm a3 8335. 33335 m:caEm 33333.: Em: 3 53333.: 3.34 333395 33333:. 3333338: 3353 353:2, 3:25am ac ac .5333 ”2.3.3 33:53 ”333338.:— .3_£aa3aa 3353 35c: 353:3 333a ”3333338: 33333: da :3 ONE 53 8:533 aa 333a ”3333383 ac Scaacc ”m1. ac 33333.: ac 3353 m 53:33 33: 2:95 33:33a 2:953: 385% ac 33% 3333338: 333353 ac:3aa_ ac 33:3 ”3353 333:9» 3>3a 5233355 Iaccua3a ”E38833: 33333: :3 3:23.... 3353—33: ac Scuacc 333 :Q: soda: afiacaa 3:383:30 Sac: ac ScEcU ac 33355330 3353330 ”3:353 333a 238.5 .23... h SE26 3. X: 2: 8. 5. 38.3.3 «3332.580» Qcaammam 33am. SQQ ca .233sz EQABEQE K 33:.“ 106 m0 000 3 .m0 .00 00 .00 0000000 3 .00 80000 000 000000000 0005% 8000080 000500 02000000 000 0000000 00002000 00 000E208 25 8000 08000000 .NA .00 00000000000 2 32000000000000 0200000 ”3 000 000000 3000000000 3 000 00000000000 ”00 000 .3 .00 8080000 003000 0 Z 0000 000000w 000m N0 .00 8000000000 00 20000 003 0 3 .00 8000000000 3 .00 8000000000 80080000. 0000000 000 00 00000 .00 0000000200 £53000 000 00000000 >000 £80000 3 000000000 w500: 000000000 m500= £22000 £5w0000 .00 8000000000 .00 022000 £5w0000 .00 05w0000 .00 £88080 000500 0000000 £22000 0000000 £53000 £88080 0000000 £22000 0000000 £22000 003000 0 2 ”200000000 00 0 £88080 00050m £22000 aw500§ 000500 £5902 £88080 00050m £88080 00050m 00 0000 000000000 _00000 00 05000 8000000000 .0000 _0000000> 3 w500: 000000000 0.00500 £22000 .w500: .00 05000 2000000000 .N0 .00—0000000 00 2000000000 M13500 022000 0 £80380 000500 22000 £2000 £22000 .w500: £2000 .NAANH 0000 022000 £22000 $500: 00000000> .00 00000 S .00—0000000 000 _00000.00> .00 00000 _0000000> 00 00000 003000 02 £88080 00050m 2000000000 £88080 00050m 000005 0000 m0 £88080 00050m £88080 00050m 3 .3 .00 00 000000000000 02000 0000 mg .00 £2000 000000000, 00 0000 .02 00 000000 _0000000> 00 50000000 ”m0 000 0200000 ”0000 Emu 00300000 H0 000 3 00 000008000 w 0000500 3 00 0000 :28 3 00 800080000 000000000 000000000 00—50008 ”mm— 00080000 ”00000003 000000— w500= £88850 .3 00 0000080000 ”0000500 :0 00 0000500 003000 02 003000 02 000 8000000000 00000080 000000000 000000000 000000000 .5085 02.00 0 508:0 000 X: 000 20 000 08.3.0 «0000500000 000000003 00000. SGQ 00 000300V 0000.00.00.00R .N 030.0 107 00220.00 0800 05 0020080080 00 8 000000 070 0w0000000 000000000000 05% £80800 0020000 0020000 .mu :80008204: .0000 020003 0500 .00000000000 00: 000m -0080 22000 -0080 22000 2 00500 000000.000 _000005 0000000000 00005000 “002000000000 .002000000000 .00 0055.000 .00500000 00000000 [00000000 00 000000 00000000 0500 .00000000 0500 .00000000 8 008000 l0=§bo>m 0000 080 .8280 82.. 000 050 00: 0050 0000 0050 020000 000000000 ...m: 00 020000 M 00003 0203 00000005 00000 00000 0000 0000 00 .0800 8000080000 00 0000 ...000800: :03 00 :0000000: g 0008 00.0 000 . 0000 00000 02 0032 000 00000000 000 000—000 000 0000000 .000800 00000 .00 0000000 000 000000 £0500 .0w0000 2003 0 0010: 60000003 .00000 .0000000 02000000 0020000 000 00 0500 :032000: 0 00.0 00 8000000000 A000 000 00000 00 ...0: 00000 050000 000 .00 00000 00 00 8000 000 0803 002.0000 003.0 N— 00 0000000 02 5 00 008050000 008300 200008: 0 00 0020 00 000 0000 .0000000> 0000 000000000, 000 .0580000 0008 00500 00000 000 :00500: 00000 00080200000 .00 0000 02 000 .00 nw500000000 =< 00000000000 0008 0000030: £22000 '8000 00000 000 0000005 0w00_ 030050000 022050000 000 00000 ”80000000 30: 00 0005000 5000005 000 000 :000000: n0 00 000000000 000080000 00000 0000000 02000800 00000002 00000000 02 0032 N1. 0000800000 005% 02000000 .00 00500 .00 000— 00 0>00 ”022000 5|.U 000—0000 .2000; w500000>0 .2000? w500000>0 ”00000 000000 .00 000800 00 H 0 0003 00 0000000 .m .< 00 080m 00000U.|2000000 .|0>000 00 080m l0>000 00 080m 0080 .w500000>m 5002.0 .200 0 5002.0 08 0.00 08 80 8. 08.3.0 «00000000090 000.0803 0000.0. 80G 00 0.003050 00080.008m K 030.0 108 0000000000 08080 0000>00000 000 0000 .0000000> .000000 0002000 .00 05000 -00 .02 00 05000 0000.0 0000200 .00 00000H00 .0000 00000000.» .00 00000 80% 000 000 .000 0000000 00002000 00 .0000 0200000 00 00 020 008V $5000 00w00 00 w 5000:": 00000 0000 0000 00000.0 00002000 .00 220 00w: 000 00 0.00.0-3 .000000000 000.0000 000000 0080 00 .8000 0000000 .0000000 0000000 0000000000 000200000 00 000 \3 05—000 Ewt 000 002V 0. 000500 00000 0080 0000000000 00000 000> 000000wnmq my. 00 w5000 .0000 ..Q: 00 A: .000 00200000 05000 000000 .9 8000080000 .05000 0000000 00 0000000 Ewt 00 000000 032000 0 800008 .0000000000 00 0.0 .00 .00 0000 000 -0000 000500-00 000000000 0.02 00002000 .00 .00 2000000800 20020000000200 0080 00000020000 00 0800 000 0200 600000000828 .008000000028 00 05000 0000 .00 0000 000 00003 _ A008 00 000 0 w500000 000 00 2w00 000 000 05:00 0800 000v 00000 0000005 £0 £000 .5 0080000828 00000000 .00000 000 .02500080 000000w 0000000 005000 0000000 -5 00 0008080 00003 00 000 0:0 £05000 0000000 0000000 805%.. 0800 000 0000.0 000080000 02.80 000500 n-00 000500 .00 0000m 50025 .800 0 808.5 08 0.00. 08 80 08 08.3.0 00000000090 00000000® 0003.0. 00000 00 0.00300V 300.00.00.80 ..0 030.0 109 020000000000 .5 be 002200 000000 000.000 0000000 :03 008 0000 000 002000000200 00000 0000 0000000030000 000 .00 0800 0000 000050000 800008000 00.5209 00 $00.00 0 000 00000w2000 0000 00 00030080000 2000000 000 00.0 000000800 00 000000 02000000 0000.800 0000000» 000 00 .0>000 00020008 00 0:000 00000 .000 .0000 000 8000 800800.00 00000000 .w5000 88.0 00800000 800080000 000 5 80 0000000000005 000 .00 0805000 000 00000000 00 0w500 00000 03 0. 002200 00 0800000 00080— 00080800 8 008m .00 0w023000 2 000000005 0008 0:00 0 .0000: 00 0. .5m008 80000 002200 00 0000.0 00 0000000 0000000 0003000 w500000 8000000000 100025 00--00200_0000 0 000 000 80000000 w50—08 00008000 .00 0003 000.000 0200 0 00 0000 0008 508 0 .00 0000. 32000008 000800 00200000 00 0.02 8000 0020000000000 .0003 0000 000 032000 0000 00030000 0000 0220000 0000000 00 2w00 00 0000 00000000000 000500 00 0000 00 00 000 0200000< 000 .05w008 8000000000 000 8000000000 000 8000000000 000 0.0.0002 000 000.0. 00000080000800 000000 00000000 £5w008 0.0000 .05w008 0.0000 £50008 000000 05w008 000000 0500008 00 0000 :0 000.0000 .2w00 000 00000000 .2w00 000 00000000 .2w00 000 00000000 .2w00 000 00000000 .2w00 000 0003 0200000000 :0 .00 00080800 0000000 0000000 0000000 0000000 0000000 0000000 0000000 0000000 0000000 000080 5 >020 000 00 000000080 0000000000. 00000300000. 000080000 0000800000. 8000300000. 0000800 .050 0 00000000 mm: 2: an: 00: 00: 002.005 0000000000» 000000.003 000000. 000Q 00 0.00an 800.000.2000 ..0 030.0 110 .0080 00022000 000023 E 022000 0 0000 0000000000 :00 000 00003 .00 0200000.» 0000 0 500000800 9 2.0 8022... .0030 2 2000000, 00500 000 00000 00 000000 00022000 000023 0:003 B s 5350 0008500 000:8 00 .2 Ba .0 .0 .00 00 2,0080 0 005 000 0000 00000 .00 00000 000 .0 000 00 02002820 020000 0022000 00 0020 00008 000 00003 00 0000 00020 00 050 80000200 02800000 020000 .00000 .0w2000008 00008 00000 003000 02 000 0203 00000. 032 00 0. 000 00082000 000 050 80.0000 000 020 80.0000 000 0.20 80.0000 00000 N— 0. -0 0.0. 5 023 000 0000.. 0.005 20 02606.0 5 00003 00000 $0 .0000 800.0 00 0000000000, 00000 9 032000 002.0 3 0 0o: 00.53 0.. 05 3 0.0 0050.000 0208 000250 0.0 3 0o 80. 000 02000 ”00 .0000000 _02w0_20 3 .00 00000000 0000000 08080 -002 .0 .080000 0000000 000020000 000080000 30-: 00 000500 0.0 .00 0200008080000 02 .00 00000000 .00 00000000 0000 02000 00208005 02000 000 m5000000 ”0000 0.02 0200000 0000 000000205 0000000002020 0000000000 ”3 .00 350000.. 0800 ”000.00 .m .0002000 0000 -00 00000000 0000 000000200 00.00 .00 00000000 00085 00000 0 .0 820008005 ”n..— .00 0000000 ”0000 000000 n0 0000 .3 .00 000 02000000 0032 002008 0000>000.0 0.00.0 .00 0000000 0200.05 0000000 000000000 0 .2203 020000 ”m 0000 .m .02w200000 00008 0 08.0 000 00008 0 000.0 0000000000 000 0200000 "0.0 0.000 ”0.0-: .00 0 8:0 .0 .0000800< .0 0800'0002 000 0300'0002 $000 0000 .00 0000000 00050m 000000000 00050m 808.5 .20... 0. 5.08.5 00. .8 08 «8 08 08.3.0 000000000000 00000000® 00030. 000Q 00 0.0030000 00000000000 ..0 030 .0 111 .0808 0000 0003 .0500 00000 00000 000 80 000 .00 00000w2000 _ 000 :03 00 000000 w558000 3 00 0:03 0 0000 0 0003 000000000 000 0.0 .NA .3 m.— .00 0w00 030008 000500 000 .00 050 00 000000000 c030 000000000 0200.05 0000 000 0000 000 003 00 000 00000 000 00 000500 .002 000 00w5 000008 0008 00 0000 00.0 _ 000 00008 w500000 00000000 .00 00000 ”Na .00 mg .00 000000000 m0 000 C085 0000 0000000.» 3 000 053 0000000 200000 00000 00000.0 0052000 000000 0000>00000 m.— .00 000000000 .00 0w00 0200000 on 000 000 0 300 00.6 000 0000000000 0200000 0000 00 .00 00000 000 050 0.00— 000 00w: 00000000000 02 :0 .mA .00 00008 00 00000000 00>00000 00000 ”0003 :0 000000w ”mg .00 .00 050 000 00000 30.3 .00 0000000 000000080 0008 000 003 Q 0>00 0 00m .00 050 000000w .NA .00 000000000 000—02000000 500 000000000 000500 0020 :10 .00 00:00:05 000.000 00000w2000 N0 .00 0000 0000>000.0 0030 0003 mg .00 05800 "000000 00000 0000000 000000000 0800 000 000 00000800 0200.05 Emu ”000000000 000500 mm.— .00 .NA 0.0 .00 000000 0000 ”3 .NA .3 .00 0:0 000 N0 03000— 000 0>00 _ .000 Z 00 0000 030.0 .00 00000 0000000 000000000 00050m 0200000 0.00.— 000000000 00050m 00000w2000 8000080000 00000.0 00002000 000 82008080 3 000 3 0200000 00008 00 00.000050 000000000 000500 0:00 0— 0000000000 .00 00500 m0 ”m0 000 0.— 000m 000000000 000500 000500 "NA 0w0000 05800000 0000000 0000000000 ”3 000000000 80 000000000 003000 0 Z 32 w5000 20000.4. 00 00000: 00.. 00.0 000m 000200000. 000000000. 08500 800%000 00— 0000900000 050002000 an 000 0100020000 050002000 00000 0000000 000 00000 0000000 .000000000 000 00000 000500 .08500 000500 .0000> 0000000000 000 .00 0000000 000500 0:00w00000 0:00w00000 00000 00085 50 000 0000300 000 0500 w— .5w008 0030 0200000 0000000 000500 Em 000 .00 02000 00085 0000300000 00000m2000 _00000 0200,05 000000 03000080200000 .000> 000800 Em 5 000 00 0520000 000000000 003000 02 .0050 0000000 00 w00000m 20.000040. 052000 2000000400 2000000000 000500 .00 0000m .5085 .200 0 208:0 08 X: 08 08 :0 083.0 0000020000000 00000000® 000000 SQQ 00 0.003020. 0000.00.00.800 K 030.0 112 dug—~50 05 00.5 8555 5:056 0005 003 and .55 00 050 57.00 0005 0005 050 353—0000 >5 m0 0500 5085 0002—30 0 00: 3.5 :5050539. 0500002068 A5305 0500.5: m 05 .«0 £000 005 05503005 N‘— 55000: 00330500050 $5305 15050: .010. 5 00395500 x 005 35005 0033050005? $203005 >009 ”36:003—500." 30—05905 =0>_w 0:0 .mEE >005 ”80595 $50500 v .m .NA ”00000003 Am5w005 0500.53 >009 v .m 002008 00 030 33 _ .8000 0030305 053305 0033050005? 30—02905 .N . 3 ”00000005 .000: 05? 050800500 000 353% ”$203005 n .v .m A $00003 ”0500002508 >000 v .m .N A 305% v .m 0: mo 00 2003 ”000805 >000 b05505 >000 ”00000003 305% .N as ”00000003 000 1253055 000 2 300330003 7: $000008Q Ewt m4 m4 ”00000003 n .v .mq ”0030003 053305 0:0 .o .w K .2. 35E :0 0:0 be 305% W: Amfimd 30> v 0505005 m .m .Nq 0505005 m .m .3 -n .m .3 3 30003 0505005 505000 3% 205 8% SEE 4003000.“ 330? 0.830300 0.3.3083 28:85 0:53 ”000003 .3 :55000 00500 505000 :5 000.0500 8 30: 00 3033 0503005 80: 00.530 NA 5300 ”m4 5 N: m 05 xooh doom 00.5 x02 00 :08 Ewt m4 ”000003 000003 305% 00330003 20600 Ewt 0950 :0 5 =< .2 ... 00 m4 008 _ 03630 0500.55 .«0 000% 0505005 £2 00950 :3w055 :35 «A ”55:00 2333 $0950 00 50% 3.005000— Ee 000:3 A8053 3 ”505000 3 ”m4 (#0 000000 3 ”5.5000 80: 050030 000003 053305 00 2:055 000 >530 02300005 0050030 0500.55 0030— 80: 050095 05300003 :0 AbtuEtS _N 0000 0003 35c 05 m0 $500005 00 Cnmtv 30? 00580005 0033003 3 ”5.5000 0950 000 05m 30m 0 :5 £080 .303 £05303 00330003 ”300500000 m4 53300 $05 05009.0 000003 305% 0 0.5 0.55 _ .0050 300005 ”m:— 000 N1— 00505.“ 050 :2 0500005 02300003 ”00300005 300 0 300 5:00 003 000 00308 :0 00330003 $530500 3 ”003.0003 3 ”0.0000003 000 $0003 _ ”00:00 :N: 05 000000 35038 ”0950 000 050 050-050 M53000 052,305 305% 605205 050355 3 00% 030 000|¢< 05 03 _ 0500 86:03 05253—5 55032 3035055 .«0 0950 .000 05m .mdv 05000 :7. 05 I305000500 ”00000003 50.0000 3 5005055 8000003 .8300.“ 5000.50 00 5300: 035 00:00 _ 0950 05 05W 305% =< 003053 N4 00000003 305%.. 050355. 000 00:50 003 _ 0030000 35,.— b 0020.30 me X: 2: NA: 2: 0003.5 $032580» 305.0:Q “00%. SuQ S 2033‘ ~§R~0mga~ .N 030& 113 .30 6300508 29:0 826 8 080808 0280050.: mo £825.80 .mzomwfi 000.55 mo 35:88 .3 5806000 05 no mam—E05500 05233 80 0303 >035 80808080 0053 0:0 820020002: 05 mo 000.33 05 2 085 5053 so 0:320 .000: 05 9 0:5 05 .5 080800 0000588 3028.50.08 3288000 588—0 05 2:00 0053 .000: .3 mo 005 $88 808v 000 A08080uv 000008 0805808 2500 88 8320 0020800 «2 05 no 0563 8020.5 5 088 0:0 0:0 0&050 085 580020808 00 v05tom0c 05 2:00 5.0300800 08 822200 5 mm 8000.8 0805808 80502 800800 0.5 008800.50 03288250 :0 .0065 0200.58 .530 00:86 638008 8 000: 05 >08 02 0:0 m2 .3 00 00:05 Ema .05-: 8m 805200000 _0..0>00 :5 do 3005500 8052 we mafia: 080 A3 00 008280 0.0608 05 :00 >05 5053 30:08 08.50005 Ewt 00 wag: 550583 SE 080 00000008 00 20>2 0:80:88 000 0:02 0:0 >85 N5 90 80088003 co 0805808 3 05 02 0:0 0>05 02:00:86 .0000 05 no 2008.00 0200.58 050t0> 02.8.58 00003 0000 25 8 3 00 020 60— .8 00000008 30825902 5030 30083 no 003530500 808 .5 00000008 w8:00 b—200amm 003590008 0:0 0803880 0:0 :85? 0.50:0; 032 .8085 0802808 .05 50 .0882 .5 .085 .00—0200; $85 0 m0 000: 8 war—00000 005% 05688 0882 mo 00 325808 3 mo 020 Ewt Stasmhotomfi £03088 258% 20 so» 0053 080 00 80508050 00950 802200 0300820 no 0035800000 .3 00950 08 00 0020800 “on 80 _ ”flora—85 808 m0 8 :5 ..<: 802200 E :5 802200 82550 8 :5 50 0&050 .5 $809. 8 ,=0 00: 8 F _ 8.5 0>50 m< ...<: 08 0250 m< E :5 ...<.. .8 m< ..<: 81. 0250 m< 05 530200800 508.5 .2: h .3085 m8 .8 8. 8. 8. .835 $032380» Eomummzm 000$. ~0an S 8033‘ 888.0qu ..m 030k 114 A038 .....5......U .30.... 08> 8.....0. A® 5.0.0.. .00 83 .... Ea .... 2.. =80...V .20.... 0.88.8. Ea 000250.. 000% 8B 000 0....5 500000885500. 85.3.0385 98> N .0000 0... ...00000880068 ..0 80.x.~ .0800... .3000. 8.0008 .0000 0... 0. 05 0.8 85:00 ..0 8.00.. t0> 0... ..0 808 0005.... 0... ..0 .0000 0... ..0 0.8.8 8w8..0 .8.w0.0 0800... 0005.8 ..0 0.008.. 8.08. 0.8 88000.... 05.000.00.50 0... 0... 8 000.00. 0808082058.. 0.... .0 0.8.... 0... 8 000.00. 008 0088.8... 0000 008 . 8.00.. 008 . 80800... mm .0. 00:00.8 0... 8 000.00. . 08> 8.....0. 0....0.> ..0 0.8.8 t0> 0... 0. m5w8E 00.05.85 0... .8580. .00.. ..0 008 . 08> w... 0.... 0.8.0008 8 000.00. 008 . 5.8.5 0.8 00 85.8.3.0... 88.385000 8.....0. 2m 0... I80800... .8m00 888. 0.... 00.00000 05 0.8 ..0 808 08 60.80208...» 00 38.00080 00.08000 ..0 0.8.8 008 >009 8.00.. ...0> 0... 8.00.. t0> 0... 0800.08.00.08 400500.080 0... .0 0.8.8 t0> 0... ..0 05.000 ..0 8&8... 8.08. ..0 8&8... 8.08. ..0 808 £508.. 0.0.5.0080 0... 8 38.00.80 0... 8 b80588 0.0 ..0 0.8.8 0... 0... (.0 0.8.8 .8308 07. 000.. 8.0008... 0... 8 000.00. . 000.00. . 000.00. . 8 000.00. . .50»... 0... 8 000.00. . .8508... 085 0. 0.00. «000.008.. ..080 w00. 30.. 305. 0. 9.08.0.5 0.. ...»..2 8.00.. 850.00., 5.80.0 .00.. 0. ..0 8w00 5.0000... 000 0.80000 0.80. 0. 800000 .8088... 0005.... w... 5880.03. .80.. ..0 000.00 000500.080 0.8-00.8 00038.. .. 8.580.. .0 0.0300080 800000008080 o. ..0 000.. ..0 8.8.8 8008.. 8.0008 38.00.80 85000 0.000.800.0000... «N 50.55... 0 .0. 000.00. .88>0 .8850 0005.00.58.30 >000 ...... ..0 0w00 8.000... .80008 . 88380... ...08... £00800... 6.. 38.0008 8.80.30 0. 0.00. 88000.... 8.0008 500.580.8000 00.. 50.00588 000500 88000... 88000... 000500 :00..0.: 005.5... 00.00.580.803 .m. 38.00.80 .8005. 00.00.. a30.2.0.5... 0005...». .8.00.. .08...0t0> 00038.. $0.800 ...... 8.00.. 830.00.. .0 00... 0.8 50000008.. 800000 850t0> ..0 80.8.8.8 58.0.8.8 38.0098 8.00.. 05:00 .80800... ..080 5 8.080. 80.8.0008... 8.000 5.3.00 A05... 0800... .00.. 0. 85.0.8 ..0 05.000 0005..» .8880... 00.0.00 .0. 000.00. . 00.0.03 0.0300080 00.0880 000.03 000 500......0 80.5 .8880... 0005.5 00.0.68... 9. 00.0800 .00... b 00.8000 mm: 2.. mm: ND. 2: «00.0.5 «800:...000\ 0:0...8:® 80.00 SQQ 0. 0L0§0=V 30.00....8m K 0305 115 APPENDIX H Features Cited by Participants when Making Matches between Radiographs 116 Table 8: Features Cited by Participants when Making Matches between Radiographs ID #1 — Unknown A/Known 14 Feature General L1 L2 L3 L4 L5 1, 18, 19, Spinous 21, 23,24, 20, 22, 14, 2, 22, 14, 22 14 16 2, 20, 22, 3, 2,21, 7,14, Processes 3,5,6,8,9, 16 16 ’ ’ 14,11,16 16 10, 12, 11 18(L). Transverse 1 , 21 , 23, 5, 20 WR)‘ N0 °“° 20(R), 21, 22, 4, 6 7 22, 6(R), 7 used L4 Processes 6, 12, 15 (L/R) 7 8 s ecificall 22,24,3,4, ’ ' p y 6(L), 8, 11 1,23, 24, 3, Gem“ 5,6,8,10, 2,3,22,24, 20,22,3.4, N0 0“" (contour, 22 22, 7, 9, 16 6 6 used L5 shape) 12’ 11’ 15’ 7’ 9' 1 14’1 specifically 17,25 cNisegne N 0 one No one N 0 one No one Trabeculae used Ll used L2 used L3 used L4 21,4 (TP) Trabeculae s ecificall s ecificall s ecificall s ecificall generally P Y P Y P Y P Y 23 10 12 No one No one No one Pedicles ’ ’ ' used Ll used L2 21 , 9 9 used L5 11 . . . spec1fically spec1fically spec1fically No one Centrum° No one Radiodensities 25 used L1 ' TP(L): 4 4 used L5 . 21 , 4, 6, 12 . spec1fically spec1fically No one No one No one No one No one Radiolucencies 12, 25 used L1 used L2 used L3 used L4 used L5 specifically specifically specifically specifically specifically 24 5 14 No one No one No one No one SAP’s/IAP’s 25’ ’ ’ used L1 used L2 4 used L4 used L5 specifically specifically specifically specifically Osteophytes No one No one No one (development! 12 used L1 11 14 used L4 used L5 lipping) specifically specifically specifically No one . No one No one No one Neural Arch Cited Neural used L1 used L2 14 14 used L5 Contour Arches s ecificall s ecificall s ecificall enerally p y p y P Y 117 Table 8: Features Cited by Participants when Making Matches between Radiographs (Continued) ID #2—Unknown B/Known 10 Feature General L1 L2 L3 L4 L5 1, 18. 19. Spinous 23, 24, 3, 8. 7 7 14 20, 22, 4, 7, 2, 19, 20, 22. “9,723,122. Processes 9.10.12, ’ 14,11 4,7,14,11 1’1 16’ ’ 13, 15. 25 . Transverse 21,23,5.9. 22,4,6.7. 3, E; °“° used 20 22 4 6 7 3°” "5°“ 19,20,22,6, Processes 12,15 16 . ’ ’ ’ ' . 7,8,14,11 spec1fically spec1fically 1, 23, 24, 4, Centrum 5,9,10,12, 16 22,3,4,7, 2,19,22,3, galg'go‘liz’ 16 (contour, shape) 13,15,17, 16 7,16 16’ ' ’ ’ 25 No one used No one used No one used No one used No one used Trabeculae 14 L1 L2 L3 L4 L5 specifically specifically specifically specifically specifically 23 9 10 No one used No one used No one used No one used No one used Pedicles 12‘ 15 ’ L1 L2 L3 L4 L5 ' specifically specifically specifically specifically specifically No one used No one used No one used Radiodensities 18. 23 , 12 L1 L2 4, 6 6 L5 specifically specifically specifically No one used No one used No one used No one used No one used Radiolucencies 22, 12 L1 L2 L3 L4 L5 specifically specifically specifically specifically specifically No one used No one used No one used No one used SAP’s/IAP’s l , 24, 5 L1 L2 L3 14 L5 specifically specifically specifically specifically Osteophytes No one used (development! 5,9, 10,12, L1 21,3 18,21,23,3, 18,21,23,3, 4,8,16 15 . 7,11 4.6.7.8,11 lipping) spec1fically No one cited No one used No one used No one used No one used Neural Arch Neural L1 L2 L3 L4 1 6 Contour Arches specifically specifically specifically specifically generally No one used No one used No one used No one used No one used Syndesmophytes 22 L1 L2 L3 L4 L5 specifically specifically specifically specifically specifically No one cited No one used No one used No one used Lamina Lamina L1 L2 23 23 L5 generally specifically specifically specifically Notes: Participant #16 answered correctly, but indicated that he/she is uncertain of the match Participant #25 noted that radiotranslucency and radiopacity were not useful because of a slightly different angle between the radiographs 118 Table 8: Features Cited by Participants when Making Matches between Radiographs (Continued) ID #3—Unknown C/Known 9 Feature General L1 L2 L3 L4 L5 1, 18. 21, Spinous 23. 24. Processes 3,9,10’ 21 20,7 7 20.3.7.14 20,3 12 19, 20, 21. Transverse 23. 12, 14, 22, 23, 7, 8, Processes 25 6 22’ 7 22' 7' 8 6 14.13.16, 15 1.23. 24, Centrum 3.5.10. 2 2 16 24.3.7.8, 19.3.4.7. 1:; m “S“ (contour, shape) 12.14.13, ’ 16 16 s ificall 15, 17, 25 9°C y No one No one used No one used Trabeculae we" L1 L2 SP: 3 9 TP: 14. 16 Trabecula specifically specifically e generally No one used Pedich 23. 10, 12 23 23. 9 23. 9 23 L5 specifically No one , No one used cited L4 Radiodensities Radiodens 2 2 4, 6. 9 specifically 4 ities generally No one uscd No one used No one used No one used Radiolucencies 25 L1 L2 21 . 22. 9 L4 L5 specifically specifically specifically specifically No one No one used No one used cited L1 L4 SAP’s/IAP’S SAP’s/IA specifically 14 14 specifically 4 P’s Enerally Osteophytes No one used No one used 2, 13, 21 , 23, No one used (development! 24.12.15 L1 L2 24.3.5.9. 23,11 L5 lipping) specifically specifically 14, 11 specifically No one No one used No one used No one used No one used No one used Neural Arch cited L1 . L2 ' L3 . L4 . L5 . Contour Neural spec1fically spec1fically spec1fically speafically spec1fically Arches generally No one No one used No one used No one used No one used Lamina cited L1 L2 L3 L4 20 Lamina specifically specifically specifically specifically generally No one used No one used No one used No one used No one used Syndesmophytes 22 L1 L2 L3 L4 L5 specifically specifically specifically specifically specifically Notes: Participant #2 answered incorrectly (misidentification) Participant #16 made a correct match. but noted that he/she is. “not 100% confident with the match” Participant #24 noted poor quality of the simulated antemortem image 119 Table 8: Features Cited by Participants when Making Matches between Radiographs (Continued) ID #4— Unknown D/Known 20 Feature General L1 L2 L3 L4 L5 Spinous 1. 18. 20, No one used No one used Processes 23,24.3,8. 2 2 L3 L4 1.7 9. 13 specifically specifically 20. 21 . 22. annsvem 1:258, 10' 6 2o 21. 22, 6 22, 6, 7. 25 23, 24.4, 7, ' 12, 14, 25 1.20.23.5, No one used ($22121? she ) 9.10.13. 25 3’22'3'7' 2.21.22.7 22 L5 ’ 9° 15 specifically No one No one used No one used No one used . No one used No one used Cited L1 L4 L5 Trabeculae Trabeculae specifically L2 ‘1' all L3 . f 11 specifically specifically generally spec1 1c y spec1 ica y No one used No one used No one used Pedicles 23. 3. 10 L1 9 3 L4 L5 specifically specifically specifically No one used No one used No one used No one used No one used Radiodensities 23 L1 L2 L3 L4 L5 specifically specifically specifically specifically specificallL No one used No one used Radiolucencies 22 L1 22, 23 6 6 L5 specifically specifically No one used No one used No one used No one used No one used SAP’s/IAP’s 24 L1 L2 L3 L4 L5 specifically specifically specifically specifically specifically Osteophytes No one used No one used (development/ 18, 10. 15 9 L2 2. 23 23 L5 lipping) specifically specificallL No one No one used No one used No one used No one used Neural Arch cited L1 L2 No one used L4 L5 Neural specifically specifically L3 specifically specifically Contour . Arches speCIfically generally No one No one used No one used No one used No one used Mammary cited L1 L2 L4 L5 Pr Mamillary specifically specifically 22 specifically specifically ocess Processes Sammy Number of No one used No one used No one used No one used No one used Vertebrae 22, 23. 4 L1 . L2 _ L3 . L4 . L5 . spec1fically spec1fically spec1fically spec1fically spec1fically Notes: Participant #2 answered incorrectly (misidentification) Participant #6 answered incorrectly (misidentification) Participant #11 chose not to make a match for Unknown D Participant #12 believes that his/her correct answer is probably correct. cannot exclude, but would need more points— he/she only used the transverse processes of L5 to make the match Participant #13 answered correctly, but indicated that he/she is not positive about the 1D Participant #14 noted that the angles of the radiographs were not quite the same Participant #16 chose not to make a match for Unknown D Participant #17 chose not to make a match for Unknown D Participant #18 answered incorrectly (misidentification) Participant #19 chose not to make a match for Unknown D, indicating that there were not enough points of similarity 120. Table 8: Features Cited by Participants when Making Matches between Radiographs (Continued) ID #5— Unknown E/Known 3 Feature General L1 L2 L3 L4 L5 1, 18, 19, 21. Spinous 23.3.5.6,9, 2,20,22,7. 2,20,22,24, Processes 10’ 12.14, 20,22,3.7 20,22,7 2.22.7 l4 7 13, 25 Transverse No one used No one used No one used No one used No one used Processes 23. 12. 15 L1 L2 L3 L4 L5 specifically specifically specifically specifically specifically No One used centrum 1'3’9'10' 22.13 22, 7 22.3.7 2.22.7 L5 (contour, shape) 12, 15, 17 . specrfically No one used No one used No one used No one used No one used Trabeculae 22 L1 L2 L3 L4 L5 specifically sppcifically specifically specifically specifically 21 23 3 9 No one used No one used No one used No one used Pedicles 12’ ‘ ‘ ’ L1 3 L3 L4 L5 specifically specifically specifically specifically No one used No one used No one used No one used No one used Radiodensities 18, 23, 6. 12 L1 L2 L3 L4 L5 specifically s cifically specifically specifically specifically No one used No one used No one used No one used No one used Radiolucencies 12 L1 L2 L3 L4 L5 specifically specifically specifically specifically specifically No one used No one used SAP’s/IAP’s 1. 24. 5, 14 11 L2 L3 4 4 specifically specifically No one used No one used No one used No one used SAPS 19 L1 20 L3 L4 L5 specifically specifically specifically specifically 18.23,24.8, No one used 2.1.19.20, No one used 05mph“ 9 10 12 14 L1 21 23 3 4 L5 (development/ 1'3 1’5 127 ' 'f 11 6 ’1' ’ 21.23,3,11 23.3 'f 1] lipping) , . , spec1 1ca y 5, , 7. 2, spec1 ica y 25 14, 11, 25 No one cited No one used No one used No one used No one used No one used Neural Arch Neural Ll _ L2 L3 L4 L5 Contour Arches specifically specifically specifically specifically specifically generally Size of No one used No one used No one used No one used No one used Vertebrae 20 L1 . L2 . L3 . L4 . L5 . spec1fically spec1fically spemfically _spec1fically spec1fically No one used No one used No one used No one used No one used Syndesmophytes 22 L1 L2 L3 L4 L5 specifically sgcifically specifically specifically specifically No one cited Intervertebral Intervertebra if? m “sed 1:; °"c “5"“ 1:; °“° “S“! 25 (b/t 4 and 25 (on 4 and Space 1 Space s ificall s ificall s ificall 5) 5) generally P‘3c Y P69 Y pec Y Notes: Participant #8 answered incorrectly (misidentification) Participant #15 answered incorrectly (misidentification, with a qualifying statement) Participant #16 chose not to make a match for Unknown E 121 REFERENCES 122 REFERENCES Angyal and Dérczy (1998) Personal Identification on the Basis of Antemortem and Postmortem Radiographs. Journal of Forensic Sciences 43: 1089-1093. Baker AM (2005) Human Identification in a Post-9/11 World: Attack on American Airlines Flight 7 and the Pentagon Identification and Pathology. National Disaster Medical System (NDMS). Bontrager KL (2004) Pocket Handbook of Radiographic Positioning and Related Anatomy. Chicago: Mosby Year Book. Brogdon BG (1998) Forensic Radiology. New York: CRC Press. Chen MYM, Pope TL, Ott DJ (2004) Basic Radiology. New York: McGraw-Hill. Christensen AM (2004a) The Impact of Daubert: Implications for Testimony and Research in Forensic Anthropology (and the Use of Frontal Sinuses in Personal Identification). Journal of Forensic Sciences 49(3): 427—430. Christensen AM (2004b) Assessing the Variation in Individual Frontal Sinus Outlines. American Journal of Physical Anthropology 127: 291-295. Ciaffi R. De Angelis D. Gherardini PF, Arcudi G, Nessi R. Comalba GP. Grandi M, and Cattaneo C. (2010) Identification from Chest X-Rays: Reliability of Bone Density Patterns of the Humerus. Journal of Forensic Sciences 55(2): 478-481. Coolidge FL (2000) Statistics: A Gentile Introduction. London: Sage Publications. Comelison JB. Fenton TW. Sauer NJ , deJong J , and Hunter BC (2002) Comparative Radiography of the Lateral Hyoid. Presented at the 2002 Annual Meeting of the American Academy of Forensic Sciences. Atlanta, GA. Dix J and Graham M (2000) Time of Death. Decomposition and Identification: An Atlas. Boca Raton. FL: CRC Press LLC. Elliott RB (1953) The Value of Roentgenology in the Identification of Mutilated and Burnt Bodies. The Journal of Criminal Law, Criminology, and Police Science 43(5): 682- 684. Evans KT. Knight B , Whittaker DK (1981) Forensic Radiology. Oxford: Blackwell Scientific Publications. Fischman SL. The Use of Medical and Dental radiographs in identification. Int Dent J 1985; 35: 301-306. 123 Hogge J P, Messmer JM. Doan QN (1994) Radiographic Identification of Unknown Human Remains and Interpreter Experience Level. Journal of Forensic Sciences 39(2): 373-377. Hulewicz B, Wilcher GW (2003) The Use of Thoracolumbar and Hip Joint Dimorphism in Identification. Journal of Forensic Sciences 48(4): 842-847. J ablonski NG, Shum BS (1989) Identification of Unknown Human Remains by Comparison of Antemortem and Postmortem Radiographs. Forensic Science International 42(3): 221-230. Kahana T , Goldin L, Hiss J (2002) Personal Identification based on Radiographic Vertebral Features. American Journal of Forensic Medicine and Pathology 23(1): 36-41. Kahana T, Hiss J (1997) Identification of Human Remains: Forensic Radiology. Journal of Clinical Forensic Medicine 4(1): 7-15. Kavanaugh SA (2002) Sella Turcica and its Use in Body Identification. Proceedings of the American Academy of Forensic Sciences Annual Meeting, Atlanta, GA. Kuehn CM. Taylor KM, Mann FA, Wilson A] . CB MB, Harruff RC. PD MD (2002) Validation of Chest X-Ray Comparisons for Unknown Decedent Identification. Journal of Forensic Sciences 47(4): 725-729. Koot, Michael, Norman J. Sauer. Todd W. Fenton (2005) Radiographic Human Identification Using Bones of the Hand: A Validation Study. Journal of Forensic Sciences 50(2): 263-268. Kumho Tire Co., Ltd.. et al. v. Carmichael et al. 526 US. 137 (http://law.onecle.co@/ussc/ 526). Lichtenstein J E. Fitzpatrick JJ , Madewell JE (1988) The Role of Radiology in Fatality Investigations. American Journal of Roentgenology 150: 751-755. Martel W, Wicks JD. Hendrix RC (1977) The Accuracy of Radiographic Identification of Humans using Skeletal Landmarks: A Contribution of Forensic Pathology. Radiology 124: 681-684. Mundorff, Amy 2., Giovanna Vidoli, Judy Melinek (2006) Anthropological and Radiographic Comparison of Vertebrae for the Identification of Decomposed Human Remains. Journal of Forensic Sciences 51(5): 1002-. .. Murphy WA. Spruill FG. Gantner GE (1980) Radiologic Identification of Unknown Human Remains. Journal of Forensic Sciences 25(4): 727-735. Murphy WA. Gantner GE (1982) Radiologic Examination of Anatomic Parts and Skeletonized Remains. Journal of Forensic Sciences 27(1): 9-18. 124 Owsley DW, Mann RW (1992) Positive Personal Identity of Skeletonized Remains using Abdominal and Pelvic Radiographs. Journal of Forensic Sciences 37(1): 332-336. Owsley DW, Mann RW. Chapman RE. Moore E, Cox WA (1993) Positive Identification in a Case of Intentional Extreme Fragmentation. Journal of Forensic Sciences 38(4): 985-996. Paolello JM and Cabo Pérez LL (2008) Elliptic Fourier Analysis of Vertebral Outlines for Victim Identification. Paper presented at the American Academy of Forensic Sciences Annual Meeting. Washington, DC. Quatrehomme G () Identification by Frontal Sinus Pattern in Forensic Anthropology. Forensic Science International 83(2): 147-153. Rich J , Tatarek NE. Powers RH. Brogdon BG. Lewis BJ, Dean DE (2002) Using Pre- and Post-surgical Foot and Ankle Radiographs for Identification. Journal of Forensic Sciences 47(6): 1319-1322. Rogers TL, Allard IT (2004) Expert Testimony and Positive Identification of Human Remains through Cranial Suture Patterns. Journal of Forensic Sciences 49(2): 203-207. Sanders I. Woesner ME, Ferguson RA. Noguchi IT (1972) A New Application of Forensic Radiology: Identification of Deceased from a Single Clavicle. American Journal of Roentgenology 115(3): 619-622. Sauer NJ . Brantley RE, Barondess DA (1988) The Effects of Aging on the Comparability of Antemortem and Postmortem Radiographs. Journal of Forensic Sciences 33(5): 1223- 1230. Scott AL. Congram D, Sweet D. Fonesca S, and Skinner M (2010) Anthropological and Radiographic Comparison of Antemortem Surgical Records for Identification of Skeletal Remains. Journal of Forensic Sciences 55(1): 241-244. Telmon N. Allery JP. Scolan V. Rouge D (2001) A Case Report Demonstrating the Value of Chest X-Rays in Comparative Identification. J. Clin. Forensic Med. 8(22): 77-80. Tilstone WJ . Savage KA. Clark LA (2006) Forensic Science: An Encyclopedia of History. Methods, and Techniques. Santa Barbara, CA: ABC-CLIO, Inc. Ubelaker. Douglas H. (1984) Positive Identification from the Radiographic Comparison of Frontal Sinus Patterns. In, Ted A. Rathbun and Jane E. Buikstra (Eds .). Human Identification: Case Studies in Forensic Anthropology. Springfield: Charles C. Thomas; 399-411. Valencuela, Aurora (1997) Radiographic Comparison of the Lumbar Spine for Positive Identification of Human Remains: A Case Report. American Journal of Forensic Medicine and Pathology 18(2). 215- 217. 125 Walsh M. Reeves P, Scott S (2004) When Disaster Strikes; the Role of the Forensic Radiographer. Radiography 10(1): 33-43. Weiler CM, Taylor KM and Harruff RC (2000) Validation of Anteroposterior Chest X- Ray Comparisons for Unknown Decedent Identification. Abstract in the Proceedings of the AAF S Annual Meeting, Reno, NV, February 21-26. 126