ATTENTION AND WORKING MEMORY: DISRUPTING FEATURE BINDING IN VISUAL WORKING MEMORY By Jonathan G. Hakun A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Psychology - Doctor of Philosophy 2013 ABSTRACT ATTENTION AND WORKING MEMORY: DISRUPTING FEATURE BINDING IN VISUAL WORKING MEMORY By Jonathan G. Hakun In the current study we investigated disruption of working memory for multi-feature visual objects through a visual change detection paradigm modified with a distraction manipulation (termed the “Binding Distraction Task”). The paradigm included three processing stages (study, maintenance, and test) with a distraction event presented during maintenance. The judgment performed on each trial involved deciding whether a single two-feature test object (a colored shape) contained the same or changed combination of shape and color as one of the study objects (i.e. contained the same feature binding). The distraction event involved presenting an array of visual objects arranged spatially in a manner similar to the study array. We compared performance on trials where distraction arrays contained objects sharing feature-overlap with the current trial’s study objects (matched arrays) with trials where distracter objects contained study-related features but no overlap with the current trial’s study set (related arrays). As compared with trials involving empty distraction arrays and baseline performance (no distraction), the presentation of matched arrays was consistently disruptive of binding change detection performance across all of the experiments presented here. The performance cost associated with related arrays however was more variable across experiments. Through an individual differences approach, we investigated an executive gating account of the distraction effect. That is, disruption is caused by a failure to keep distracter objects from reaching visual working memory (VWM) thereby displacing study objects. Based on this account we predicted that individuals with high working memory capacity (WMC), as measured through complex span and single feature change detection task performance, should be more resistant to distraction than low WMC individuals. However, the results of our individual differences analysis failed to provide support for the executive gating hypothesis (costs were unrelated to WMC). Alternatively, we proposed that distraction costs may reflect a special case of contingent attention capture. That is, attention may be biased toward distracter objects that overlap with the features active in VWM. To explore this hypothesis further we additionally conducted an fMRI study. The results of our fMRI analysis indicated that regions of the brain that are commonly involved in the voluntary and stimulus-driven control of attention responded in a manner consistent with our contingent capture hypothesis. Regions involved in the voluntary control of attention (superior parietal and prefrontal) parametrically scaled in activation in accordance with the proposed demands associated with each distraction condition. However, as further evidence against the executive gating hypothesis, activation in all of the regions involved in voluntary control did not relate to performance costs. On the other hand, regions associated with the contingent capture of attention (specifically, the right ventrolateral prefrontal cortex, VLPFC, and temporo-parietal junction) only reliably responded to the filled distracter arrays (related and matched arrays). Further, activation within the right VLPFC was positively related to the performance cost observed in both conditions. Altogether the results of the current study support a stimulus-driven account of disruption of VWM for bound features. Specifically, we conclude that maintaining multiple bound representations in VWM creates a context in which distracter objects containing feature-overlap capture attention and result in the downstream impoverishment of stored representations. ACKNOWLEDGEMENTS I would be remiss if I did not acknowledge the many people who have immeasurably aided, inspired, and supported my goal of reaching this milestone. To my immediate and extended family, especially Mom, Dad, and Laura, thank you for the unconditional love and support you’ve given me as I’ve found my way through this process. You have each made immeasurable contributions to my perspective, motivation, and goals and I could not have persevered through the tides without you. To my future forever partner in crime, Dr. Allison Fleming. I could have in no way foretold that the greatest asset I’d gain in life by attending Michigan State would come in the form of a Mets fan from Queens. Thank you for being the source of endless inspiration, a nonjudgmental advocate, allowing me to process just about every idea I’ve had in the past 4 years out loud, and for being my best friend. Wherever our careers take us across this country, you’ll always be my own kind of home. To my friends, especially the boys in 39M: thank you for remaining confident in my abilities, trusting in my dedication to our mutual interests, and for the mental health that a few hours on stage each month can provide. RnR PhD! Along my academic path I’ve met several key people who have either given me a chance, inspired my interest in the field, served as a source of excellent mentorship, or contributed all of the above. The first of these is Dr. Jonathan Brumberg. From our childhood philosophical debates and wonderings to the times you shared your intrigue with the field of cognitive science after college, the light bulb turned on in my mind because of you. To Dr. Anna iv Rose Childress, thank you for giving a guy with a social work degree a chance to train in brain imaging. I will forever be indebted. To Dr. Daniel Langleben, thank you for the opportunity to make my first mark on the field and for facilitating my direction into graduate school. To all the members of my dissertation committee, Drs. Taosheng Liu, Zach Hambrick, and Tim Pleskac, thank you for your guidance and direction throughout the comprehensive exam and dissertation process. To Drs. Tim Pleskac and Erik Altmann, I’d especially like to thank the two of you for leaving your doors open to me. While the specific projects, topics, and methods you mentored for me have not coalesced into a published paper [yet], you have made a profound contribution to my scientific thinking that finds its way into just about every project I take on. Finally, to Dr. Susan Ravizza, it was a privilege to be your first graduate student, and an honor to now be part of your Neuro [family] Tree. Thank you first and foremost for those 5 now notorious words: “you don’t wanna go there.” Without being provoked to attend MSU with that (strikingly convincing) phrase I would have missed out on an outstanding graduate advisor. Thank you for the years of hard work you put into helping me develop as a scientist, for the countless resources you provided, and for the care you show in your work. I’ll do my best to be a reflection of the same in my future work. All of the most worthy endeavors in my life have begun with the support of the people who mean the most to me. Thank you all. Oh, and to Pasha and Yuri, together we arrived in MI and together we’ve moved onto new horizons. Thanks for all the love, affection, and company, I couldn’t have done it without you guys. v TABLE OF CONTENTS LIST OF TABLES vii LIST OF FIGURES viii CHAPTER 1 GENERAL INTRODUCTION 1 1 CHAPTER 2 DISRUPTION OF BINDING CHANGE DETECTION Experiment 1 – Distracter Array Presented 250ms after Study Array Methods Participants Binding Distraction Task Results Corrected Recognition Response-time (RT) Analysis Exposure to Distracters RT for Test Objects Interim Discussion Experiment 2 - Distracter Array Presented 1300ms after Study Array Methods Participants Binding Distraction Task Visual Array Task Automated Complex Span Measures Automated Operation Span Automated Symmetry Span Procedures Results Binding Distraction Task Corrected Recognition Response-Time Analysis Exposure to Distracter Array RT for Test Objects Individual Differences Measures Interim Discussion General Discussion Conclusions 14 14 18 18 18 19 22 22 23 23 24 24 26 26 26 27 27 28 28 28 30 30 30 31 31 32 32 33 35 35 44 vi CHAPTER 3 DISRUPTING BINDING CHANGE DETECTION THROUGH THE CONTINGENT CAPTURE OF ATTENTION Methods Participants Binding Distraction Task (BDT) Image Acquisition Image Analysis Preprocessing Individual Subject Analysis Group Analysis Region-of-Interest Analysis (ROI)/Percent Signal-Change Results Behavioral Results Accuracy Response Time Imaging Results Whole-Brain Analysis ROI Analysis Costly Activity Protective Activity Discussion Beyond the VAN and DAN Anterior Insula and Anterior Cingulate Control and Storage of VWM Representations Conclusions 45 58 58 58 62 62 62 63 63 64 64 64 64 65 66 66 72 73 74 76 80 81 85 89 CHAPTER 4 GENERAL DISCUSSION 91 91 REFERENCES 98 vii 45 LIST OF TABLES Table 1 – Correlation Table of Binding Distraction Task and Individual Differences Measures 34 Table 2 – Results of Whole-brain Analysis and Contrasts Between Conditions 75 viii LIST OF FIGURES Figure 1. Study Design. (TOP) Example of each color and shape used to construct study objects (MIDDLE) Timecourse of each trial (BOTTOM) Example of each distraction condition. In Experiment 1 the pre-distracter fixation lasted 250ms and post-distracter fixation lasted 2250ms. In Experiment 2 the pre-distracter fixation lasted 1300ms and the post-distracter fixation lasted 1200ms. For interpretation of the references to color in this and all other figures, the reader is referred to the electronic version of this dissertation. 22 Figure 2. Corrected Recognition Rates and RT for Experiment 1. A significant reduction in accuracy was observed between no distraction and empty as well as empty and related. No difference was observed between related and matched. Key: N = No Distraction, E = Empty, R = Related, M = Matched. 23 Figure 3. RT for Distracter Array and Test Object for Experiment 2. (LEFT) RT for the distracter array. (RIGHT) RT for the test object. 24 Figure 4. Corrected Recognition Rates for Experiment 2. A significant reduction in accuracy was observed between no distraction and empty as well as between related and matched. No difference was observed between empty and related. 31 Figure 5. RT for Distraction Array and Test Object. (LEFT) RT for response to the increase in fixation during the distracter array. (RIGHT) RT for test object. 32 Figure 6. fMRI Task Design. (TOP) Example of all colors and shapes used to construct study objects (MIDDLE) Timecourse of each trial (BOTTOM) Example of each distraction condition. 61 Figure 7. Corrected Recognition for Each Condition. A significant reduction in accuracy was observed between empty and related, and empty and matched arrays. No difference was observed between no distraction and empty or related and matched conditions. 65 Figure 8. RT for Test Objects. No difference in RT was observed across any condition at test. 66 ix Figure 9. fMRI Main-Effect of Condition and Planned Comparisons. (TOP) Regions showing a significant main-effect of condition (MIDDLE) Results of planned comparisons overlaid on the main-effect of condition (BOTTOM) Results of the contrast matched > empty overlaid the results of related > empty. 67 Figure 10. Regions Showing Parametric Increase in Activity across Conditions. Results of matched > empty thresholded at FWE-corrected p < 0.05. Several regions showed a significant increases in activation across each condition including the SPL/Prec, bilateral IPS, and bilateral LOC. Two sub-clusters within each the left and right IPS and left and right LOC were identified through the contrast of matched > related. As each smaller cluster showed the same pattern of activation across conditions, percent signal-change estimates shown here are only for the inferior IPS and lateral LOC/Occ Fus clusters. 69 Figure 11. Regions of the DAN Showing Onset- and Object-Sensitivity. The left FEF and IFG (black outlines) displayed array onset- and object-sensitivity while the right FEF and IFG (blue outlines) only showed object-sensitivity. 70 Figure 12. Regions Showing only Array Object-sensitivity. Several regions showed object-sensitivity including those associated with the VAN. 71 Figure 13. Sub-cortical Regions Showing a Main-effect of Condition. 72 Figure 14. Individual Differences in ROI Activation and Distraction-costs. Scatter-plots showing the relationship between distraction-costs for related (blue) and matched (green) array conditions and (A) right AI, (B) right LOC/Occ Fus, (C) left Pallidum, (D) right dorsal LOC. 74 x CHAPTER 1 GENERAL INTRODUCTION The features of a visual object present in the environment often occupy the same spatial location, or are contiguous with one another. However, specific features are represented neurally in early visual cortex across distinct populations of neurons that are selectively tuned to each feature (e.g. color, orientation, etc; Tootell, Dale, Sereno, & Malach, 1996). How the cognitive system reconciles the integration of neurally disparate visual features into emergent integrated mental object representation is what is commonly referred to as the ‘binding’ problem (Treisman, 1996). Debate exists over binding is resolved in a spatial manner (Treisman, 1980, 1996; J. Wolfe, 1994; J. M. Wolfe, Cave, & Franzel, 1989) or through objectbased principles (Lee & Chun, 2001; J. M. Wolfe & Bennett, 1997; see Scholl, 2001 for review). However, a commonality among these various models of visual perception is the agreement that the deployment of visual attention is the key mechanism that arranges these neurally disparate features into an emergent integrated mental object representation (Chun & Johnson, 2011; Lee & Chun, 2001; Treisman, 1998; Treisman & Zhang, 2006; Wheeler & Treisman, 2002; J. Wolfe, 1994; J. M. Wolfe & Bennett, 1997). In other words, the deployment of attention to a visual object results in the mental binding of its features. Visual information presented for exceedingly brief periods of time is only available at the percept for a fleeting moment (on the order of ~150ms) before it is no longer accessible (e.g. ‘iconic’ memory in Sperling, 1960). However, information that is attended before the decay or disruption of its iconic trace (e.g. through backward masking) is given a chance to stabilize into a more durable memory form, a process often referred to as visual working 1 memory consolidation (Vogel & Luck, 2002; Vogel, Woodman, & Luck, 2006; Woodman & Vogel, 2005). Given the role of attention in binding features in perception, early theories of visual working memory (VWM) had proposed that post-consolidated multi-feature objects in VWM would require sustained attention in order to remain bound (Wheeler & Treisman, 2002). However, recent studies have rejected this hypothesis, as the relationships between features of multi-feature objects (i.e. the ‘bindings’) appear to become self-sustaining after consolidation into VWM (Allen, Baddeley, & Hitch, 2006; Delvenne, Cleeremans, & Laloyaux, 2010; Gajewski & Brockmole, 2006; Johnson, Hollingworth, & Luck, 2008; Makovski, Shim, & Jiang, 2006; Yeh, Cheng-Ta, & Yu-Chin, 2005; although see Fougnie & Marois, 2009). Evidence in support of this proposal has been generated through dual-task studies that involve performing an attentiondemanding secondary task such as visual search (Johnson et al., 2008), letter judgments (Yeh et al., 2005), or backward counting (Allen et al., 2006) during the period between memory encoding and test (i.e. working memory maintenance; although see Fougnie & Marois, 2009). By demonstrating that no greater cost to memory was incurred for two-feature objects (e.g. colored-shapes) than single features (color-only) during the suspension of attention, these researchers have concluded that sustained attention is not necessary to maintain feature bindings after information has been consolidated into VWM. The freedom to deploy attention to ongoing perceptual input during visual working memory (VWM) maintenance may reflect the flexibility of the system to stay aware of relevant environmental information while information is active in working memory. Further, it may afford the opportunity to integrate information across eye movements (Irwin, Zacks, & Brown, 1990) and during scene formation (Hollingworth, 2005; c.f. Johnson et al., 2008). However, as 2 the representational space of VWM is capacity-limited (Cowan et al., 2005; Luck & Vogel, 1997; Vogel, Woodman, & Luck, 2001) the question of what happens to active representations when attention is deployed to new information remains an area of active study (Allen et al., 2006; Alvarez & Thompson, 2009; Fiacconi & Milliken, 2012b, 2012c; Ueno, Allen, Baddeley, Hitch, & Saito, 2011; Ueno, Mate, Allen, Hitch, & Baddeley, 2011; Wheeler & Treisman, 2002). The specific capacity limitations of VWM are still widely debated (limited by the number of objects versus feature detail in Alvarez & Cavanagh, 2004; Awh, Barton, & Vogel, 2007; Barton, Ester, & Awh, 2009; Bays, Catalao, & Husain, 2009; Vogel et al., 2001). However, for objects defined by only a few visual features (i.e. ‘simple’ visual objects in Alvarez & Cavanagh, 2004; Awh et al., 2007; Xu & Chun, 2006) the representational space displays characteristics indicative of an object-level representational system (Vogel et al., 2001; Zhang & Luck, 2008, 2011). In other words, the capacity of VWM for two-feature objects appears limited to approximately 3-4 visual objects regardless of whether the objects are defined by one or two features (Delvenne & Bruyer, 2004; Luck & Vogel, 1997; Olson & Jiang, 2002). Under low demand (i.e. less than capacity) new single-feature objects may be appended to the contents of VWM during maintenance without cost to existing active representations (Woodman & Vogel, 2005). However, recent research suggests that multi-feature objects may not be as robust as single features when VWM is appended with new information (Allen et al., 2006; Ueno, Allen et al., 2011). In a sense, research in this arena has concluded that bindings in VWM may not be as durable as single feature representation. In a study of VWM for multi-feature objects, Allen and colleagues (2006) observed that backward counting could be performed during VWM maintenance for two-feature objects 3 (colored-shapes) without a loss of object bindings. While they took this as evidence that attention does not need to be sustained during maintenance, in the same study, they additionally observed a memory advantage for study arrays that were presented simultaneously rather than sequentially (Allen et al., 2006). Further, during sequential presentation they observed an advantage for the final object presented over earlier objects in the array (i.e. a recency effect). They speculated that the advantage may be related to a specific vulnerability of bound object representations, e.g. a tendency for later objects in sequential arrays to retroactively disrupt memory for earlier encoded objects (Allen et al., 2006). However, it was unclear whether this effect was limited to the active updating of VWM with each object, or could be explained by the suspension of attention to memory during study of each object. In order to evaluate whether interference was due to encoding or the suspension of attention, Ueno and colleagues (2011) recently adapted the simultaneous array change detection task of Allen and colleagues (2006) with a distracter manipulation (visual suffix procedure adapted from (Hitch, 1975). Ueno and colleagues (2011) had participants study four two-feature visual objects on every trial, briefly followed (250ms) by the presentation of a to-be-ignored visual suffix (a single two-feature object). Across several experiments they observed that suffixes constructed from features never studied (e.g. unique color and shape never used to generate the study stimuli; ‘implausible’ suffixes) could be efficiently ignored. However, in an experiment where objects constructed from the same parent set of shapes and colors used throughout the task (e.g. ‘plausible’ suffixes) a specific disruption of memory for multi-feature objects was observed (Ueno, Allen et al., 2011). Further, the results of a cued-recall follow-up study indicated that 4 the features of plausible suffixes were more likely to be retrieved in error than implausible suffixes, an indication that they may have been appended to the contents of VWM (Ueno, Mate et al., 2011). The fact that objects containing ‘study-related’ features were unable to be efficiently ignored, and were potentially appended to the contents of VWM, suggested to Ueno and colleagues that the disruption effect was related to a failure of top-down attentional set (what they called a ‘general executive’ process). That is, while attention may have been prepared to gate visual suffixes not containing task relevant features, objects containing features encoded into VWM as part of study objects across the task were allowed access to representation. Thus disruption, may in effect, be related to appending VWM with a new multi-feature visual object regardless of whether it was voluntarily (Allen et al., 2006) or involuntarily (Ueno, Mate et al., 2011) gated into memory. The results of Ueno and colleagues (2011) are among the few pieces of evidence indicative of a vulnerability of VWM for bound features to be disrupted during maintenance (also see Fougnie & Marois, 2009 for evidence from multiple object tracking). However, a key element of the general study design used to test the sustained attention hypothesis warrants careful consideration in order to accept the disruption effect as evidence for fragile feature binding. That is, the critical test between the single- and multi-feature object conditions is based on two potentially different judgments. The hypothesis being tested by Ueno and colleagues, as well as others (Delvenne et al., 2010; Johnson et al., 2008; Yeh et al., 2005), has been that VWM for bound features is just as robust as VWM for single features. The prediction that the alternative hypothesis makes, that bound features are more fragile than single features, requires demonstrating that whatever experimental manipulation is employed causes 5 a greater cost to memory for multi-feature as compared to single feature objects. This hypothesis has been tested in each of the aforementioned studies through an adaptation of the now benchmark VWM change detection task of Luck & Vogel (Luck & Vogel, 1997). The change detection task involves studying a set of single or multi-feature objects, and then after a brief delay judging whether a test display contains the same features as during study or contains a new feature (e.g. a new color in the test display). Performance on this task has been shown to reliably correlate with other measures of working memory capacity (Cowan et al., 2005; Shipstead, Redick, Hicks, & Engle, 2012) and represents a promising avenue for studying individual differences in the top-down control of attention (Fukuda & Vogel, 2009b; Vogel & Machizawa, 2004; Vogel, McCollough, & Machizawa, 2005). However, it is important to point out that the specific adaptations that have been made to the task to compare memory for single versus multi-feature objects may contain a minor confound that limits the explanatory power of any evidence for the alternative hypothesis. Memory for single and multi-feature objects in each of these studies has been tested by asking participants to memorize the color, shape, or combination of color and shape present in each of four multi-feature study objects (usually varied across blocks of the task). At test for single features (color or shape), a single object is presented and the participant is asked to judge whether the object contains, for example, a color memorized at study or a new color altogether (i.e. old-new judgment). At test for multi-feature objects the participant is asked to judge whether the specific combination of shape and color present in the test object is the same or changed across objects from the study array (i.e. binding change judgment). While we are unaware of any study that has directly evaluated whether these two types of judgments are 6 subserved by fundamentally different memory retrieval processes in VWM (e.g. see familiarity versus recollection memory; Yonelinas, Zhang, & Shapiro, 2012), it is important to point out that the features present in the test object vary across conditions. In the memory for single features condition the test object contains a new feature that was not studied on that trial, where in the multi-feature condition only features from the study array are present in either a consistent or inconsistent configuration (paired-association) from study. On the surface at least, these two judgments involve two different questions: 1) was this feature studied? & 2) were these features studied as part of the same object? If the loss of performance Ueno and colleagues observed for binding change decisions in the plausible suffix condition is a result of adding additional information to VWM, then it would be important to control for the possibility that adding information to VWM may have an unbalanced effect on binding change judgments. For example, the old-new judgment may only require a sense of familiarity that a color was not studied on that trial and the binding change judgment may require a more explicit recollection of which features were paired together at study. Accordingly, the result of increasing the number of active representations in memory may multiplicatively increase the search space for binding change decisions (e.g. from all combinations of 4 colors and 4 shapes to 5 and 5) while only increasing the search space for single feature by one feature (e.g. from 4 to 5 colors). While again, this is only a speculation at this point the repeated demonstration that binding change judgments are more difficult overall than old-new decisions for single features leads us to believe that this task design may affect binding change decisions in an unbalanced manner (Allen et al., 2006; Delvenne et al., 2010; Gajewski & Brockmole, 2006; Johnson et al., 2008; Ueno, Allen et al., 2011; Wheeler & 7 Treisman, 2002). If this were the case, the current methodology would not allow a differential effect on binding change judgments to be dissociated from a vulnerability of bindings represented in VWM. Therefore, we propose that in view of elucidating the mechanisms that lead to a loss of binding change detection performance, the conditions under which binding change detection performance is disrupted should be studied further. The results of Ueno and colleagues (2011a/2011b) do suggest that the active contents of VWM are modified through exposure to a visual suffix during maintenance. Further, their study has demonstrated that binding change detection performance is disrupted specifically when visual information presented after encoding contains visual features studied on trials throughout the task (e.g. plausible suffixes). At minimum these observations suggest that there are certain varieties of ongoing visual sensory input that may reduce the ability to make a binding change judgment about study features. In the following chapters we sought to further evaluate the conditions under which ongoing visual sensory input is disruptive of binding change detection, and begin to describe the mechanisms that lead to disruption of VWM for bound features. Based on anecdotal evidence from several recent studies reviewed below we expected that beyond distracter objects constructed of non-overlapping study features (‘related’ objects in the following chapters), distracter objects containing feature overlap with the study set (‘matched’ objects) would be equally if not more likely to disrupt binding change detection performance. By varying the information presented during VWM maintenance for bound features within subjects while preserving the same judgment across conditions, we sought to evaluate whether binding change detection could be variably disrupted and whether 8 the boundaries of disruption would help inform the mechanisms through which disruption occurs. In a study of VWM, Wheeler & Triesman (2002) observed a ‘single-probe advantage’ for detecting binding changes among objects studied in a visual array. In their VWM binding task they asked participants to memorize the color and shape of multiple visual objects simultaneously presented in a visual array. At test they presented either a single object containing a single color and shape from the objects encoded in memory (i.e. single-probe test) or displayed every object from the array (i.e. whole-display test). At test participants were asked to determine whether the objects present contained the same pairing of features as during study or whether the features had changed across objects (i.e. feature binding swap). While it was unclear whether a direct statistical test was conducted to compare across experiments (c.f. Johnson et al., 2008) an advantage was observed for detection of binding swaps during their single-probe experiment as compared with the whole-display experiment (although see Johnson for failure to replicate in a within subjects design). Wheeler & Triesman initially interpreted this effect as reflecting the necessity to expend a greater amount of attention on the whole-display test arrays than the single-probe (Wheeler & Treisman, 2002). As previously mentioned, dual task studies directly manipulating the sustained attention hypothesis have all but ruled out this interpretation (Allen et al., 2006; Delvenne et al., 2010; Gajewski & Brockmole, 2006; Johnson et al., 2008). However, the results of the following recent studies suggest that this effect may likely be driven by interference caused by perceiving a reassociation of active features in VWM during the whole-display test (i.e. new color-location bindings). 9 In a study by Alvarez & Cavanaugh (2009) participants monitored the location of four colored circles as they rotated around the center of a dynamic display. These circles were visible and then occluded for 250ms at a time as they passed behind masked regions of the display. On each trial participants monitored these circles for a change in location (a swap in the location of two of the circles) for the final 250ms that these objects were visible on screen before terminating behind one of the four masked regions of the display. While participants were highly sensitive to these feature swaps (~75% detection rate), an analysis of free recall on miss trials indicated that participants reported the swapped-in color for 80% of trials in which they were unaware of a change in location. Given the probed color had only been visible for the last 250ms of the 2-4s period that these circles rotated about the screen, and participants failed to acknowledge the location swap for these trials, Alvarez and colleagues concluded that binding information (in this case color-location) may be disrupted rapidly and potentially outside of awareness (Alvarez & Thompson, 2009). Another more recent set of studies by Vaquero, Fiacconi and colleagues further suggests that perception of information containing partial feature-overlap with the contents of VWM may disrupt memory and hold other significant implications for the phenomenology of explicit memory (Fiacconi & Milliken, 2012a, 2012b, 2012c; Vaquero, Fiacconi, & Milliken, 2012). In their studies, participants performed a simple identification task in which the goal was to locate a target “O” that appeared in one of four designated locations on the screen (see also Kahneman, Treisman, & Gibbs, 1992). In one instantiation of their experiments, the test array was preceded by a priming array in which the target letter appeared either in the same location as at test or in the location of the non-target item. For example, the O’s eventual location was 10 either primed for repetition (“identity-match”), or was occupied by the non-target X (“identitymismatch”). In addition to slowing of RT for target identification, mismatch trials produced another important effect in that 12 of the 14 participants failed to recognize that a strong contingency (75%) for the target O to be predicted by the non-target item’s location in the priming array (e.g. X). When the procedure was reversed and the strong contingency was placed on the location-match trials (e.g. the target O appeared in the location primed by the priming array), 11 of the 12 participants reported acknowledging the contingency (Vaquero et al., 2012). In a series of follow-up experiments aimed to explain the source of this contingency blindness, Fiacconi and colleagues discovered that participants may have a profound loss of memory for the priming array, performing at chance during recall of the location of the O in the priming array (Fiacconi & Milliken, 2012c). They conducted the same type of prime-test experiment with a follow-up on each trial asking participants to recall where the X or O was located in the prime array. On identity-mismatch trials, where the X or O occupied one another’s location between prime and test arrays, recall performance was near chance for the initial priming array. They interpreted this observation as evidence that the source of contingency blindness observed in their earlier experiments may have been the rapid disruption of VWM for feature binding. While these studies (Alvarez & Thompson, 2009; Fiacconi & Milliken, 2012c) have demonstrated that exposure to new information that contains partial feature-overlap with the contents of VWM may disrupt earlier encoded bindings, the disruption observed in these studies was limited to a loss of explicit memory for the binding between an object and its 11 studied location in space. By including a partial feature-overlap distracter condition in the current set of studies we sought to evaluate whether visual sensory input containing partial feature-overlap would have the same effect on binding change detection. In Chapter 2 (Experiment 1), we present an adaptation of the suffix paradigm termed the “Binding Distraction Task” (BDT) where we sought to further investigate disruption of VWM for bound features. In addition to plausible (“related” in Chapters 2 and 3) distracter objects, we included a feature-matched distracter condition (“matched” in Chapters 2 and 3). Additionally, in order to begin to examine the mechanisms through which visual sensory input disrupts binding change detection, participants in Experiment 2 of Chapter 2 also completed three standard measures of working memory capacity. Ueno and colleagues (2011) proposed that the failure to ignore plausible suffixes may map onto a general executive process, for instance an inability to properly set top-down attention. While working memory capacity has been shown to be a reliable correlate of the control of attention during working memory encoding (“working memory capacity” in Engle, 2002; Kane & Engle, 2003; Kane, Poole, Tuholski, & Engle, 2006; McNab & Klingberg, 2008; Vogel et al., 2005), the relationship between working memory capacity and robustness to interference during VWM maintenance has not been studied. Through an individual differences approach in Experiment 2 we evaluated whether individual differences in the executive control of attention could account for disruption of binding change detection performance. The results of Chapter 2 indicated that related and matched distracters are both disruptive of VWM for bound features when presented shortly after the offset of the encoding array. However, while only feature-matched distracters were specifically disruptive when 12 presented later during VWM maintenance, disruption was unrelated to measures of working memory capacity. A growing body of research suggests that the contents of VWM may automatically guide attention to new visual information that is related to information active in working memory (see Soto, Hodsoll, Rotshtein, & Humphreys, 2008; Soto & Humphreys, 2009 for review). The pattern of results observed in Chapter 2 led us to speculate that VWM for bound features is not disrupted through a failure to properly set attention. Rather we propose that disruption may occur as a result of the capture of attention triggered by a correspondence between currently (matched) and recently (related) active VWM traces and ongoing visual sensory input. We speculated that the resources responsible for signaling for a shift of attention to objects with task relevant features may be harnessed by related and matched distracters and cause a disruption of VWM maintenance. To follow-up on this hypothesis, in Chapter 3 we present the results of an fMRI study designed to map disruption of VWM for bound features onto neural activity within regions of the brain associated the voluntary and stimulus-driven control of attention (e.g. Corbetta, Patel, & Shulman, 2008). 13 CHAPTER 2 DISRUPTION OF BINDING CHANGE DETECTION The bindings between features of multi-feature objects represented in visual working memory (VWM) are surprisingly robust to the suspension of attention (Allen et al., 2006; Delvenne et al., 2010; Gajewski & Brockmole, 2006; Johnson et al., 2008; Makovski et al., 2006; Yeh et al., 2005; although see Fougnie & Marois, 2009). The observation that multi-feature objects are as robust to the suspension of attention during VWM maintenance as single-feature objects has been taken as key evidence that bound VWM representations are not as fragile as once suspected (Kahneman et al., 1992; Treisman, 1996; Wheeler & Treisman, 2002; J. M. Wolfe & Bennett, 1997). That is, they are self-sustaining in the absence of focused attention. However, recent work by Ueno and colleagues (2011) has suggested that the ability to detect changes in bindings between multi-feature objects (“binding change detection”) may be specifically disrupted after exposure to to-be-ignored visual objects containing study-related features. For example, Ueno and colleagues recently demonstrated between subjects that binding change detection performance is more greatly reduced following exposure to distracter objects constructed from the same parent set of features used to generate study objects (“plausible” visual suffixes) than after exposure to visual objects that contain features never studied throughout the task (“implausible” suffixes in Ueno, Allen et al., 2011). They described their ‘plausible’ suffix effect as related to a general executive process (e.g. Baddeley, 1984). Specifically, they argued that disruption represented a failure to gate out to-be-ignored information from VWM representation (Ueno, Allen et al., 2011), and provided cued recall 14 evidence that the features of plausible suffixes may have indeed become active in VWM (Ueno, Mate et al., 2011). We proposed in the last chapter that due to a potential confound between memory type (single or bound features) and the judgment made at test (old-new versus binding change), it may be too early to conclude that bound features are more fragile than memory for single features. Rather, at minimum the results of Ueno and colleagues suggest that objects constructed from study-related features (“related” objects in the current study) lead to a significant and specific loss of the ability to detect binding changes among study objects. While this may be an indication that bound features are fragile, their observation warrants further attention in order to understand the factors that lead to a loss of binding change detection performance. In a between subjects design, Ueno and colleagues demonstrated that objects containing study-related features were more disruptive of binding change detection performance than objects containing irrelevant features. As a first step to replicate and extend the findings of Ueno and colleagues in Experiment 1 we compared binding change detection performance within subjects across several distraction conditions. In order to replicate the ‘plausible’ suffix effect in our design we compared performance after exposure to a distraction event containing no study-related features (“empty” distracter arrays) with a distracter array containing objects constructed from the same parent set of features as the study objects (“related” distracter arrays). In addition, we also evaluated binding change detection performance when the distracter array contained objects sharing partial-feature overlap with the study array (“matched” distracter arrays). Objects in the matched distracter array matched the color of each study object, but contained new shapes. As was discussed in Chapter 1, 15 evidence from several recent studies suggests that perceiving a feature maintained in VWM associated with a new visual feature may, in some cases profoundly (Fiacconi & Milliken, 2012c), affect memory for the shared feature’s original association (Alvarez & Thompson, 2009; Fiacconi & Milliken, 2012c; Wheeler & Treisman, 2002). As participants were instructed to attend to displays where these reassociations occurred in each of these studies (e.g. detect when features changed in (Alvarez & Thompson, 2009), it is unclear whether feature-matched information needs to be voluntarily attended in order for it to disrupt memory. We predicted that if the quality of being ‘study related’ was the key feature that caused plausible suffixes to disrupt performance in Ueno and colleagues (2011) then matched objects should be equally if not more disruptive than related objects in Experiment 1. In view of replicating the results of Ueno and colleagues in our modified design, in Experiment 1 we presented the distracter array 250ms after the offset of the study array. The choice of a 250ms interval between the offset of the study array and presentation of the distracter object (i.e. visual suffix) in Ueno and colleagues (2011) was not arbitrary as it was intended to match the sequential object-to-object presentation rate in Allen and colleagues (2006; see Chapter 1). However, while the to-be-ignored plausible suffixes in Ueno and colleagues produced a similar interference effect in the absence of voluntary encoding, we suspected that the influence of plausible suffixes over binding change detection at such a presentation interval may not be trivial. Recent work on VWM consolidation suggests that while consolidation of some single feature objects (e.g. objects defined only by their color) may occur at a rate of 50ms per object (Vogel et al., 2006), more recent evidence suggests that multi-feature visual objects are rate-limited by the slowest consolidating feature, with visual 16 shapes consolidating at a rate of ~200ms per object (Woodman & Vogel, 2008). Additionally, other evidence suggests that VWM consolidation may not occur massively in parallel (Mance, Becker, & Liu, 2012). With four multi-feature objects being studied on every trial and a 1200ms encoding period, we suspected disruption at this interval could be related to interference with VWM stabilization rather than disruption of fragile bindings. For example, if all information from the study array is not consolidated by the offset of the study array, disruption could be related to interference with pre-stabilized memory traces (i.e. backward masking). Alternatively, the onset of new visual information at 250ms could prevent freshly encoded VWM traces from being attentively refreshed (i.e. redirect attention before objects are visually rehearsed for the first time after the offset of the study array; Chun, 2011). Accordingly, in order to provide a stronger test of the ability for ongoing visual input to disrupt binding change detection in Experiment 2 we presented distracter arrays 1300ms after the offset of the study array. In addition, in an attempt to identify the mechanism responsible for the decrement in binding change detection performance observed during distraction, in Experiment 2 we had BDT participants complete three measures of working memory capacity. Ueno and colleagues (2011) speculated that the plausible suffix effect was related to a general executive process. Specifically, they argued that disruption occurs through a failure to gate plausible suffixes from VWM representation. The construct of an executive attentional gate or ‘filter’ has been an important aspect of models of attention (Vogel et al., 2005) and working memory capacity (WMC as “Executive Attention” in Engle, 2002) for some time. For example, the ability to maintain the contents of VWM while properly gating/filtering out irrelevant information has 17 been targeted as the key construct that separates high from low working memory capacity individuals (see “Executive Attention” in Engle, 2002). Poole and Kane (2009) characterize the ability to constrain (limit the scope of what is being attended) or restrain (prevent the deployment of attention) attention as the key executive mechanisms that support efficient use of working memory resources (Poole & Kane, 2009). The relationship between complex span task performance (e.g. operation span) and attentional filtering tasks such as the Erikson flanker, Stroop color, and anti-saccade tasks have served as the primary external criteria for such a conclusion. Specifically in the VWM domain, it has been argued that individuals with lower VWM spans (as measured through the single feature change detection task) may fail to gate irrelevant information during VWM encoding and maintenance (Vogel et al., 2005). For example, individuals with lower VWM span show increased storage-related neural activity after studying an array containing to-be-remembered and to-be-ignored objects (EEG activity indicating storage of irrelevant information; Vogel et al., 2005) and slower release from attention-capture to distracters (Fukuda & Vogel, 2009b) as compared with high span individuals. In Experiment 2, we predicted that if the decrement in binding change detection performance was related to an executive gating construct, then individuals with high WMC should be less prone to disruption than low WMC individuals. Experiment 1 – Distracter Array Presented 250ms after Study Array Methods Participants: Fifty (Age: M(SD) = 19.8(1.81); 30 Female, 20 Male; handedness (R/L) = 47/3) Michigan State University undergraduate students participated in the current study for course 18 credit. All materials were approved by the Michigan State Office for Protection of Human Subjects. Binding Distraction Task: Stimulus presentation was conducted on a Dell Optiplex desktop computer and generated by E-Prime software. All stimuli were displayed on a 19” computer monitor (measured diagonally) positioned approximately 24” away from the participant, and responses were made on a standard QWERTY keyboard using numeric pad “1” (same) and “2” (changed). All stimuli were presented on a neutral gray background. Each trial began with the onset of a fixation cross to signal the beginning of a trial (300ms) immediately followed by the onset of four unique colored-shapes presented around the corners of an invisible square centered on the screen for 1200ms (“study array”). The invisible square occupied approximately 11 x 11 degrees of visual angle (left-right and top-bottom), with each visual object measuring 3.7 x 3.7 degrees of visual angle. After a 250ms fixation delay one of four distraction events was presented: 1) no distraction; 2) empty array; 3) related object array; & 4) matched object array. In the no distraction condition 1 additional second of fixation was included to match the duration of the other three conditions. In the empty array condition empty boxes drawn in 5pt black outline surrounding the location of study array objects were presented. In the related object condition the same black outline array was filled with colored shapes generated from the same parent set of features as the study array but not matching any object in the study array on shape or color. In the matched array the black outline box array was filled with colored shapes generated from the same parent set of shapes as the study array (not matching in shape) but matching the study array objects in color (each matched distracter 19 matched 1 object from the study array in color). In the matched condition, color-matched objects were never displayed in the same location as the corresponding object of the memory set. Distracters were present on the screen for a minimum of 1 second after which point the fixation cross grew from 14pt font to 24pt font. After the fixation cross expanded in size, participants made a press on the SPACE bar which cleared the distracters from the screen and returned the fixation cross to standard 14pt font size. The expansion of the central fixation manipulation was included to ensure that all participants were attending the center of the screen with their eyes open. The trial would not continue until a SPACE response was made. After the distracters were cleared, another fixation delay was presented for 2250ms followed by the central presentation of the test object (presented directly over the fixation cross, center screen). The test object always included one shape and one color present in the study array. At test, participants judged whether the combination of shape and color in the test object was the ‘same’ as one of the objects at study or ‘changed’ across objects from study regardless of its location in the study array(test object = 50% probability of same or changed). Participants were given as much time as necessary to make a response to the test object, and instructed to emphasize accuracy over speed. After a response, the test object was cleared and a solid gray background was presented for 2000ms before the onset of the next trial. Eight colors and eight shapes constituted the parent set from which all stimuli were generated including: white, black, 20 yellow, cyan, blue, red, magenta, and green; circle, diamond, teardrop, tall rectangle, square, heart, triangle, and star (see Figure 1). Prior to participation in the Binding Distraction Task, participants were provided instructions and twenty practice trials with onscreen trial by trial feedback (correct/incorrect) to acquaint them with the experimental procedures. Participants were instructed that the experiment was a test of visual memory and were provided with a visual depiction of each trial condition. The instructions included an emphasis on memorizing the combination of shape and color present in each study object, and that the test object would include one shape and one color that were BOTH present in the study set. Participants were instructed to press SPACE as quickly as possible after the fixation cross expanded as to remove the distracters from the screen. Participants completed 40 trials per condition (20 change and 20 no-change) for a total of 160 trials. The experiment was run in 5 blocks of 32 trials with a self-paced break between each block (totaling approximately 45mins including practice). 21 Figure 1. Study Design. (TOP) Example of each color and shape used to construct study objects (MIDDLE) Timecourse of each trial (BOTTOM) Example of each distraction condition. In Experiment 1 the pre-distracter fixation lasted 250ms and post-distracter fixation lasted 2250ms. In Experiment 2 the pre-distracter fixation lasted 1300ms and the post-distracter fixation lasted 1200ms. For interpretation of the references to color in this and all other figures, the reader is referred to the electronic version of this dissertation. Results Corrected Recognition (hit-rate minus false alarm-rate): Twelve of the fifty participants were excluded from the final analysis due to poor performance on the no distraction condition (≤ 20% corrected recognition). Participants failing to reach 25% corrected recognition rate without distraction fell within one confidence interval of chance performance and were removed from further analysis (95% confidence interval around 0% corrected recognition rate). Corrected recognition for each condition was subjected to a 1 x 4 repeated measures ANOVA where a significant main-effect was observed (F(3,111) = 9.11, MSE = .17, p < 0.001). To follow up on the shape of this main-effect, planned comparisons (paired t-tests) were conducted between empty vs no distraction, related vs empty, and matched vs related. The results of these three planned comparisons revealed a significant distraction cost associated with the 22 empty distracters compared to no distraction (t(37) = 2.72, p = 0.01), a significant cost of related distracters compared with empty distracters (t(37) = 2.38, p = 0.02), and no difference between matched and related distracters (t(37) = 0, p = 1.0; see Figure 2). Figure 2. Corrected Recognition Rates and RT for Experiment 1. A significant reduction in accuracy was observed between no distraction and empty as well as empty and related. No difference was observed between related and matched. Key: N = No Distraction, E = Empty, R = Related, M = Matched. Response-time (RT) Analysis: 1 x 4 repeated measures ANOVAs were conducted on RT for both the distracter array and the test object. Exposure to Distracters: A significant main-effect was observed for RT during the distracter array (F(3,11) = 2.69, MSE = 162835, p = 0.05). Planned comparisons revealed that this maineffect was driven by a significant reduction in RT between the no distraction and empty conditions (t(37) = 2.5, p < 0.05), however no increase in RT was observed between empty and related (t(37) = 0.57, p = 0.57), or between related and matched (t(37) = 0.21, p = 0.84). The reduction in RT between no distraction and empty could likely be explained by the distracter 23 array for empty acting as an alerting signal (Posner, Klein, Summers, & Buggie, 1973), as the fixation increase always occurred 1000ms following the onset of the array. Without the onset of a distracter array during no distraction, RT to the fixation increase for the no distraction condition was at a disadvantage. RT for Test Objects: No significant main-effect was observed in the 1 x 4 repeated measures ANOVA for RT during the test object (F(3,111) = 0.14, MSE = 6805, p = 0.93; for all RT results see Figure 3). Figure 3. RT for Distracter Array and Test Object for Experiment 2. (LEFT) RT for the distracter array. (RIGHT) RT for the test object. Interim Discussion In Experiment 1, the onset of an empty array resulted in a reliable cost to binding change detection performance. The observation that task irrelevant information (empty array) impinged upon performance is consistent with both the effect of implausible suffixes in Ueno, et al (2011) and the various attention suspension manipulations discussed above (Allen et al., 2006; Delvenne et al., 2010; Gajewski & Brockmole, 2006; Johnson et al., 2008; Ueno, Allen et 24 al., 2011; Yeh et al., 2005). A common theme among studies investigating the role of attention in maintaining bound features in VWM is that occupying attention voluntarily (e.g. through visual search in Johnson et al., 2008) results in a small but reliable decrement to performance for single-feature and binding change detection. However, the current observation is among only a few pieces of evidence that directing attention away from VWM maintenance involuntarily results in a cost to change detection (e.g. see implausible suffixes in Ueno, Allen et al., 2011). Further, the results of Experiment 1 replicate and extend the plausible suffix effect to feature-matched objects and generalize the effect to a modified experimental design. Both related and matched objects caused a significant reduction in memory for bound features as compared with the cost associated with the empty array. No difference was observed between related and matched conditions suggesting that feature-matched distracters are equally as disruptive to binding change detection performance as study-related objects containing no feature-overlap. The increase in cost to performance between empty and filled arrays (both related and matched objects) suggests that the content of ongoing visual sensory input during VWM maintenance may selectively impinge upon binding change detection. However, the lack of a difference in cost between related and matched objects prevents making any further inference about the mechanisms through which these objects disrupt performance. In essence, the only conclusion that can be drawn from the results of Experiment 1 is that visual sensory input containing study-related features is more difficult to ignore than input lacking such features (e.g. empty arrays). 25 Experiment 2 – Distracter Array Presented 1300ms after Study Array As was discussed in the introduction, we suspected that the presentation of a distraction array in Experiment 1 may have coincided with ongoing VWM consolidation processing. If consolidation was incomplete within 250ms of the study array offset the filled distracter arrays (e.g. related and matched) may have acted as a backward mask to iconic traces or disrupted the ability to refresh recently stabilized traces. As a stronger test of the influence of ongoing visual input on arguably more stabilized and refreshed VWM traces in Experiment 2 we presented the distraction array 1300ms after the offset of the study array, at the mid-point of the duration of the maintenance interval. In addition, to begin evaluating the mechanisms that lead to disruption of binding change detection participants in Experiment 2 also completed three measures of working memory capacity. The addition of these WMC measures to the experimental session allowed us to test the hypothesis that disruption of binding change detection is related to the executive control of attention (e.g. Engle, 2002; Vogel et al., 2005) Specifically, we were able to test the prediction that high WMC individuals would show a greater resistance to disruption than low WMC individuals. Methods Participants: One hundred and fifteen (Age: M(SD) = 19.5(2.97); 83 Female, 32 Male; handedness (R/L) = 108/7) Michigan State University undergraduate students participated in the current study for course credit. All materials were approved by the Michigan State Office for Protection of Human Subjects. 26 Binding Distraction Task: All parameters of Experiment 2 were identical to Experiment 1 with the exception of the onset time of the distracter array during maintenance. In Experiment 2 the distracter array was presented 1300ms after the offset of the study array. Critically, in order to equate the total length of the maintenance interval between study and test the postdistraction fixation was reduced from 2250ms in Experiment 1 to 1200ms in Experiment 2. Visual Array Task: The Visual Array Task (i.e. “change detection task”) was adapted directly from Luck & Vogel, 1997). On every trial a study array of 3 or 6 colored squares was displayed for 100ms, followed by a 900ms delay period, and a single-probe test object for 2000ms. Each study object was a colored square approximately 2 x 2 degrees of visual angle displayed at random locations about the display screen, separated by at least 2 degrees of visual angle. Each square was a unique color pulled from a set of 7 potential colors: blue, red, magenta, cyan, green, black, or white. The test object at the end of each trial was a single colored square occupying the location of one of the study objects. With 50% probability the test object was either the same or different color than the object present at the same location during study. The participant’s job on each trial was to indicate whether the color of the single test object was the same (numeric pad “1”) or changed (numeric pad “2”) as compared with the object present in that location at study. Visual Array Task performance was scored using Cowan’s k (Rouder, Morey, Morey, & Cowan, 2011). To obtain each individual’s k, the Hit-rate for each set-size was penalized by the False Alarm-rate and multiplied by the set-size for that condition. 27 The k obtained through this formula (S(HIT-FA) as described in (Cowan et al., 2005; Rouder et al., 2011) was then averaged across set-sizes. Automated Complex Span Measures: Both Complex Span Tasks administered were selfcontained (practice and instructions) and automated. A full description of the development, testing parameters, and reliability of the automated versions of these tasks can be found elsewhere (Redick & Thomas, 2012). A brief description of each task can be found below. Automated Operation Span (verbal complex span): The operation span task involved memorizing letters while making arithmetic judgments between the presentations of each letter. On each trial, a participant was presented a single letter followed by an example equation (e.g. 4 + (1-3) = 3, True/False). After the presentation of up to 5 letters (and solving 5 interleaved arithmetic judgments) the participant was asked to recall, in order, each of the letters memorized on that trial. Automated Symmetry Span (spatial complex span): The symmetry span task involved memorizing visual locations while making symmetry judgments between the presentations of each location. On each trial, a participant is presented with a red square located in a white-colored 5x5 matrix of squares followed by a visual pattern that they were to judge as symmetrical/non-symmetrical. After the presentation of up to 5 locations and symmetry judgments the participant was asked to recall, in order, each of the locations that were memorized in the square matrix. 28 Scores for each Complex Span Task were calculated according to the proportional scoring method (see (Kane et al., 2006)). Participants received credit for each item reported in the correct serial position within each trial. In other words, a participant could add two points to their span score for recalling 2 of the 5 letters in the correct serial position in a set-size 5 Operation Span trial without correctly recalling all of the letters from the list. This method of scoring has been shown to highly correlate with the full credit scoring method (i.e. participants only receive credit for a trial in which all items are recalled correctly) and results in less skewed group distributions as it reduces the weight of high set-size trials (see Kane et al., 2006). Span Scores therefore represented the total proportion of all correctly recalled items over all items in the task. It is important to note that scoring according to the full credit approach did not change the pattern of results reported here. Each participant’s Span Score for each task was then converted to a z-score based on the whole group distribution of span scores. A “Complex Span Score” was calculated by averaging performance across both Complex Span Tasks. This averaging was done in order to reduce the impact of task- and modality-specific variance, yielding a more general measure of working memory capacity similar to the factor approach commonly used in studies of Complex Span. 29 Procedures: Each participant completed the four tasks in a fixed order: Binding Distraction Task, Operation Span, Visual Array Task, & Symmetry Span. Prior to beginning the Operation Span Task participants were told that instructions and practice for the task were self-contained and to see the experimenter after completion. Prior to the visual array task, participants were given instructions, several examples of “same” and “changed” trials, and asked to verbally complete two example trials before beginning. Responses in the Visual Array Task were made in the same manner as the Binding Distraction Task (1 = same; 2 = changed). Participants completed 120 trials of the Visual Array Task (60 for set-size 3 and 60 for set-size 6; 30 change and 30 no-change for each set-size). After completing the Visual Array Task participants were given the same instructions for the Symmetry Span Task as they were the Operation Span Task. After completing the session each participant was provided a debriefing form explaining the aim of the study. Results Binding Distraction Task: Twenty-five of the one hundred and fifteen participants were excluded from the final analysis due to poor performance on the no distraction condition (a similar rate of exclusion as in Experiment 1; 24% in Experiment 1 and 22% in Experiment 2). 30 Corrected recognition: Corrected recognition rates for each condition were subjected to a 1 x 4 repeated measures ANOVA where a significant main-effect was observed (F(3,267) = 12.31, MSE = .25, p < 0.001). To follow up on the shape of this main-effect, planned comparisons (paired t-tests) were conducted between empty vs no distraction, related vs empty, and matched vs related. The results of these three planned comparisons revealed a significant distraction cost associated with the empty distracters compared to no distraction (t(89) = 3.51, p = 0.001), no difference between related and empty distracters (t(89) = 0.64, p = 0.52), and a significant cost of matched distracters compared with related distracters (t(89) = 2.24, p = 0.03; see Figure 4). Figure 4. Corrected Recognition Rates for Experiment 2. A significant reduction in accuracy was observed between no distraction and empty as well as between related and matched. No difference was observed between empty and related. Response-Time Analysis: One-way repeated measures ANOVAs were conducted on RT for both the distracter array and the test object. 31 Exposure to Distracter Array: A significant main-effect was observed for RT during the distracter array (F(3,267) = 8.13, MSE = 319034, p < 0.001). Planned comparisons revealed that this maineffect was driven by a significant reduction in RT between the no distraction and empty conditions (t(89) = 5.6, p < 0.001), a marginal increase in RT between empty and related (t(89) = 2.23, p = 0.06), and a significant increase in RT between related and matched (t(89) = 3.15, p = 0.002). In order to evaluate whether the increase in distracter array RT between conditions may was related to performance a correlation analysis was conducted between distracter array RT (i.e. distracter exposure time) and distraction-cost for each condition (loss in corrected recognition rate for each condition compared to no distraction). Importantly, no relationship was observed between RT at the distracter array and distraction-cost for any condition (all p > 0.5) suggesting that the degree of disruption was unrelated to exposure duration. RT for Test Objects: A significant main-effect was observed for RT during the test object display (F(3,267) = 3.33, MSE = 76261, p = 0.02). Planned comparisons revealed only a significant increase in RT during the matched condition as compared with the related condition (t(89) = 2.0, p < 0.05; for all RT results see Figure 5). 32 Figure 5. RT for Distraction Array and Test Object. (LEFT) RT for response to the increase in fixation during the distracter array. (RIGHT) RT for test object. Individual Differences Measures: One participant was removed from the individual differences analysis for not successfully completing either complex span task (scoring a zero on the operation span, and completing only one symmetry span trial correctly) leaving N = 89 in the final sample. A significant positive correlation was observed between the no distraction condition of the BDT and WMC (Complex Span Factor, r = .28, p < 0.05; and k, r = .21, p < 0.05). Additionally, while no relationship was observed between the loss of performance (i.e. ‘distraction-cost’) and WMC measures during the empty and related conditions, distraction-cost for the matched distraction condition was positively associated with Complex Span (see Table 1). Although this correlation was reliable, the direction of the relationship was the opposite of what the control of attention hypothesis would predict. As Complex Span increased, so did the degree of distraction in the BDT. We suspected this relationship may have reflected a floor artifact, with the performing participants performing closest to chance without distraction having the smallest distraction-cost. To follow up on this possibility we performed a mediansplit analysis on the no distraction condition of the BDT (MEDIAN = 45% corrected recognition 33 rate) and correlated matched distraction-cost with Complex Span for those lying above and below median performance without distraction. No significant relationship between Complex Span and matched distraction-cost was observed for either subgroup (below median, r = .05, p = .75; above median, r = .14, p = .4) indicating that the positive relationship between whole group Complex Span and matched distraction-cost is likely an artifact of floor performance. See Table 1 for a summary of all task and condition correlations, as well as the sample’s mean performance on each individual difference measure. 34 Table 1. Correlation Table of Binding Distraction Task and Individual Differences Measures Task/Condition 1 2 3 4 5 6 7 8 9 BDT – No distraction .47(.15) - - - - - - - - BDT – Empty .6* .4(.23) - - - - - - - BDT – Related .42* .5* .38(.2) - - - - - - BDT – Matched .32* .5* .56* .34(.2) - - - - - BDT – Empty-Cost .1 -.74* -.26* -.35* .07(.18) - - - - BDT – Related-Cost .36* -.03 -.7* -.33* .36* .08(.19) - - - BDT – Matched-Cost .43* -.03 -.23* -.73* .4* .57* .13(.21) - - VAT – k .21* .19 .10 .05 -.06 .06 .1 2.8(.6) - Complex Span .28* .07 .14 -.02 .16 .09 .23* .33* .51* NOTE: BDT = Binding Distraction Task; VAT = Visual Array Task; Mean and sd for each condition/task are located along the diagonal of the table, with the exception of Complex Scan which contains the cross-correlation between Complex Span Tasks. * = significant at p < 0.05. Values in BOLD represent the key cross-correlations between cost on the BDT and the individual differences measures. 35 Interim Discussion In Experiment 2 we replicate the results of Experiment 1 in that the sudden onset of an empty array is costly to binding change detection performance. However, we provide evidence that feature-matched objects are more disruptive than study-related objects when presented later during the maintenance interval. Further, we show that related objects are no more disruptive than an empty array. The results of our individual differences analysis failed to provide evidence that WMC is related to disruption of binding change detection. This observation in particular suggests that the cost associated with ongoing visual sensory input during VWM maintenance observed here and possibly elsewhere (Ueno, Allen et al., 2011; Ueno, Mate et al., 2011) may not be related to the executive control of attention. While a lack of a negative relationship between WMC and distraction-cost in the BDT, particularly while controlling for possible floor effects, amounts to a negative result the potential implications of these findings are discussed in more detail below. General Discussion The set of experiments presented here were conducted in order to further investigate the impact of ongoing visual sensory input on VWM maintenance for multi-feature objects. Research investigating the role of attention in maintaining object feature bindings in VWM has generally concluded that feature bindings are self-sustaining in the absence of attention (Allen et al., 2006; Delvenne et al., 2010; Gajewski & Brockmole, 2006; Makovski et al., 2006; Yeh et al., 2005). However, Ueno and colleagues recently demonstrated that the content of visual sensory input during maintenance is consequential to change detection performance 36 specifically for binding change judgments (Ueno, Allen et al., 2011). While we argue that their experimental methods may be insufficient to conclude that feature binding is fragile in VWM, we view their observation as a critical demonstration that binding change detection can be disrupted involuntarily during VWM maintenance. Accordingly, in the current study we further evaluated the boundary conditions and mechanisms that lead to disruption of binding change detection performance. In Experiments 1 and 2 we showed that presenting an empty array during VWM maintenance is capable of reliably disrupting binding change detection performance. This result is consistent with previous work demonstrating that suspending attention during VWM maintenance for single and multi-feature objects results in a minor cost to performance on change detection (Allen et al., 2006; Delvenne et al., 2010; Gajewski & Brockmole, 2006; Makovski et al., 2006; Ueno, Allen et al., 2011; Yeh et al., 2005). However, the majority of these studies required attention to be voluntarily sustained to a secondary task during the maintenance interval (e.g. Delvenne et al., 2010; Johnson et al., 2008). The cost to performance observed for empty arrays across both experiments suggest that the abrupt onset of task irrelevant information may direct attention away from maintenance in a stimulus-driven manner, for example by ‘capturing’ attention through its abrupt onset (Theeuwes, 1994). Such a capture effect is consistent with Lavie’s load theory and Kiyonaga and Egner’s shared resource theory of attention: the view that when working memory is under a high load, the ability to engage in early attention filtering is compromised (e.g. selection in Lavie, Hirst, de Fockert, & Viding, 2004; see also internal/external tradeoff in Kiyonaga & Egner, 2012). Moreover in support of this interpretation, while we did not vary the WM load across the studies presented 37 here, at a lower VWM load (3 objects) in another study we failed to find a reliable cost associated with the empty array condition (see Chapter 3). If capturing attention away from VWM maintenance through the sudden onset of an array containing no visual objects is enough to disrupt binding change detection, then the further decrement observed for related and matched objects may reflect an additional source of performance cost. In Experiment 1 we observed that both related and matched objects were more costly to binding change detection performance than the empty array. Critically, this observation replicates and extends the plausible suffix effect to a modified experimental design. However, we speculated that presenting filled arrays 250ms after the offset of the study array may have disrupted consolidation and/or the opportunity to engage in attentive refreshing (Chun, 2011). Specifically, while the outline of the empty arrays simply surrounded the location of the study objects’ positions, the filled arrays (related and matched) may have acted as a backward mask to the study objects reducing the time available to stabilize feature information from the study array. Alternatively, given consolidation may have occupied the entirety of the encoding interval, the presentation of new visual objects may have prevented the array from being attentively refreshed after the offset of the study array. While studies such as Vogel, et al (2006) suggest that VWM consolidation may occur as rapidly as 50ms per visual object, this exceedingly fast estimate was based on consolidation of objects defined by a single color (e.g. colored squares, Vogel et al., 2006). More recent research has suggested that multi-feature object consolidate at a speed that is rate limited by the slowest consolidating feature, for example shapes at ~210ms per object (Woodman & Vogel, 2008). Further, the number of objects that can be consolidated in parallel remains an open matter of debate, with 38 a recent study by Mance and colleagues (2012) suggesting that the upper-limit may be in the range of 1-2 single-feature objects (Mance et al., 2012; see also Huang, Treisman, & Pashler, 2007). If the consolidation process for multi-feature objects is indeed as limited or slow as these studies would indicate, it would be difficult to disentangle the fragility of bindings in VWM from disruption of the stabilization process (consolidation or attentive refreshing). After increasing the interval to 1300ms in Experiment 2, we observed that related objects were no longer more disruptive than the empty array. However, we observed that objects containing partial-feature overlap with the contents of VWM still contribute an additional cost to binding change detection. In order to try and characterize the mechanisms that lead to a failure to ignore distracter arrays we had participants in Experiment 2 complete three measures of WMC. As was discussed above, measures of WMC are highly related to the ability to constrain and restrain the allocation of attention (Poole & Kane, 2009; “executive attention” in Engle, 2002) and the ability to filter irrelevant information during VWM encoding (Vogel et al., 2005). Similarly, we expected that the ability to filter distracter arrays in our task during VWM maintenance would also be related to WMC. Surprisingly, we observed no relationship between distraction-cost and Complex Span or VAT k. Taken together these results limit the explanatory power of the hypothesis proposed by Ueno and colleagues (2011) that the plausible suffix effect is related to a ‘general executive’ gating process. First, if the effect was the result of related objects harnessing top-down attention set (i.e. inadvertently passing a selective filter) it is unclear why related objects would lose their ability to pass such a filter when presented later during maintenance. Second, given WMC has been shown to correlate with similar filtering abilities during WM encoding it is also unclear why this ability would be 39 lost during VWM maintenance. Given a lack of support for the general executive gating hypothesis, as will be discussed below, our results lead us to believe that disruption may occur through a stimulus-driven mechanism with matched objects causing a significant impairment of VWM for the study array (Alvarez & Thompson, 2009; Fiacconi & Milliken, 2012c; Huang & Pashler, 2007; Olivers, Meijer, & Theeuwes, 2006; Soto et al., 2008; Soto & Humphreys, 2009). It has been well documented that attention may be automatically guided to visual input that matches the active contents of VWM (Folk, Remington, & Johnston, 1992). This type of stimulus-driven capture of attention is referred to as ‘contingent capture’ (Folk et al., 1992; Folk, Remington, & Wright, 1994; Lavie et al., 2004; Theeuwes, 1994). Often contingent capture is studied in the context of visual search where irrelevant information matching the task goal or top-down set (e.g. search for a ‘blue’ object) draws attention and slows performance (Folk et al., 1992). However, recent research suggests that the active contents of VWM can guide attention (i.e. cause contingent capture) even when this content is not relevant to the goals of the task (e.g. color interference during visual search for orientation; (Huang & Pashler, 2007; Soto & Humphreys, 2009, see Soto et al., 2008) for review). This form of contingent capture is reasoned to emerge as a function of the contents of VWM forming a type of automatic ‘top-down’ bias to attend information containing feature overlap (Soto et al., 2008; Soto & Humphreys, 2009). Evidence for contingent capture in the current study is provided in Experiment 2 where only distracter arrays containing partial-feature overlap with the study array were more disruptive to performance than empty and related arrays. As was discussed in Chapter 1, several recent studies have provided evidence that when visual sensory input containing 40 feature-overlap with the contents of VWM is attended, the memory for an object and its original location in space (i.e. object-location binding) may be rapidly lost (Fiacconi & Milliken, 2012c) or rebound to a new location (Alvarez & Thompson, 2009). The results of Experiment 2 extend these findings to conditions where objects containing feature-overlap are instructed to be ignored. However, it is unclear at this point whether matched objects in Experiment 2 were disruptive due to a change in the location of study colors in the matched array (e.g. objectlocation binding) or a change in the color-shape binding (i.e. intra-object binding). There is evidence that rehearsal in VWM, i.e. attentive refreshing (Chun, 2011), is spatially-based (Yeh et al., 2005). In other words, objects are attended in memory in a manner that corresponds to their location at study. For example, providing a predictive spatial cue during maintenance improves performance for that location at test (Gajewski & Brockmole, 2006; Yeh et al., 2005). In the current study, objects in the matched array were always presented in locations that did not correspond to the common feature in memory (to equate the matched arrays with related arrays where each location in the array contained new feature). Thus, the current study does not allow us to adjudicate between whether matched objects disrupt binding change detection by contingently capturing attention to new spatial locations or through interference with intraobject feature bindings. However, in either case we would argue that the net result of attending an object containing feature overlap would be either a reassociation of features in memory (e.g. see also object-file theory in Kahneman et al., 1992) or an increase in set of active associations between features (i.e. increase in decision noise for binding change detection). The possibility that the effect observed in Experiment 2 may be driven by the contingent capture of attention raises another potential explanation for the effects observed in 41 Experiment 1. We suggested that the ability to interpret disruption at 250ms after the offset of the study array as evidence for fragile bindings is somewhat limited as at least two alternative explanations can be entertained (e.g. masking or interruption of initial rehearsal). However, the specificity for study-related objects to disrupt memory at this interval here and elsewhere (Ueno, Allen et al., 2011) suggests a common mechanism could account for the effects observed across both experiments. Various influential models of working memory representation view the phenomenology of short-term storage (i.e. what an individual can readily access in WM) as a collection of memory traces that are currently active in the ‘focus of attention’ (Cowan, 1999; Oberauer, Suss, Wilhelm, & Wittman, 2003; Unsworth, Spillers, & Brewer, 2012). Whether a single or multiple objects can be maintained in the focus of attention at any time remains a matter of debate (Cowan, 2001; Oberauer et al., 2003). However, these models find agreement in the theory that memory traces outside the current focus of attention can still affect behavior, even though they may be less explicitly retrievable. These traces are referred to as ‘active’ but outside the focus of attention in models such as Cowan’s ‘embedded process model of WM’ (Cowan, 1999). In some cases these memory traces affect behavior because they were recently in the focus of attention, an effect referred to as proactive interference (Craig, Berman, Jonides, & Lustig, 2013; Jonides & Nee, 2006; Kane & Engle, 2000; Lustig, May, & Hasher, 2001; May, Hasher, & Kane, 1999; Monsell, 1978; Nelson, Reuter-Lorenz, Persson, Sylvester, & Jonides, 2009). For example, recently active memory traces may intrude on subsequent performance for word list learning, a type of proactive interference referred to as the ‘recent-probes’ effect (Craig et al., 2013; Jonides & Nee, 2006; Monsell, 1978; Nelson et al., 2009). In the recent-probes task participants memorize an array 42 of four words per trial and are probed after a delay to judge whether a single word was present in the study array. On recent-negative trials, for example, the probe does not match on the current trial, but matches the previous trial’s study array resulting in an increase in RT and reduction in accuracy. The parent set of features in both the current study and that presented by Ueno and colleagues (2011) contained only 8 shapes and 8 colors. With such a limited parent set to draw from related arrays here and plausible suffixes in Ueno, et al (2011) had a high likelihood of being studied recently if not frequently on the previous trial. In the current study, related arrays were not probe objects and not part of the study array. However, given the high probability of containing features still active in memory (albeit possibly outside the focus of attention), attention may have been contingently guided to related objects in Experiment 1 bringing recent traces back into the focus of attention (similar to the recent-negatives effect). At first glance, this account of the effects observed in Experiment 1 is slightly problematic, as it doesn’t readily explain why related objects no longer disrupt performance more than an empty array when presented later during maintenance in Experiment 2. However, this could be due to the same argument we presented earlier, that distracter arrays may have been presented during ongoing stabilization of the study arrays (e.g. consolidation or initial attentive refreshing). Recent evidence suggests that the contents of VWM are not efficiently dropped from memory (e.g. removed from the focus of attention) on a trial to trial basis during change detection (Hartshorne, 2008; Logie & Brockmole, 2009; Makovski & Jiang, 2008). If objects containing recently active features (i.e. plausible suffixes, or study-related objects) are 43 presented prior to complete consolidation of the study array, the representational space of VWM may still have the capacity to reactivate a recent trace as part of the focus of attention. In fact, the results of Ueno and colleagues’ (2011b) cued recall experiment indicate that features of plausible suffixes can become active representations in VWM. However, the observation that suffix features intruded on cued recall does not necessarily dissociate between whether a trace was encoded or reactivated from memory. We suspect that after stabilization of the study array (e.g. 1300ms later in maintenance), the ability to reactivate the traces to the focus of attention may be reduced if not lost as capacity is filled with stabilized study array objects. While this account of disruption is only a speculation at this point it represents a parsimonious common mechanism for all effects observed here and elsewhere (e.g. Ueno, Allen et al., 2011; Ueno, Mate et al., 2011). In order to provide stronger evidence for this hypothesis, it would be important to directly manipulate suffix or related array object recency in a future study. Conclusions The results of the current study extend previous observations suggesting that binding change detection performance can be disrupted involuntarily via visual sensory input presented during VWM maintenance. Across both experiments we observed that the capture of attention away from VWM maintenance for multi-feature objects is disruptive of binding change detection. However, the content contained within filled arrays contributes an additional cost to performance that appears to be related to feature-overlap between objects in the array and the active contents of VWM. We speculate that disruption occurs because attention is 44 contingently captured by the objects contained within filled arrays, and that attending such objects has the ability to disrupt existing binding information and/or increase decision noise. We propose that further research should investigate the role that proactive interference plays in disrupting binding change detection. Specifically, we suggest that exposure to features recently encoded into VWM may be capable of capturing attention in a manner similar to the currently active contents of VWM. 45 CHAPTER 3 DISRUPTING BINDING CHANGE DETECTION THROUGH THE CONTINGENT CAPTURE OF ATTENTION In the previous chapter we demonstrated that binding change detection performance is impaired after the presentation of new visual information during VWM maintenance (e.g. visual distraction arrays). The results presented in Chapter 2 indicated that performance may be impacted in a graded manner depending on the visual features contained within distraction arrays. For example, consistent with previous research on the voluntary suspension of attention during VWM maintenance (Allen et al., 2006; Delvenne et al., 2010; Gajewski & Brockmole, 2006; Johnson et al., 2008; Makovski et al., 2006; Ueno, Allen et al., 2011; Yeh et al., 2005), the sudden onset of an empty visual array may result in the involuntary capture of attention away from visual rehearsal and results in a general cost to performance. In replication of a recent study by Ueno and colleagues (2011), the presentation of visual arrays containing objects constructed of study-related features results in a further reduction in performance (Ueno, Allen et al., 2011). However, these ‘related’ arrays appeared to only affect performance more than empty arrays when presented briefly after the offset of the study array. Distracter arrays containing objects with partial-feature overlap (e.g. color-matched) with the study array resulted in a reliably greater cost to performance than empty arrays when presented either briefly after the offset of the study array or later during maintenance. It was recently proposed by Ueno and colleagues (2011) that disruption of binding change detection performance occurs as a result of a failure to gate visual objects present at the percept from VWM representation. Specifically, they argued that such a failure results in replacement of information represented in VWM (Ueno, Mate et al., 2011). Indeed, the ability 46 to set attention to relevant information during VWM encoding (attending relevant information while ignoring irrelevant/distracting information) has been shown to be an important underpinning of working memory capacity (Engle, 2002; Fukuda & Vogel, 2009b; Poole & Kane, 2009; Vogel et al., 2005). However, in Chapter 2 we failed to observe a relationship between distraction during VWM maintenance and several tasks argued to be sensitive to individual differences in the control of attention (e.g. visual array task and complex span; Engle, 2002; Fukuda & Vogel, 2009b; Poole & Kane, 2009; Shipstead et al., 2012; Vogel et al., 2005). In failing to observe such a relationship, we hypothesized that disruption may be stimulus-driven. For example, we argued that objects containing features that overlap with the current or recently active contents of VWM may reflexively capture attention (Huang & Pashler, 2007; Olivers et al., 2006; Olivers, Peters, Houtkamp, & Roelfsema, 2011; Soto et al., 2008). It is currently unclear what the downstream effect of attention being captured by such objects is on VWM for the study array. However, we proposed in the previous chapter that attending features previously studied may retrieve recent representations back into the focus of attention (Cowan, 1999; Craig et al., 2013; Jonides & Nee, 2006; May et al., 1999; Monsell, 1978; Nelson et al., 2009), where attending currently active features may result in a reassociation of common and new features (Alvarez & Thompson, 2009; Fiacconi & Milliken, 2012c; Kahneman et al., 1992; Wheeler & Treisman, 2002). In either case, we proposed that disruption occurs through a common mechanism leading to poorer performance at test. That is, currently or recently active traces in VWM may automatically guide attention to to-beignored visual information presented during VWM maintenance (Huang & Pashler, 2007; Olivers et al., 2006; Soto et al., 2008). In the current study, we tested this hypothesis by 47 relating performance on the Binding Distraction Task (BDT) to neural activity evoked during the presentation of distracter arrays. Disruption of Binding Change Detection In their study, Ueno and colleagues (2011) constructed a binding change detection task that included a distraction manipulation (the “suffix” task). On every trial participants studied an array of four two-feature objects (colored shapes) and judged at the end of the trial whether a single test object contained the same or changed combination of shape and color as an object from the study set. In order to examine the impact of new visual information presented during maintenance, they centrally presented a suffix object 250ms after the offset of the study array. By varying the visual features contained within the suffix they observed that certain visual features are more likely to disrupt binding change detection performance than others (Ueno, Allen et al., 2011; Ueno, Mate et al., 2011). Specifically, they observed that objects containing features studied throughout the task (“plausible” suffixes) impaired performance more than suffixes containing features never studied on any trial (“implausible” suffixes). The labeling of such suffixes as ‘plausible’ and ‘implausible’ was linked to the researcher’s theory of how suffixes impinged on performance. For example, that objects containing features studied throughout the task would be perceived as ‘plausible’ members of the study set and allowed access to VWM representation. In order to disentangle a theory of how visual objects impact performance from an operational term describing the features contained, in the previous chapter and current study we referred to such objects as containing ‘study-related’ features (or ‘related’ for short). 48 In our previous study, we were able to replicate and extend the findings of Ueno and colleagues (2011) in a modified design. In our version of the suffix task (the BDT) we presented a distraction array containing four new colored shapes drawn from the same parent set of features used to construct the study objects (“related” distracters) or four color-matched shapes (“matched” distracters). The superiority of matched distracters, and the aforementioned lack of a relationship with measures of WMC, led us to speculate that disruption may be dependent on the active contents of VWM. Specifically, we argued that overlap between the currently or recently active contents of VWM may guide attention to matched and related objects contained within distraction arrays. As will be described below, this hypothesis makes specific predictions regarding the patterns of activity that should be evoked across different networks of the brain during the presentation of distracter arrays in the BDT. Neural Attention Networks Theories of attention generally point to two influences over the likelihood that sensory input receives attentive processing: top-down or ‘voluntary’ set and stimulus-driven salience (Desimone & Duncan, 1995; Folk et al., 1992; Theeuwes, 1994; Treisman, 1980; J. Wolfe, 1994; Yantis, 2008). The goals of a given task (i.e. classify an object’s color) or the knowledge that a forthcoming target contains a given feature (e.g. is ‘blue’ in color) or will appear in a certain location in space (i.e. upper-left of a grid) are examples of voluntary attentive states that are thought influence what information receives priority at the percept. During visual search these top-down biases/sets are thought to constrain the search space either spatially or segregated 49 along feature-based dimensions (e.g. limited to only objects containing ‘blue’). In other cases, the goals of a task are thought to selectively constrain attention to a given feature dimension, e.g. classifying an object’s color as opposed to its shape (Chiu & Yantis, 2009; Meiran, Kessler, & Adi-Japha, 2008). In models of attention (e.g. Wolfe, 1994) these biases are thought to be weighted along with stimulus-based salience in order to guide attention to objects that are considered the most salient in the environment. For example, objects that abruptly onset or are deviants along a given dimension, i.e. have a unique motion trajectory to other stimuli or are color or shape singletons, are thought to contain greater stimulus-based salience (Yantis, 2008). In prevailing models such as Wolfe’s Guided Search model (Wolfe, 1994), information available at the percept is thought to be prioritized for attention in terms the combined weighting of voluntary bias and stimulus-driven salience. How the brain implements top-down biases and responds to salient sensory stimuli has been the topic of considerable research in recent years (see Corbetta et al., 2008; Desimone & Duncan, 1995; Miller & Cohen, 2001; Yantis, 2008). Generally, research has focused on regions of the brain that correspond to the sustained and transient control of selective attention (Ikkai & Curtis, 2008; Kastner, Pinsk, De Weerd, Desimone, & Ungerleider, 1999; Liu, Slotnick, Serences, & Yantis, 2003; Serences, Schwarzbach, Courtney, Golay, & Yantis, 2004; Yantis, 2008; Yantis et al., 2002; Yantis & Serences, 2003) as well as regions that respond to exogenous cues to reprioritize/shift attention (e.g. stimulus-driven ‘capture’; Arrington, Carr, Mayer, & Rao, 2000; de Fockert, Rees, Frith, & Lavie, 2004; Kincade, Abrams, Astafiev, Shulman, & Corbetta, 2005; Serences et al., 2005). To distill well over a decade of research into a brief summary, there is a general agreement that the top-down or “voluntary” control of attention is thought 50 to be carried out by areas of the dorsal parietal and frontal cortices, specifically the superior parietal lobule/precuneus (SPL/PreC), intraparietal sulci (IPS), and frontal eye fields (FEF; for review see Yantis, 2008; & Corbetta et al., 2008). The SPL, IPS, and FEF often increase in activation during intentional shifts of attention (Yantis, 2008), resistance to distracting information including the prevention of involuntary saccades (Curtis, 2006), and when attention is selectively set to specific object features (e.g. color or direction of motion in (Kastner et al., 1999; Liu et al., 2003). In addition, specific plans for shifts of attention to an area of space or feature can be decoded from subtle patterns of activation within the dorsal parietal cortex (Chiu & Yantis, 2009; Esterman, Chiu, Tamber-Rosenau, & Yantis, 2009). The dorsal situation of these regions in the human brain has led some researchers to refer to this network as the dorsal attention network (DAN in Corbetta et al., 2008)). As was described above, prevailing models of attention also propose that various stimulus-based properties have the ability to guide attention. These properties (e.g. abrupt onsets, novel/unique features, etc) can cause attention to be ‘captured’ briefly by the stimulus’s presence. There is considerable debate as to whether certain properties are universally salient (i.e. ‘pure’ capture; Lavie et al., 2004; Theeuwes, 1994) or are only salient when they are related in some way to the goals of the task or top-down set (i.e. ‘contingent’ capture; Folk et al., 1992). However, brain imaging evidence suggests that specific neural interactions between the DAN and a complementary ventral network may mimic the boundaries of this debate. Early description of the neural regions involved in stimulus-driven attention was primarily based on spatial cueing (e.g. Arrington et al., 2000) and novelty detection paradigms (e.g. oddball tasks; Braver, Barch, Gray, Molfese, & Snyder, 2001; Linden 51 et al., 1999). In addition to activating regions of the DAN, these tasks also drove activation in regions such as the inferior parietal lobule (e.g. temporo-parietal junction, TPJ) and ventrolateral prefrontal cortex (VLPFC, specifically the anterior insula, AI and inferior frontal gyrus, IFG). This evidence was initially taken to suggest that the TPJ and VLPFC are responsible for detecting salient information in the environment and serve as the bottom-up ‘ventral’ complement to the DAN (see “circuit-breaker” analogy in Corbetta, Kincade, & Shulman, 2002). However, the observed behavior of these regions across several task and sensory domains has since led to a reappraisal of this ventral attention network (VAN) as it appears to be considerably modulated by the state of the voluntary attention system (Corbetta et al., 2008). In the domain of spatial attention capture, the presence of uninformative spatial cues (Kincade et al., 2005) and feature singletons that contain no overlap with the target of search (de Fockert et al., 2004) have been shown to capture attention yet exclusively result in an upregulation of activity of the DAN, perhaps reflecting a reinstantiation of top-down set in the presence of otherwise distracting information (c.f. Corbetta et al., 2008). However, when these spatial cues share a feature with the top-down set (e.g. are colored the same as the corresponding target) they drive activation of both the VAN and DAN. In terms of pure versus contingent theory of stimulus driven capture, activation of the VAN in these studies corresponds to conditions under which top-down biases come into line with stimulus-based features. In other words, the VAN is activated when attention is contingently captured by an environmental stimulus. A strong demonstration of this property of the VAN was provided by Serences and colleagues (Serences et al., 2005). In their study, participants were instructed to attend a 52 central visual stream of letters while waiting for a target to appear in a specified color. Along the left and right periphery of the central stream a to-be-ignored letter stream was presented in either a neutral gray color, a salient but unique color (e.g. light green), or in a color that matched the target for the central stream (e.g. red). Importantly, Serences and colleagues (2005) observed that the TPJ and VLPFC only increased in activity when the peripheral stream included letters that were color-matched to the target color, suggesting that attention was contingently captured when peripheral distracters matched the top-down attention set (e.g. the color associated with the target in the central stream). The above demonstrations of the dependence of the VAN on the state of voluntary attention set reside in the domain of the spatial capture of attention. Importantly, as would be predicted by recent behavioral work on the non-spatial contingent capture of attention (Folk, Leber, & Egeth, 2008) the dependence of the VAN on voluntary attention extends beyond spatial attention (Shomstein, 2012). In the VWM domain, activation in the TPJ has been shown to be suppressed during VWM maintenance commensurate with the number of objects maintained in memory (Todd & Marois, 2005). Todd and colleagues (2005) observed that the degree of suppression of the TPJ during VWM maintenance was predictive of a failure to detect transient visual onsets. Consistent with shared resource theories of attention (Kiyonaga & Egner, 2012; Lavie et al., 2004) this observation suggests that the occupation of voluntary attention both suppresses the VAN and reduces the likelihood that salient stimuli (e.g. abrupt onsets) will capture attention. Critically, this type of VWM load induced suppression of the VAN can reversed when information present in the environment is related to the top-down set (Anticevic, Repovs, 53 Shulman, & Barch, 2010; Majerus et al., 2011). A particularly strong example of this was demonstrated by Anticevic and colleagues (2010) who presented various distracting stimuli during VWM maintenance. Suppression of the TPJ, as well as other task-induced deactivated regions (regions of the default mode network, DMN), was observed during VWM maintenance. However, while emotional and neutral distracter stimuli resulted in increases in the DMN, taskrelated distracters (distracter shapes that were physically similar to the study objects) resulted in an increase in TPJ activity (Anticevic et al., 2010). In the verbal WM domain Majerus and colleagues (2012) have also observed a similar TPJ distracter effect when letters were presented during verbal WM maintenance for letter sets. While the TPJ was reliably suppressed during maintenance of large letter sets (6 letters) and unresponsive to distracter stimuli, during maintenance of smaller sets (2-4 letters) the TPJ remained sensitive to the onset of a distracter letter during maintenance (Majerus et al., 2011). Altogether, the observed selectivity of the VAN for environmental information that matches voluntary set has led to a critical reappraisal of the role that the VAN plays in the guidance of attention. That is, the VAN appears to be responsive to the contingent capture of attention rather than serving as a pure salience detector. This clarification of the roles of the VAN and DAN in the guidance of attention suggests that the DAN may be responsible for setting and further up-regulating top-down set in the presence of distracting information, while the VAN is responsible only for reappraising set in the presence of visual input that contains task relevant features. Corbetta & Shulman (2008) refer to this process as the VAN contributing a ‘reorienting’ signal, when information related to the top-down set is present in the environment. Importantly, this distinction between networks makes a critical prediction 54 regarding activation to distracter arrays in the BDT. We observed in our previous study that empty arrays are capable of disrupting binding change detection performance, indicating that they may capture attention away from VWM maintenance. However, because empty arrays contain no feature overlap with the study array or task goal we proposed that their abrupt onset is the only feature that garners any stimulus-driven salience. Similar to the aforementioned spatial cueing studies (de Fockert et al., 2004; Kincade et al., 2005) we expected that the presentation of empty arrays would only result in an up-regulation of topdown set and would be marked by an increase in activation of the DAN. Although, we expected that empty arrays would not increase activation within the VAN (e.g. TPJ/VLPFC) as they do not contain features that would induce the contingent capture of attention. We expected that filled arrays (related and matched objects) would drive activation of the DAN as well as the VAN. This prediction is somewhat of a departure from the types of tasks and outcome measures that have been used to study contingent capture and warrants further characterization. Typically, contingent capture has been studied in the context of visual search when the target of search and distracting information are present in a more temporally proximal timeframe (i.e. either simultaneously present during search, or briefly preceding the onset of the target; (Folk et al., 1992; Theeuwes, 1994). The capture of attention is thought to covertly or overtly direct attention away from a target object or the location in space that a target object will occupy (or conversely, promote attention to a location in space). The redirection of attention to an invalid object or point in space has the effect of slowing responsetime to a search target and a reducing accuracy for judgments made at the target’s location. Capture is often brief-lived, but recent evidence suggests that considerable individual 55 differences exist in terms of the speed at which individuals ‘recover’ from the effects of capture (Fukuda & Vogel, 2011). The delay between the onset of distracter arrays and test in the current study are well beyond even the longest individual estimates of recovery time (e.g. 350 ms in Fukuda & Vogel, 2011). Therefore, in the current study we were not attempting to provide evidence of capture through response-time to test objects as would be characteristic in studies of visual search. Rather, similar to previous ERP studies which have identified electrophysiological components of brain activation that correspond to capture (Fukuda & Vogel, 2009a), our goal was to use a neural localization theory of attention to provide evidence of capture during VWM maintenance. Specifically, we predicted that activation to distracter arrays in regions that are associated with the contingent stimulus-driven capture of attention would predict the loss of performance observed at test in the BDT. Additionally in our study, related and matched objects are not necessarily related to the task goal, per se, to detect binding changes in the test object. However, recent evidence suggests that the active contents of VWM may operate in a manner similar to top-down set in causing attention to be contingently captured by the presence of feature-overlap in the environment (Huang & Pashler, 2007; Olivers et al., 2006; Soto et al., 2008). For example, attention may be captured by an object containing color-overlap with the contents of VWM during search for orientation, even when color is irrelevant to the search task (Soto et al., 2008). Given the high degree of feature overlap contained within matched object arrays, we expected that matched objects would contingently capture attention and drive activation of the VAN. However, the pattern of activation expected for related objects was less clear. On one hand, the increase in the amount of information present in related arrays may result in greater 56 activation of the DAN than empty arrays. However, the results of our behavioral work with the BDT (see Chapter 2) indicate that related arrays may also capture attention and activate the VAN. In our previous study, we had observed that related objects were capable of disrupting performance when presented early in VWM maintenance (250ms after offset of the study array) but less so when presented later during maintenance (1300ms). Further, this finding was consistent with previous work suggesting that objects containing study-related features are disruptive at this brief interval (Ueno, Allen et al., 2011; Ueno, Mate et al., 2011). We proposed that this graded ability of related objects to disrupt performance was related to the efficiency with which old VWM traces are replaced in memory. The limited number of features used to construct study objects in our task and previous studies (8 colors and 8 shaps, see also (Ueno, Allen et al., 2011) leads to a high likelihood for related distracter objects to contain features studied on the previous trial. While VWM appears to be efficiently updated with new information (Vogel et al., 2001; Woodman & Vogel, 2005), recent evidence suggests that old information is not efficiently ‘removed’ from representation (Makovski & Jiang, 2008). For example, in a change detection study by Makovski and Jiang (2009) participants were less likely to judge the feature as having changed on the current trial if the test feature was studied at that location on the previous trial. This type of carryover effect (i.e. proactive interference effect) is consistent with several attention-based theories of working memory (Cowan, 1999; Oberauer et al., 2003; Unsworth et al., 2012). In these models, working memory traces become active when studied, of which a limited number are capable of being maintained in the ‘focus of attention’ at any time. Proactive interference effects such as the one observed by Makovski & 57 Jiang (2009) are thought to emerge as a function of older objects being removed from focus of attention but remaining in an accessible state (e.g. secondary memory in (Unsworth et al., 2012). Therefore, we proposed that study features may remain active enough on a trial to trial basis to guide attention to related distracter objects. However, due to variability in the degree of overlap from trial to trial (e.g. features were randomly selected for each trial) we did not make any specific prediction as to the strength of activation expected in the VAN for related objects as compared with matched objects. In the current study we modeled activity evoked by the presentation of distracter arrays (empty, related, and matched), along with a regressor set to an equivalent jittered time-point during maintenance in the no distraction condition to serve as a maintenance baseline. We expected two patterns of activation: 1) in the DAN we expected a significant increase in activation between no distraction the three distracter array conditions. Additionally, assuming the VAN is responsible for sending a signal to the DAN in the presence of stimuli that are related to the contents of VWM, we expected that activity in the DAN would further increase to filled arrays (related and matched) as compared to the empty array; 2) in the VAN we expected activity to increase only between the empty and filled array conditions with no significant increase between no distraction and the empty array condition. In our previous study we failed to find a relationship between measures of the control of attention (e.g. working memory capacity) and performance costs in the BDT. As a result, we proposed that disruption of binding change detection performance may be dependent on the contingent capture of attention. In order to evaluate whether disruption of performance was related to either the control or contingent capture of attention we conducted an individual differences analysis. If binding 58 change detection performance is impaired through a failure to gate related and matched objects, then performance should be marked by a failure to engage regions associated with the control of attention (DAN regions). In other words, participants with the highest cost to performance should have the smallest increase in activation in DAN regions, an indication of a failure to engage in the control of attention (i.e. a negative correlation between DAN activity and distraction-cost). On the other hand, if performance is impaired as a result of attention being contingently captured by related and matched objects, then participants with the highest cost to performance should have the greatest increase in activation of VAN regions. In other words, there should be a positive correlation between activity in VAN regions and distractioncost. Methods Participants: Participants were 26 healthy, right-handed, English-speaking, Michigan State University undergraduate and graduate students (18 Female), ages (M = 22.04, range = 19-28). Participants were paid for their participation in the study. The study was approved by the Michigan State University Office for the Protection of Human Subjects. Experimental Procedure: Binding Distraction Task (BDT): All stimuli were presented on a neutral gray background. Each trial began with the onset of a fixation cross to signal the beginning of a trial (500ms) immediately followed by the onset of 3 unique colored-shapes presented around the corners of an invisible square centered on the screen for 1500ms (“study array”). The exact corners of the invisible square at which the 3 objects appeared were randomly chosen on each trial. The 59 invisible square occupied approximately 11 x 11 degrees of visual angle (left-right and topbottom) with each visual object measuring 3.7 x 3.7 degrees of visual angle. After a variable fixation delay (2000-4000ms jittered at 6 intervals) one of four distraction events was presented for 2000ms: 1) no distraction; 2) empty array; 3) related object array; & 4) matched object array. In the no distraction condition 2 seconds of additional fixation was included to match the duration of the other three conditions. In the empty array condition empty boxes drawn in 5pt black outline surrounding the location of study array objects were presented. In the related object condition the same black outline array was filled with 3 colored shapes generated from the same parent set of features as the study array but not matching any object in the study array on shape or color. In the matched array the black outline box array was filled with 3 colored shapes generated from the same parent set of shapes as the study array (not matching in shape) but matching the study array objects in color (each matched distracter matched 1 object from the study array in color). In the related and matched conditions, distracter objects always appeared at 3 random locations within the array. After the offset of the distracter array, the fixation cross remained onscreen for a variable interval matched to the jittered pre-distracter fixation delay so that the total study to test interval was 8000ms (e.g. pre-distraction fixation = 2000ms + 2000ms distracter array + 4000ms post-distraction fixation). After the post-distraction array interval a single test object was presented centrally (presented directly over the fixation cross, center screen). The test object always included one shape and one color present in the study array. At test, participants judged whether the combination of shape and color in the test object was the ‘same’ as one of 60 the objects at study or ‘changed’ across objects from study regardless of its location in the study array(test object = 50% probability of same or changed). The test object remained visible for 2000ms. After the offset of the test object the screen remained in solid gray background until the reappearance of the fixation cross signaled the beginning of the next trial (see Figure 6). Participants were instructed that the experiment was a test of visual memory and were provided with a visual depiction of each trial condition. The instructions included an emphasis on memorizing the combination of shape and color present in each study object, and that the test object would include one shape and one color that were BOTH present in the study set. It was emphasized that participants respond as accurately as possible while attempting to make a response during the 2000ms window while the test object was present on the screen. However, they were informed that on trials where they were not prepared to respond while the test object was visible, a response could still be made after the test object was cleared from the screen. They were urged to maintain fixation throughout the maintenance interval, and particularly while the distracter array was present. They were told that the goal of the study was to examine how well they could ‘mentally ignore the distracter array while the array was within their field of view.’ Thus, they were asked to not utilize alternative strategies such as closing their eyes or darting their eyes to an area of space outside the array. Upon debriefing all participants reported following these instructions. 61 Participants completed approximately 40 trials per condition (+/- 1 trial per condition) for a total of 160 trials. The experiment was run in 10 blocks of 16 trials with a break between each block while the next fMRI scan was prepared. Participants were in the scanner for a total of approximately 75 minutes including setup time, anatomical acquisition, between run breaks, and experimental runs. Eight colors and eight shapes constituted the parent set from which all stimuli were generated including: white, black, yellow, cyan, blue, red, magenta, and green; circle, diamond, teardrop, tall rectangle, square, heart, triangle, and star (see Figure 1). Prior to participation in the Binding Distraction Task, participants were provided instructions and twenty practice trials with onscreen trial by trial feedback (correct/incorrect) to acquaint them with the experimental procedures. Additionally, participants were given a chance to ask any questions about the task and briefed on what to expect in the scanner environment (e.g. scanner noise during runs, how to use the response glove, and restricting head motion). Figure 6. fMRI Task Design. (TOP) Example of all colors and shapes used to construct study objects (MIDDLE) Timecourse of each trial (BOTTOM) Example of each distraction condition. 62 Image Acquisition: MRI data acquisition was performed on a 3T GE Signa Scanner. Functional data were collected with a Blood Oxygenation Level Dependent (BOLD) echo-planar imaging (EPI) sequence (TR/TE = 2000/27.7 ms, FOV = 220 mm, matrix = 64 x 64, slice-thickness/gap = 3.4/0 mm). For anatomical reference, registration of functional data, and for normalization of functional data to a standard T1 template (Montreal Neurological Institute, MNI) a T1 magnetization prepared, rapid-acquisition gradient echo (MPRAGE, TR/TE = 11.876/5.012 ms, FOV = 240 mm, matrix = 192 x 256, slice-thickness/gap = 1.5/0 mm) sequence was used to collect a high-resolution image of the participant’s brain. Task stimuli were presented via E-Prime (version 2.0, Psychology Software Tools, Inc., Pittsburgh, PA) and projected from a shielded projector onto a screen mounted within the bore of the MRI scanner. The projector’s image was reflected to the participant’s visual field with a head-coil mounted mirror. Button-responses were logged with a BrainLogics Fiber Optic Response System glove (Psychology Software Tools, Inc., Pittsburgh, PA). Imaging Analysis: Preprocessing: fMRI and MRI data were preprocessed and analyzed using FMRIB’s Software Library (FSL) fMRI Expert Analysis Tool (FEAT; Smith et al., 2004). Functional data were brainextracted (Smith, 2002), motion-corrected to the median functional image using b-spline interpolation (4 df), high-pass filtered (60s/cycle), and spatially smoothed (9mm full width at half maximum (FWHM), isotropic). The anatomical volume was brain-extracted and registered to the standard space T1 MNI template using tri-linear interpolation with FMRIB’s Linear Image 63 Registration Tool (FLIRT, 12 df; Jenkinson & Smith, 2001)). The median functional image was registered to the anatomical volume, and then transformed to the MNI template. Individual Subject Analysis: Statistical images were created using FEAT with an improved General Linear Model (GLM; Smith et al., 2004). Regressors were created by convolving binary time-course files containing the onset time for each condition with a canonical hemodynamic response function (double gamme HRF). Each individual subject model contained a regressor for each of the 4 conditions (no distraction, empty array, related array, matched array) at each stage of performance (encoding, distraction, and retrieval) resulting in 12 timecourse regressors. Each regressor was entered into the GLM along with its temporal derivative and 6 motion nuisance regressors (motion in x, y, z, roll, pitch, and yaw). Group Analysis: A 1 (stage) x 4 (condition) repeated measures ANOVA was conducted on statistical maps generated from the individual subject analysis for each stage of the BDT. As the encoding and retrieval ANOVAs did not result in any significant main-effect of condition, only the results of the 1 x 4 ANOVA on distracter activity are reported below. After identifying voxels exhibiting a significant main-effect of condition, paired-samples contrasts were conducted in a second-level GLM using a binarized mask containing only voxels showing a whole-brain FWE-corrected p < 0.05 main-effect of condition (z > 4.3). This masking process restricted planned comparisons to regions showing a significant main-effect of condition. For all within-subjects comparisons, individual subject beta-images were entered 64 along with a regressor per subject to account for subject-specific variance. Group analyses were performed using FSL’s FLAME higher-level analysis tool (Woolrich et al., 2009), and all Fand T-statistics were converted to unit-normal Z-statistics. Region-of-Interest Analysis (ROI)/Percent Signal-Change: Mean percent signal-change for each ROI was generated using FSL’s featquery utility which calculates percent signal-change based on mean effective regressor height for voxels contained within the interrogated mask image. To create an ROI mask image for each region a spherical mask was generated with a radius of 5mm centered at the local peak identified within each cluster (see Results for ROI-peak definition procedure). Mean percent signal-change was calculated across all voxels falling within the spherical mask. All correlations between mean percent signal-change estimates and distraction-costs in the BDT were performed in SPSS. Distraction-cost for each condition was defined as the difference in corrected recognition rate (CR; hit rate minus false-alarm rate) for each condition with no distraction. As an example, distraction-cost for the matched condition was calculated by subtracting matched CR from no distraction CR, such that a larger distractioncost was indicative of a greater loss of performance for the matched condition as compared with no distraction. Results Behavioral Results: Accuracy (Corrected Recognition = Hit rate minus False-alarm rate): A 1 x 4 (condition) repeated measures ANOVA was conducted on accuracy for the BDT where a significant main-effect of condition was observed (F(3,75) = 7.39, MSE = 0.09, p < 0.0001). Planned comparisons were 65 conducted to reveal the shape of the main-effect of condition. While no significant reduction in accuracy was observed between no distraction and empty (t(25) = -0.15, p = 0.88) a significant reduction in accuracy was observed between both related and empty (t(25) = 2.52, p = 0.02) as well as matched and empty (t(25) = 3.77, p = 0.001). No significant difference in accuracy was observed between matched and related (t(25) = 0.91, p = 0.37; see Figure 7). Figure 7. Corrected Recognition for Each Condition. A significant reduction in accuracy was observed between empty and related, and empty and matched arrays. No difference was observed between no distraction and empty or related and matched conditions. Response time (RT): The results of a 1 x 4 (condition) repeated measures ANOVA on mean RT to the test object revealed no significant main-effect of condition on RT (F(3,60) = 1.16, MSE = 6497, p = 0.33). See Figure 3 for mean RT per condition. 66 Figure 8. RT for Test Objects. No difference in RT was observed across any condition at test. Imaging Results: Whole-Brain Analysis: The results of the 1 x 4 (condition) repeated measures ANOVA on activity to distraction revealed a significant main-effect of condition across large clusters covering much of the prefrontal, parietal, and occipital cortices, as well as several subcortical regions (see Figure 9). The goal of the whole-brain analysis was to identify regions that show selective response to empty and filled arrays. In order to identify regions showing increase in activation to the onset of the empty and filled arrays two types of planned comparisons were performed: 1) array onset-sensitive (empty > no distraction) and 2) array object-sensitive (related > empty; matched > empty). Significant voxels were those surviving a main-effect-masked, voxel-wise threshold of FWE-corrected p < 0.05 (z > 4.0) during planned comparisons. 67 Figure 9. fMRI Main-Effect of Condition and Planned Comparisons. (TOP) Regions showing a significant main-effect of condition (MIDDLE) Results of planned comparisons overlaid on the main-effect of condition (BOTTOM) Results of the contrast matched > empty overlaid the results of related > empty. A limited number of regions displayed array onset-sensitivity and no array object-sensitivity. That is, they increased in activation to the abrupt onset of the empty arrays and increased no further for filled arrays (primary visual cortex, left frontal pole; parahipppocampal cortex/posterior thalamus see Table 2 and Figure 9). As was predicted for regions involved in the control of attention (i.e. DAN), all other regions identified in the array onset-sensitive contrast showed further increases in activity during the array object-sensitive contrasts. 68 Further, the results of the array object-sensitive contrast of matched > empty entirely subsumed that of related > empty (see Figure 9). No regions showed increased activity during the inverse of each of these contrasts. Therefore, the contrast of matched > empty was used to guide identification of significant clusters of activation. Regions displaying array onsetsensitivity and further array object-sensitivity included the SPL/Prec, bilateral IPS, and bilateral LOC (see Figure 10). At a slightly more liberal threshold of p < 0.001, k > 10 contiguous voxels two additional regions, the left IFG and left FEF, also displayed the same pattern of array onsetand array object-sensitivity (see Figure 11). The fact that activity resulting from the contrast of matched > empty entirely subsumed the related > empty contrast led us to perform a follow-up contrast between matched > related. The contrast of matched > related revealed that activity in the bilateral IPS, SPL/Prec, and bilateral LOC showed an additional significant increase between object conditions. Thus, these regions showed a whole-brain significant parametric increase across every level of the design: empty > no distraction, related > empty, and matched > related. 69 Figure 10. Regions Showing Parametric Increase in Activity across Conditions. Results of matched > empty thresholded at FWE-corrected p < 0.05. Several regions showed a significant increases in activation across each condition including the SPL/Prec, bilateral IPS, and bilateral LOC. Two sub-clusters within each the left and right IPS and left and right LOC were identified through the contrast of matched > related. As each smaller cluster showed the same pattern of activation across conditions, percent signal-change estimates shown here are only for the inferior IPS and lateral LOC/Occ Fus clusters. The results of the array object-sensitive contrasts revealed several regions that displayed specific activation only to filled arrays (i.e. no increase in activation to empty arrays). These regions included several aspects of the VAN, along with other unpredicted regions: the right TPJ (angular gyrus), bilateral AI, ACC, right IFG, right middle frontal gyrus (MFG), and right FEF. See Figures 11 and 12 for visual depiction of these regions along with percent signal-change estimates for every condition. 70 Figure 11. Regions of the DAN Showing Onset- and Object-Sensitivity. The left FEF and IFG (black outlines) displayed array onset- and object-sensitivity while the right FEF and IFG (blue outlines) only showed object-sensitivity. 71 Figure 12. Regions Showing only Array Object-sensitivity. Several regions showed objectsensitivity including those associated with the VAN. One region, the left pallidum, revealed in the 1 x 4 main-effect of condition was not described by the above planned comparisons. A post-hoc contrast between matched > no distraction revealed that this regions were only significantly more active during the matched array condition than no distraction, with sub-threshold increase during empty and related conditions (see Figure 13). 72 Figure 13. Sub-cortical Regions Showing a Main-effect of Condition. ROI Analysis: The goal of the ROI analysis was to relate activation to distracter arrays to behavioral performance at test. As the 1 x 4 main-effect of condition resulted in large contiguous clusters covering multiple brain regions and lobes, the peak of the contrasts between conditions were used as a form of data reduction. In this way, ROI analysis was datadriven (i.e. a functional ROI approach) and limited to aspects of large clusters showing a maximal conditional effect. In other words, local cluster peaks were isolated from the large contiguous clusters (e.g. the 1 x 4 main-effect occipital-parietal cluster covering IPS, SPL, IPL, LOC, and primary visual cortices) based on the results of lower-level contrasts. As was mentioned before, the results of the matched > empty contrast entirely subsumed that of the related > empty contrast. The peak of clusters revealed in the contrasts of matched > empty were considered as the maximal effect of condition and identified as functional ROIs for 73 offline analysis in SPSS (see below). For those regions showing a parametric increase between conditions (IPS, LOC, SPL/Prec), the contrast of matched > related was used to identify smaller clusters within IPS (superior and inferior) and LOC (dorsal and lateral) as this contrast resulted in several sub-clusters of activity within the large cluster spanning all of these regions in the matched > empty contrast (see Table 2 for a list of where each peak was identified for the ROI analysis). To identify the regions associated with the loss of accuracy during the related and matched object conditions (i.e. ‘distraction-cost’) a correlation analysis was performed between mean percent signal-change estimates and distraction-cost for each condition. Distraction-cost for each condition was calculated by subtracting corrected recognition rates for each condition (related and matched) from corrected recognition rates for no distraction. Thus, a higher distraction-cost corresponds to a greater loss of accuracy compared to no distraction. Costly Activity: A significant positive correlation between activity evoked by related and matched object arrays and related and matched distraction-cost was observed in both the right AI (related r = 0.5, p = 0.01; matched r = 0.42, p = 0.03), and right LOC/occipital fusiform (R LOC/OF; related r = 0.46, p = 0.02; matched r = 0.41, p = 0.04). Activity in the left AI was also positively related to distraction-cost but only for related object arrays (related r = 0.45, p = 0.02; matched r = 0.06, p = 0.79). 74 A second cluster of activity in the dorsal aspect of the R LOC was positively related to matched distraction-cost (r = 0.41, p = 0.04) but unrelated to related distraction-cost (r = 0.25, p = 0.21). Conversely, activity in the left pallidum (L Pall) was positively related to related distraction-cost (r = 0.43, p = 0.03) but unrelated to matched distraction-cost (r = -0.04, p = 0.87). See Figure 14 for scatter-plots. Protective Activity: Activity in no region was inversely associated with distraction-cost. In other words, no region was identified in which increases in engagement of that region resulted in less distraction-cost. Figure 14. Individual Differences in ROI Activation and Distraction-costs. Scatter-plots showing the relationship between distraction-costs for related (blue) and matched (green) array conditions and (A) right AI, (B) right LOC/Occ Fus, (C) left Pallidum, (D) right dorsal LOC 75 Table 2. Results of Whole-brain Analysis and Contrasts Between Conditions. Region Sub-region Left Hemisphere Prefrontal FEF IFG AI Frontal pole -36 -44 -34 -42 4 4 22 50 50 30 -4 -16 7.41 8.88 5.87 4.94 X* X* Parietal Superior IPS Inferior IPS -12 -28 -70 -70 48 28 9.24 9.75 X X X X X X M>R M>R Occipital Lateral LOC/Occ Fusiform Dorsal LOC Primary Visual Cortex -42 -40 -10 -68 -86 -94 -16 10 -8 11.06 9.31 9.87 X X X X X X X M>R M>R E > ND Midline ACC SPL/Precuneus 0 -2 20 -64 46 50 7.77 9.28 Sub-Cortical R PHG/Thalamus L PHG/Thalamus L Pallidum 20 -20 -14 -30 -30 -2 -6 -6 -4 7.01 7.17 4.5 Right Hemisphere Prefrontal FEF IFG MFG AI 30 50 40 36 4 8 32 22 52 34 24 -6 6.31 7.72 6.44 6.35 Parietal Superior IPS Inferior IPS TPJ (R Ang) 12 32 54 -70 -70 -46 52 26 28 10.4 10.01 6.02 X X X X X X X M>R M>R M>E Occipital Lateral LOC/Occ Fusiform Dorsal LOC 40 34 -88 -84 -18 16 11.7 9.62 X X X X X X M>R M>R X Local Peak Y Z Zmax 76 OnsetObjectROI M>R sensitive sensitive definition X X X X X X X M>E M>E M>E E > ND X X* X M>E M>R E > ND E > ND M > ND X X X X M>E M>E M>E M>E Table 2 (cont’d) Primary Visual Cortex 8 -90 -12 10.65 X E > ND Note: Coordinates are in MNI space, M = matched, R = related, E = empty, ND = no distraction, peak coordinates were identified by the contrast listed under ‘ROI Definition’, Zmax values correspond to the main-effect of condition, * = reliable at p < 0.001, uncorrected for multiple comparisons. Discussion In the current study we sought to provide neuroimaging support for our hypothesis that binding change detection performance is disrupted through the contingent capture of attention. Distracter arrays presented during VWM maintenance contained ‘to-be-ignored’ visual information that we (see Chapter 2) and others (Ueno, Allen et al., 2011; Ueno, Mate et al., 2011) had previously shown to disrupt binding change detection performance. Specifically, we expected that feature-overlap between the currently (matched) or recently (related) active contents of VWM would cause attention to be guided to distracter objects and disrupt the ability to make binding change judgments. We anticipated that this would be reflected in both a behavioral cost to performance and selective activation of neural networks associated with the voluntary and stimulus-driven guidance of attention. Further, we expected that if disruption of performance is stimulus-driven then activity among regions associated with the contingent capture of attention (VAN) would be positively related to performance costs at test. Behaviorally, we observed a significant performance cost associated with the presentation of filled arrays (related and matched objects) during VWM maintenance, and no cost associated with the presentation of the empty array. We also observed a dissociation in the pattern of activation produced by empty and filled arrays. As was predicted, the absence of task relevant features in empty arrays resulted in a lack of activation in the VAN (TPJ/AI/right IFG) as well as other prefrontal regions (right FEF, ACC, and MFG). These regions however, 77 were significantly driven by the onset of both related and matched object arrays. Further, increase in activation of a prefrontal aspect of the VAN (the right AI) was predictive of performance costs for related and matched objects. Both empty and filled arrays drove activation of regions associated with the control of attention (i.e. DAN regions). Moreover, a parametric increase in activation was observed in the SPL/Prec, bilateral IPS, left FEF, and left IFG across every level of the design. The results of our ROI analysis revealed that activation in these regions was unrelated to performance, indicating that disruption did not likely result from a failure to engage in the control of attention (i.e. attentive gating). Still, this pattern of activation in the DAN is noteworthy for several reasons. Previously (see Chapter 2), we had shown that the abrupt onset of an empty visual array during VWM maintenance resulted in a general cost to performance that was consistent when the empty array was presented earlier or later during maintenance (250ms or 1300ms after the offset of the study array). The fact that empty arrays were no longer disruptive of performance in the current study may have resulted from the reduction in VWM load between studies (from 4 to 3 objects per trial). The ability to engage in early filtering of distracting information has been shown to improve under conditions of lower WM load (Lavie et al., 2004). Further, VWM maintenance is thought to be, at least in part, resolved through an attentive refreshing process (e.g. Chun, 2011). The voluntary suspension of this refreshing process has been previously shown to result in a small but reliable cost to VWM across several studies of change detection (Allen et al., 2006; Delvenne et al., 2010; Gajewski & Brockmole, 2006; Johnson et al., 2008; Makovski et al., 2006; Yeh et al., 2005). If attentive refreshing and early filtering indeed rely on a shared resource as some would argue (see Awh, Anllo-Vento, & Hillyard, 2000; Chun, 2011; 78 Kiyonaga & Egner, 2012)) then the reduction in VWM load in the current study may spare the ability to keep representations in a refreshed state while ignoring some types of information presented during maintenance. Further, the activation in the DAN for empty arrays may reflect this operation as a significant increase was observed between no distraction and empty arrays in the absence of performance cost. We expected to observe an increase in activation between empty and filled arrays in the DAN; however, the additional increase observed between related and matched objects was surprising and indicates one of two possibilities. On one hand, the increase in activation between empty and filled arrays in these regions could be due to the increase in information present in filled arrays. However, the significant increase between related and matched arrays suggests that these DAN regions are either sensitive to the qualitative difference between related and matched objects (a quality that is non-spatial as they occupy exactly the same number of regions of space), or they are being driven by signals from other parts of the brain that are sensitive to this difference (as will be discussed below). Our previous behavioral work with the BDT indicated that matched objects were more disruptive of binding change detection performance than related objects; an effect we felt was directly related to the active contents of VWM. However, we found no evidence for this behaviorally in the current study as no significant difference in accuracy was observed between conditions. Despite this lack of a behavioral distinction, we observed a significant whole-brain threshold difference in activity in the SPL/Prec, bilateral IPS (superior and inferior), and LOC (dorsal and lateral). While failing to reach whole-brain statistical thresholds, this pattern was evident in nearly every other region identified as a functional ROI, including all regions of the 79 VAN. As was reviewed in the introduction, the specific sensitivity of the VAN to stimuli that are related to the top-down set has been taken as evidence that the VAN is responsible for sending a reorienting signal to the DAN (see Corbetta et al., 2008). This signal is thought to result in the capture of attention and only occur when information in the environment matches the contingencies of the top-down set (i.e. contingent capture in Serences et al., 2005). We argued based on recent evidence that the contents of VWM may operate as a form of top-down set (i.e. create a top-down contingency, see Soto et al., 2008), and that the overlap between matched objects and the study set would drive activation of the VAN. The type of graded response in the VAN to related and matched objects fits well with our previous behavioral data suggesting that matched objects are superior to related objects in the disruption of performance in the BDT. Further, this gradation may reflect the degree of overlap between the active contents of VWM and distracter objects (matched = all of the study set; related = some portion of the previous study set) as well as the strength of the underlying representations (matched = focus of attention; related = active but outside the focus of attention). If our data are in fact related to the guidance of attention by the active contents of working memory, then our results stand as an extension of the work of Soto and colleagues (2008) to the guidance of attention by the previously active contents of VWM. We would argue that if the role that the VAN plays in the control of attention is to signal when behaviorally relevant information is present in the environment, then the parametric increase in the DAN may reflect the summation of signals across networks (onset-sensitivity in the DAN + graded object-sensitivity in the VAN). Based on our ROI analysis, it appears that the right AI, out of all VAN regions, may either be the most sensitive to this feature-overlap or 80 responsible for triggering the reorienting signal as increases in activity in the right AI led to greater performance costs. However, it is not possible to dissociate between these possibilities in the current design as has been done elsewhere (see Shulman et al., 2009). The results of Shulman and colleagues (2009) had indicated that the right AI shares unique functional resting connectivity with regions outside the VAN (e.g. the basal ganglia) and not with other areas of the VAN (e.g. right TPJ). Further, their study indicated that the right AI only becomes active when a shift of attention is necessary, while the TPJ appeared sensitive to violations of expectancy (e.g. attention capturing) and cues to shift attention (Shulman et al., 2009). This dissociation in function taken along with our current results leads us to propose here that the right AI may be the primary source of the binding change detection decrement observed in our own (see Chapter 2) and other’s research (Ueno, Allen et al., 2011; Ueno, Mate et al., 2011). Beyond the VAN and DAN It would be intractable to attempt to describe the pattern of activation, functional relevance, and/or role that all of the functional regions revealed by the current study. Our analysis revealed a significant main-effect of condition across nearly every region of the brain ever implicated in studies of attention, working memory, and cognitive control. Further, the purpose of our study led to the implementation of a task design that limits our ability to explain every significant effect observed here. However, some of our current findings complement while others challenge existing theories of regional function. Our motivation for the current study was to evaluate a critical hypothesis regarding why some types of visual input present during VWM disrupts the ability to perform binding change 81 detection. Using Corbetta’s influential model of attention as a guideline for our predictions, we cast a rather large net over areas of the superior frontal and parietal cortices as areas that are involved in the control of attention and regions of the ventral parietal and frontal cortices as areas responsive to the contingent capture of attention. In doing so, we provided sufficient evidence in support for our hypothesis that binding change detection is disrupted by related and matched distracters because they contingently capture attention. Based on their model, we proposed that for several regions (e.g. SPL/Prec, IPS, LOC), the parametric increase observed may emerge as a function of signal summation between the DAN’s sensitivity to abrupt visual onsets plus a reorienting signal contributed from elsewhere (e.g. the VAN). This observation potentially demonstrates the flexibility of the DAN system to engage control in a graded manner under varying demand from the environment. However, this graded pattern of activation in the superior parietal lobe may pose a challenge to existing models of VWM representation (e.g. Mitchell & Cusack, 2008; Song & Jiang, 2006; Todd & Marois, 2004; Vogel et al., 2005; Xu & Chun, 2006). Additionally, several regions that are less often implicated in the control of attention showed sensitivity to the onset of distraction arrays (e.g. ACC, PHG/Thal, Pallidum). Below we review the potential implications of these effects as they relate to current models of VWM and cognitive control. Anterior Insula and Anterior Cingulate The ACC is less often identified as a region that is involved in the control of attention (for review see Corbetta et al., 2008; Yantis, 2008). However, it is frequently observed in studies of working memory and cognitive control (Botvinick, Braver, Barch, Carter, & Cohen, 82 2001; Botvinick, Cohen, & Carter, 2004; Braver et al., 2001; Brown & Braver, 2005; Cole & Schneider, 2007; Hakun & Ravizza, 2012; Owen, McMillan, Laird, & Bullmore, 2005). The ACC is commonly thought to be involved in the monitoring or resolution of conflict during response planning (Badre & Wagner, 2004; Botvinick et al., 2001; Botvinick et al., 2004) and the detection or prediction of errors during task performance (Brown & Braver, 2005, 2008; Cole & Schneider, 2007). In both cases, the ACC appears primarily sensitive to response-related contingencies. In the cases where it has been observed in the control of attention, the ACC shows similar motor response-related activation. Specifically, in studies of visual attention it has been shown to be involved with the inhibition of pre-potent ocular movements (i.e. prevention of saccades to abrupt visual onsets; (Curtis, 2006). Participants in the current task were instructed to maintain a central fixation throughout the maintenance interval. The increase in activation in the ACC could be due to anti-saccade related activity. However, it is unclear why the ACC would not be activated by the abrupt onset of empty arrays, and further why it would increase in activation between related and matched distracters. The selective response of the ACC to distracter objects in the current study may be reflective of its recently implicated role in the detection of salience along with the AI (for review see Menon & Uddin, 2010). Similar to the ACC, the AI has received considerably less attention than other areas of the parietal cortex (e.g. TPJ) in studies involving the stimulus-driven capture of attention (see Chang et al., 2012). In part, this may be due to the varying nomenclature associated with the region (VLPFC in Serences et al., 2005; IFG in Kincade et al., 2005; VFC/AI in Corbetta et al., 2008). Additionally, the consistent observation that the AI is involved in the retrieval and processing of emotional information (Phan, Wager, Taylor, & Liberzon, 2002), nociception 83 (Craig, Chen, Bandy, & Reiman, 2000), and other body-state related (e.g. empathy; mirrored cognition Gallese, Keysers, & Rizzolatti, 2004; Jackson, Meltzoff, & Decety, 2005) research has generally shifted the view of the AI as part of the extended ‘limbic’ system rather than a more cognitively-oriented region of the brain. However, recent advances in resting functional connectivity research, coupled with fMRI of cognitive tasks within the same participants (e.g. Shulman et al., 2009) has revealed that at least a subsection of the AI plays an integral role in higher cognitive function (Menon & Uddin, 2010). For example, the same aspect of the superior AI that showed sensitivity to cues to shift attention in Shulman et al (2009) also showed resting state functional connectivity with other regions of the brain such as the basal ganglia, which has recently gained a good deal of attention as a source for cognitive control (see Awh & Vogel, 2008; Hazy, Frank, & O'Reilly R, 2007; McNab & Klingberg, 2008). Both resting functional connectivity and meta-analytic approaches to studying AI function have begun to shed light onto why the right AI is observed across such a wide variety of emotional and cognitive tasks (Kurth, Zilles, Fox, Laird, & Eickhoff, 2010; Touroutoglou, Hollenbeck, Dickerson, & Feldman Barrett, 2012). One perspective suggests that there are likely two neighboring sub-regions within the right AI with functional specialization for emotional and cognitive processing: a ventral aspect which shares connectivity with regions typically associated with emotional and nociceptive process (e.g. the amygdala), and a dorsal which shares connectivity with regions associated with executive function (e.g. working memory performance, Touroutoglou et al., 2012; also see Taylor, Seminowicz, & Davis, 2009). However, a critical review by Menon and colleagues (2010) proposes that direct cortical projections between the AI and the ACC may allow for what they call ‘rapid access to the motor 84 system’ and constitutes a ‘salience network’ in the brain (Menon & Uddin, 2010). The idea of a salience network between the AI and ACC has emerged based on the observation that expansive networks of the brain including the AI-ACC appear to reliably activate during attention, working memory, and cognitive control task performance, while only the AI-ACC appears to be sensitive to stimuli of personal salience (e.g. pain, fear, personal threat; Menon & Uddin, 2010). Further, based on studies of resting-state functional connectivity the AI-ACC appears to constitute its own functional network among regions involved in cognitive control tasks (Seeley et al., 2007). Assuming that detection of salient stimuli is relevant for attention to the environment as well as salient bodily states, Menon and colleagues (2010) propose that the AI-ACC network is responsible for detecting salient stimuli (whether outside or inside the body) and rapidly integrating this information with action-plans that are executed through working memory and attention networks. In the current study we observed object-sensitive and graded (matched > related, ROI-only) activation of the AI-ACC, while only observing a significant relationship with performance in the AI. The fact that no response was required to distracter arrays in the current study may account for the lack of a relationship between ACC response and related and matched object performance costs. However, the specific response of the ACC to filled arrays, and the graded increase in response to matched over related objects, indicates that this region may be acting together with other regions such as the AI in the detection of visual salience in the current study. 85 Control and Storage of VWM Representations The results of our current study present a challenge to current neural models of VWM storage. Several recent studies have observed sustained activation of the superior and inferior IPS, as well as the LOC during VWM maintenance (Mitchell & Cusack, 2008; Song & Jiang, 2006; Todd & Marois, 2004; Xu & Chun, 2006, 2009). Additionally, electrophysiological research has observed sustained activation of the posterior cortex during VWM maintenance (contralateral delay activity in Vogel & Machizawa, 2004; Vogel et al., 2005). In both cases (fMRI and EEF), the amplitude of sustained activity appeared to monotonically scale with estimates of the number of objects maintained in VWM (e.g. Cowan’s k in Rouder et al., 2011). While many have taken this as evidence that the posterior parietal cortex is responsible for the storage of visual objects (Mitchell & Cusack, 2008; Song & Jiang, 2006; Todd & Marois, 2004; Xu & Chun, 2006, 2009), others have argued that this activity reflects only the attentive refreshing process proposed to maintain representations in VWM (Magen, Emmanouil, McMains, Kastner, & Treisman, 2009). Our current study was designed to be sensitive to fast event-related activations, and accordingly we chose a deconvolution approach to fMRI analysis. Due to high degree of BOLD overlap between stages of our task, as well as the choice to include a maintenance variable set to a jittered midpoint of maintenance during no distraction trials, our analysis was not sensitive to identify regions showing sustained activity during maintenance. On the other hand, our design proved to be efficient at capturing activity above and beyond any sustained activation. In the current study, we observed parametric increases in activation within both the IPS and LOC during the presentation of empty, related, and matched object arrays. This type of graded increase over levels of our design mimics the monotonic increase observed with increases in 86 storage demand (Todd & Marois, 2005; Xu & Chun, 2006). While this increase does not explicitly rule out the possibility that object representations are stored in the parietal cortex, it strongly suggests activity-level in the IPS is not explicitly linked to the number of objects maintained in VWM (see also Magen et al., 2009). Another theory related to IPS/posterior parietal activation during VWM maintenance is the ‘unnecessary storage’ hypothesis. Studies such as Vogel and Machizawa (2005) and McNab & Klingberg (2008) have taken individual differences in activation levels (fMRI and ERP) in the posterior parietal cortex (PPC) as an index of storage of irrelevant information from study arrays (McNab & Klingberg, 2008; Vogel et al., 2005). For example, individuals who perform more poorly on change detection paradigms (i.e. low working memory capacity individuals) show equivalent activation in the PPC during low-load trials containing to-be-ignored distracters as high working memory capacity individuals on high-load trials (Vogel et al., 2005). These researchers have argued that the increased levels of activation during low-load trials containing distracters indicates that low working memory capacity individuals are storing distracters along with the study items (McNab & Klingberg, 2008; Vogel et al., 2005). One could argue that the increase in activation observed here in the IPS for the related and matched conditions reflects an appending of VWM with distracter objects. However, it is unclear why related objects would increase activity in the IPS less than matched objects as related objects contain more new features than matched objects. Further, matched objects were marked by a significant increase in activation in the IPS without creating a greater loss of performance than related objects. Lastly, the unnecessary storage hypothesis would predict a positive 87 relationship between individual differences in activation of the PPC and performance decrements at test. Critically, we observed no such relationship. A recent study by McNab & Klingberg (2008) has also indicated that unnecessary storage can be predicted by a failure to engage a working memory gating mechanism implemented by the basal ganglia (BG; McNab & Klingberg, 2008). Indeed, the possibility of a BG-mediated gating mechanism has gained considerable attention in theories of attentional control (Awh & Vogel, 2008) and executive function (Hazy et al., 2007). We observed a significant modulation of the same aspect of the left BG, the left Pallidum, as McNab & Klingberg (2008) in our study. However, the direction of the relationship between activation in the left Pallidum and performance cost at test (for related objects) was the opposite of what would be predicted by their theory. In their study, higher working memory capacity individuals showed greater activation of the left Pallidum during presentation of instructional cues (e.g. ‘attend only yellow’). Further, increase in activation in the BG resulted in reduction of their parietal index of unnecessary storage (e.g. difference in IPS activity for target + distracter study arrays minus load-equated target-only study arrays). They took this relationship as evidence that the BG may play a key role in gating unnecessary information from VWM representation. In our study we found that increases in activation of the left Pallidum resulted in greater performance costs at test for related objects. Although activation in the left Pallidum was overall greater for matched than related objects, we found no relationship with performance for matched objects. This may be an indication that matched objects disrupt performance through a different mechanism than related objects. Alternatively, the relationship observed for related objects may reflect larger individual differences in vulnerability to related object disruption. In either 88 case, the observation of a positive relationship between left Pallidum activation and performance costs at test indicates that while activation of the left Pallidum may be facilitative of efficient gating during encoding, it may be detrimental to VWM maintenance. On the other hand, activation in visual association cortex during related and matched arrays was positively related to performance at test (see LOC/Occ Fus above). Primary and associative visual cortices are thought to be the target rather than a source of control signals during selective attention processing (Yantis, 2008). Thus, we would argue that an increase in LOC activation during presentation of distraction arrays may have reflected an increase in control signaling originating in other areas of the dorsal parietal cortex. However, the positive relationship between LOC activation and performance decrements suggests that exposure to related and matched arrays may have resulted in an increase or a modification to the number of active features in VWM. Based on work on monkey physiology (Tanaka, 1996), modal maintenance models of human visual object representation propose that short-term representation of objects occurs through simulation of an object’s features across regions of primary sensory cortex (see Raffone & Wolters, 2001). In such models, object- or groupinglevel indices that relate information about an object as a whole (e.g. the object’s identity/semantic relationships) are thought to be decoded by larger receptive-field neurons in the temporal cortex. However, cortical oscillations between these neurons and small receptive-field neurons in early visual cortex are thought to resolve the basic features associated with a maintained object. Importantly, several recent studies have demonstrated that visual features maintained in VWM can be decoded from activation patterns in early visual cortex (Ester, Serences, & Awh, 2009; Harrison & Tong, 2009; Serences, Ester, Vogel, & Awh, 89 2009). If the relationship between activation in the LOC and performance costs at test is an indication of features being appended to VWM, then the results of the current brain imaging study may provide a basis for our theory of how distracter objects disrupt binding change detection performance. That is, exposure to new objects containing feature-overlap with the current or recently active contents of VWM may retrieve old representations back into the focus of attention (e.g. related objects effectively re-activate old traces) or cause existing object representations to be associated with new visual features (e.g. matched objects cause rebinding). Although this theory is based on several behavioral accounts of memory carry-over effects (e.g. proactive interference in Craig et al., 2013; Jonides & Nee, 2006; Kane & Engle, 2000; Lustig et al., 2001; May et al., 1999; Monsell, 1978; Nelson et al., 2009) and binding confusion (e.g. Alvarez & Thompson, 2009; Fiacconi & Milliken, 2012c; Kahneman et al., 1992; Wheeler & Treisman, 2002) further brain imaging research would be necessary to make a stronger case for our theory. Conclusions The results of the current study suggest that binding change detection performance is disrupted by visual sensory information that matches the current or recently active contents of VWM. Feature-overlap between VWM representation and ongoing sensory input during maintenance appears to contingently capture visual attention and disrupt the ability to make a binding change decisions at test. Individual differences in activation observed in the right AI and right LOC/Occ Fus during exposure to objects containing feature-overlap with recently (related objects) and currently (matched objects) active VWM traces was predictive of the performance cost observed at test for these conditions. Overall, while providing support for 90 our theory that the contents of VWM may be disrupted through the contingent capture of attention, other effects observed in the current study represent a challenge to existing theories of VWM representation. The parametric increase in activation observed in the IPS suggests that activity in these regions provides more information than the number of objects maintained in VWM. Further, our results indicate that activation in subcortical structures such as the left Pallidum, while facilitative of performance during VWM encoding, may be detrimental during VWM maintenance. Overall, by varying the feature content of distracting information during VWM we were able to reveal the mechanisms through which binding change detection is disrupted and raise new questions about the contribution parietal and basal ganglia structures make to VWM maintenance. 91 CHAPTER 4 GENERAL DISCUSSION In the previous chapters, across two experimental platforms (both a behavioral individual differences approach and a functional brain imaging approach) we evaluated the mechanisms through which VWM for bound multi-feature objects is disrupted. The experimental design for the current study was partly motivated by the designs utilized by other researchers in the field (Johnson et al., 2008; Ueno, Allen et al., 2011) and was chosen as a means to test the previously proposed executive gating hypothesis. As a starting point, it was important to demonstrate that some types of visual information are reliably capable of disrupting VWM maintenance. In our behavioral work, this proved to be the case with objects that contain feature overlap with the contents of VWM. On the other hand, objects referred to as “plausible” study objects and found by other researchers to be particularly disruptive of memory for multi-feature objects (Ueno, Allen et al., 2011) were not as capable of producing a reliable distraction effect in our study. On one level this finding in Chapter 2 was disappointing, as the inconsistency seemed to indicate that these ‘related’ objects may not disrupt VWM for the same reason as the matched objects. Importantly, the results of the fMRI study presented in Chapter 3 seemed to indicate that they may indeed as distraction-costs associated with each condition were related to activation in a common neural region (the right anterior insula). However, the critical contribution that this set of studies makes to the body of existing knowledge about attentional control and VWM is that disruption of VWM maintenance for bound features may be caused by a special case of stimulus-driven contingent attention capture rather than a failure to 92 voluntarily ignore/gate ongoing visual sensory input at the percept. That is, the control of attention may be harnessed involuntarily by a correspondance between the active contents of working memory and a stimulus present in the environment (similar to Soto et al., 2008). In the case of feature-matched distracter objects, this was the currently active features in VWM; in the case of distracter objects containing ‘study-related’ features, this was the recently active contents of VWM. The observation that objects containing partial feature overlap with the study objects are capable of reliably disrupting binding change detection performance throughout the maintenance interval served as an initial benchmark to test the gating hypothesis. As an external criterion measure of executive attentional control we chose complex span performance given its previous validation as a measure of executive attention, particularly the ability to maintain information in working memory in the face of a competing task (Engle, 2002). We expected that high working memory capacity individuals would be better able to resist disruption than low capacity individuals in our sample. We however found no such relationship. This result may have emerged for several reasons. Our preferred explanation was that distraction-costs in the BDT are not caused by a failure to control attention; rather they result from the capture of attention. However, while this hypothesis is supported by the results of our fMRI study it is important to note two major caveats to this explanation. First, while we chose two versions of the complex span task as a measure of attentional control due to its previous validation as such (Engle, 2002) it is possible that these tasks do not tap the key executive gating construct of interest. Alternatively, it is possible that flanker or Stroop task performance may serve as a more direct measure of executive filtering/gating. 93 Indeed, as the results of our individual differences analysis amount to a null result it would be important to identify a criterion measure to which BDT distraction-costs do relate. For example, if Stroop performance was found to be unrelated but release time from capture during visual search did reliably predict BDT distraction-costs a more compelling case for our hypothesis could be made. Second, performance on our behavioral task was overall particularly poor. In Chapter 2, twenty-four percent of participants in Experiment 1 and twenty-two percent of participants in Experiment 2 were excluded for having near-chance performance without distraction. While it seems easily justifiable to require above-chance performance without distraction for all participants included in an analysis of distraction effects, the overall low performance without distraction is an indication of a poorly calibrated task. We proceeded with a VWM load-level of four objects per trial throughout our behavioral study, as several previous studies of VWM for bound features had required the same (Johnson et al., 2008; Ueno, Allen et al., 2011). In our study, this VWM load may have been too difficult for more than 20% of our sample in each experiment. Alternatively, this rate of chance performance may reflect the proportion of individuals who self-dismissed their participation in the experiment. In either case, while exclusion did not impact the individual differences calculations (no relationship between WMC and distraction-cost was observed with inclusion of all participants) poor calibration of the task could have selectively prevented us from sampling true distraction-costs from the lower performing individuals. In other words, with a lower VWM load we may have been able to obtain estimates of distractibility from the lowest WMC individuals, assuming that low WMC is 94 the key feature that prevented them from reaching above-chance performance without distraction. While the exclusion criterion in Chapter 2 was based on requiring better than chance performance without distraction (at least one confidence interval better than chance), it could be argued that a 25% corrected recognition rate was particularly high given that the group mean after exclusion was short of 50%. To explore the effect of a more conservative criterion several follow up analyses were conducted. Including the poorest performing participants in the final analysis of both experiments resulted in less than a 40% mean corrected rate without distraction and a loss of almost every reliable effect in Chapter 2 Experiment 2 (though the critical empty > related effect in Experiment 1 was spared). Given that having no exclusion criteria in a sample containing 11 participants falling at or within 5% of chance performance (across Experiment 1 and 2) seemed a bit unreasonable, we also explored our results with a more conservative cut-off of 10% or greater corrected recognition without distraction. This criterion led to the inclusion of 10/12 excluded participants from Experiment 1 and 16/25 excluded participants in Experiment 2. With the 10% criterion the overall corrected recognition rate across conditions was reduced, but all effects observed previously with the 25% criterion remained reliable. The pattern of results with different criteria suggests that the poorest performers make a substantial mark on the aggregate effects if left in the group analysis. However, finding a common pattern of reliable results across two inclusion thresholds suggests that the results presented here were not a factor of selective data trimming. It is unclear whether any participants were removed from the other studies mentioned here, specifically Ueno et al. In fact, after removal of the chance performers in our study the 95 corrected recognition rate in both experiments without distraction was nearly identical to that observed by Ueno and colleagues (2011) as was the magnitude of our distraction-cost effectsize for related objects in Experiment 1 (in replication of their results). Thus, while we would argue that removal of chance performers may have solved a floor performance issue with regard to distraction-cost effects, the problem of not sampling from the entire WMC distribution could prevent the ability to truly test for criterion relationships with such an executive control construct. In the future it may be helpful to calibrate the task to each individual’s abilities without distraction before proceeding to individual differences analysis. In proceeding to an fMRI study on this topic we chose to reduce the VWM load to three objects per trial, given the poor performance in the behavioral study. This was done to minimize the cost (both temporally and monetarily) associated with having to remove so many participants from the final analysis. While this change was successful in reducing our attrition rate, the change in VWM load did impact the pattern of distraction costs. With three objects the empty arrays no longer produced a reliable distraction-cost. Further, the matched condition was no longer more disruptive than the related condition. The lack of a cost associated with the empty condition is important because it invalidates the potential criticism that the empty outlines are shaped like squares, which based on our theory of how related objects disrupt VWM could potentially account for the disruptiveness of empty arrays in Chapter 2. Remarkably though, despite not resulting in a significant distraction-cost the onset of the empty arrays was selectively registered by regions of the DAN and critically not the VAN. As was outlined in the introduction to Chapter 3, this boundary condition is imperative to our hypothesis. If empty arrays were to activate the VAN as well, the case that these regions are 96 selective to contingent rather than pure capture is weakened. The fact that they did not incur a distraction-cost however, does not allow us to draw a strong conclusion about whether any universally salient distracter object (perhaps an emotional or extremely bright stimulus) that does result in a cost to BDT performance would also fail to activate the VAN. Based on the literature reviewed in Chapter 3, we expect that such a stimulus would generally not activate the VAN; however, we are aware of at least one study that can substantiate the claim that a surprising yet behaviorally completely irrelevant stimulus would activate the VAN (Asplund, Todd, Snyder, Gilbert, & Marois, 2010). Thus, the boundary between the strong claims made here that the VAN should not respond to universally salient, but only to contingently salient, visual events requires future investigation utilizing parametric within-subject experimental manipulations such as we have presented. In conclusion, this doctoral dissertation was initially proposed as a study of VWM executive gating. Based on existing studies and the hypothesis space surrounding the topic, we expected that the current research design would be capable of tapping individual differences in a VWM gating mechanism and would reveal the neural mechanisms responsible for preventing distraction during VWM maintenance. After it was carried out this study presented a case where the observed data did not fit the existing theory, and we were required to shift theoretical perspectives accordingly. The key features of binding change detection tasks like the BDT and suffix task, unlike single feature change detection, create a context in which memory for intra-object feature relationships can be evaluated under competing attentional demands. The type of disruption effect observed in the BDT and elsewhere, in the absence of evidence to the contrary, is fairly and easily relegated to the failure of an executive process like 97 executive gating/control of attention. However, in convergence with other task environments commonly deemed as measures of executive function such as task switching (see Kiesel et al., 2010; Logan, Schneider, & Bundesen, 2007), an interaction between low-level stimulus features at the percept and active memory traces may be capable of explaining much if not all of the effects observed. After reviewing the results of both studies it appeared that such a mechanism, with minimal caveat, could account for our observations. That is, the focus of attention is in part controlled by dynamic interactions between executive control settings (e.g. the active memory traces voluntarily encoded into VWM) and the information provided by the environment (e.g. the features at the percept). Taken together, the results of the current study suggest that this dynamic is responsible for disruption of VWM maintenance for bound features. 98 REFERENCES 99 REFERENCES Allen, R. J., Baddeley, A. D., & Hitch, G. J. (2006). Is the binding of visual features in working memory resource-demanding? J Exp Psychol Gen, 135(2), 298-313. Alvarez, G. A., & Cavanagh, P. (2004). The capacity of visual short-term memory is set both by visual information load and by number of objects. Psychol Sci, 15(2), 106-111. Alvarez, G. A., & Thompson, T. W. (2009). Overwriting and rebinding: tasks underestimate the binding capacity of visual working memory. Vis cogn, 17(1), 141-159. Anticevic, A., Repovs, G., Shulman, G. L., & Barch, D. M. (2010). When less is more: TPJ and default network deactivation during encoding predicts working memory performance. Neuroimage, 49(3), 2638-2648. Arrington, C. M., Carr, T. H., Mayer, A. R., & Rao, S. M. (2000). Neural mechanisms of visual attention: object-based selection of a region in space. J Cogn Neurosci, 12 Suppl 2, 106117. Asplund, C. L., Todd, J. J., Snyder, A. P., Gilbert, C. M., & Marois, R. (2010). Surprise-induced blindness: a stimulus-driven attentional limit to conscious perception. J Exp Psychol Hum Percept Perform, 36(6), 1372-1381. Awh, E., Anllo-Vento, L., & Hillyard, S. A. (2000). The role of spatial selective attention in working memory for locations: evidence from event-related potentials. J Cogn Neurosci, 12(5), 840-847. Awh, E., Barton, B., & Vogel, E. K. (2007). Visual working memory represents a fixed number of items regardless of complexity. Psychol Sci, 18(7), 622-628. Awh, E., & Vogel, E. K. (2008). The bouncer in the brain. Nat Neurosci, 11(1), 5-6. Baddeley, A. (1984). The fractionation of human memory. Psychol Med, 14(2), 259-264. Badre, D., & Wagner, A. D. (2004). Selection, integration, and conflict monitoring; assessing the nature and generality of prefrontal cognitive control mechanisms. Neuron, 41(3), 473487. Barton, B., Ester, E. F., & Awh, E. (2009). Discrete resource allocation in visual working memory. J Exp Psychol Hum Percept Perform, 35(5), 1359-1367. Bays, P. M., Catalao, R. F., & Husain, M. (2009). The precision of visual working memory is set by allocation of a shared resource. J Vis, 9(10), 7 1-11. 100 Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychol Rev, 108(3), 624-652. Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004). Conflict monitoring and anterior cingulate cortex: an update. Trends Cogn Sci, 8(12), 539-546. Braver, T. S., Barch, D. M., Gray, J. R., Molfese, D. L., & Snyder, A. (2001). Anterior cingulate cortex and response conflict: effects of frequency, inhibition and errors. Cereb Cortex, 11(9), 825-836. Brown, J. W., & Braver, T. S. (2005). Learned predictions of error likelihood in the anterior cingulate cortex. Science, 307(5712), 1118-1121. Brown, J. W., & Braver, T. S. (2008). A computational model of risk, conflict, and individual difference effects in the anterior cingulate cortex. Brain Res, 1202, 99-108. Chang, C. F., Hsu, T. Y., Tseng, P., Liang, W. K., Tzeng, O. J., Hung, D. L., et al. (2012). Right temporoparietal junction and attentional reorienting. Hum Brain Mapp, 34(4), 869-877. Chiu, Y. C., & Yantis, S. (2009). A domain-independent source of cognitive control for task sets: shifting spatial attention and switching categorization rules. J Neurosci, 29(12), 39303938. Chun, M. M. (2011). Visual working memory as visual attention sustained internally over time. Neuropsychologia, 49(6), 1407-1409. Chun, M. M., & Johnson, M. K. (2011). Memory: enduring traces of perceptual and reflective attention. Neuron, 72(4), 520-535. Cole, M. W., & Schneider, W. (2007). The cognitive control network: Integrated cortical regions with dissociable functions. Neuroimage, 37(1), 343-360. Corbetta, M., Kincade, J. M., & Shulman, G. L. (2002). Neural systems for visual orienting and their relationships to spatial working memory. J Cogn Neurosci, 14(3), 508-523. Corbetta, M., Patel, G., & Shulman, G. L. (2008). The reorienting system of the human brain: from environment to theory of mind. Neuron, 58(3), 306-324. Cowan, N. (1999). An embedded-process model of working memory. In A. Miyake & P. Shah (Eds.), Models of working memory: Mechanisms of active maintenance and executive control (pp. 62-101). New York, NY: Cambridge University Press. Cowan, N. (2001). The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behav Brain Sci, 24(1), 87-114; discussion 114-185. 101 Cowan, N., Elliott, E. M., Scott Saults, J., Morey, C. C., Mattox, S., Hismjatullina, A., et al. (2005). On the capacity of attention: its estimation and its role in working memory and cognitive aptitudes. Cogn Psychol, 51(1), 42-100. Craig, A. D., Chen, K., Bandy, D., & Reiman, E. M. (2000). Thermosensory activation of insular cortex. Nat Neurosci, 3(2), 184-190. Craig, K. S., Berman, M. G., Jonides, J., & Lustig, C. (2013). Escaping the recent past: Which stimulus dimensions influence proactive interference? Mem Cognit. Curtis, C. E. (2006). Prefrontal and parietal contributions to spatial working memory. Neuroscience, 139(1), 173-180. de Fockert, J., Rees, G., Frith, C., & Lavie, N. (2004). Neural correlates of attentional capture in visual search. J Cogn Neurosci, 16(5), 751-759. Delvenne, J. F., & Bruyer, R. (2004). Does visual short-term memory store bound features? Visual Cognition, 11(1), 1-27. Delvenne, J. F., Cleeremans, A., & Laloyaux, C. (2010). Feature bindings are maintained in visual short-term memory without sustained focused attention. Exp Psychol, 57(2), 108-116. Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annu Rev Neurosci, 18, 193-222. Engle, R. W. (2002). Working memory capacity as executive attention. Current Directions in Psychological Science, 11(1), 19-23. Ester, E. F., Serences, J. T., & Awh, E. (2009). Spatially global representations in human primary visual cortex during working memory maintenance. J Neurosci, 29(48), 15258-15265. Esterman, M., Chiu, Y. C., Tamber-Rosenau, B. J., & Yantis, S. (2009). Decoding cognitive control in human parietal cortex. Proc Natl Acad Sci U S A, 106(42), 17974-17979. Fiacconi, C. M., & Milliken, B. (2012a). Contingency blindness: location-identity binding mismatches obscure awareness of spatial contingencies and produce profound interference in visual working memory. Mem Cognit, 40(6), 932-945. Fiacconi, C. M., & Milliken, B. (2012b). On the role of attention in generating explicit awareness of contingent relations: evidence from spatial priming. Conscious Cogn, 20(4), 14331451. Fiacconi, C. M., & Milliken, B. (2012c). Visual memory for feature bindings: The disruptive effect of responding to new perceptual input. Q J Exp Psychol (Hove). 102 Folk, C. L., Leber, A. B., & Egeth, H. E. (2008). Top-down control settings and the attentional blink: Evidence for nonspatial contingent capture. Vis cogn, 16(5), 616-642. Folk, C. L., Remington, R. W., & Johnston, J. C. (1992). Involuntary covert orienting is contingent on attentional control settings. J Exp Psychol Hum Percept Perform, 18(4), 1030-1044. Folk, C. L., Remington, R. W., & Wright, J. H. (1994). The structure of attentional control: contingent attentional capture by apparent motion, abrupt onset, and color. J Exp Psychol Hum Percept Perform, 20(2), 317-329. Fougnie, D., & Marois, R. (2009). Attentive Tracking Disrupts Feature Binding in Visual Working Memory. Vis cogn, 17(1-2), 48-66. Fukuda, K., & Vogel, E. K. (2009a). Human variation in overriding attentional capture. J Neurosci, 29(27), 8726-8733. Fukuda, K., & Vogel, E. K. (2009b). Individual differences in recovery time from attentional capture. Psychol Sci, 22(3), 361-368. Fukuda, K., & Vogel, E. K. (2011). Individual differences in recovery time from attentional capture. Psychol Sci, 22(3), 361-368. Gajewski, D. A., & Brockmole, J. R. (2006). Feature bindings endure without attention: evidence from an explicit recall task. Psychon Bull Rev, 13(4), 581-587. Gallese, V., Keysers, C., & Rizzolatti, G. (2004). A unifying view of the basis of social cognition. Trends Cogn Sci, 8(9), 396-403. Hakun, J. G., & Ravizza, S. M. (2012). Cognitive control: preparation of task switching components. Brain Res, 1451, 53-64. Harrison, S. A., & Tong, F. (2009). Decoding reveals the contents of visual working memory in early visual areas. Nature, 458(7238), 632-635. Hartshorne, J. K. (2008). Visual working memory capacity and proactive interference. PLoS One, 3(7), e2716. Hazy, T. E., Frank, M. J., & O'Reilly R, C. (2007). Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system. Philos Trans R Soc Lond B Biol Sci, 362(1485), 1601-1613. Hitch, G. J. (1975). The role of attention in visual and auditory suffix effects. Mem Cognit, 3(5), 501-505. Hollingworth, A. (2005). The relationship between online visual representation of a scene and long-term scene memory. J Exp Psychol Learn Mem Cogn, 31(3), 396-411. 103 Huang, L., & Pashler, H. (2007). Working memory and the guidance of visual attention: consonance-driven orienting. Psychon Bull Rev, 14(1), 148-153. Huang, L., Treisman, A., & Pashler, H. (2007). Characterizing the limits of human visual awareness. Science, 317(5839), 823-825. Ikkai, A., & Curtis, C. E. (2008). Cortical activity time locked to the shift and maintenance of spatial attention. Cereb Cortex, 18(6), 1384-1394. Irwin, D. E., Zacks, J. L., & Brown, J. S. (1990). Visual memory and the perception of a stable visual environment. Percept Psychophys, 47(1), 35-46. Jackson, P. L., Meltzoff, A. N., & Decety, J. (2005). How do we perceive the pain of others? A window into the neural processes involved in empathy. Neuroimage, 24(3), 771-779. Jenkinson, M., & Smith, S. (2001). A global optimisation method for robust affine registration of brain images. Med Image Anal, 5(2), 143-156. Johnson, J. S., Hollingworth, A., & Luck, S. J. (2008). The role of attention in the maintenance of feature bindings in visual short-term memory. J Exp Psychol Hum Percept Perform, 34(1), 41-55. Jonides, J., & Nee, D. E. (2006). Brain mechanisms of proactive interference in working memory. Neuroscience, 139(1), 181-193. Kahneman, D., Treisman, A., & Gibbs, B. J. (1992). The reviewing of object files: object-specific integration of information. Cogn Psychol, 24(2), 175-219. Kane, M. J., & Engle, R. W. (2000). Working-memory capacity, proactive interference, and divided attention: limits on long-term memory retrieval. J Exp Psychol Learn Mem Cogn, 26(2), 336-358. Kane, M. J., & Engle, R. W. (2003). Working-memory capacity and the control of attention: the contributions of goal neglect, response competition, and task set to Stroop interference. J Exp Psychol Gen, 132(1), 47-70. Kane, M. J., Poole, B. J., Tuholski, S. W., & Engle, R. W. (2006). Working memory capacity and the top-down control of visual search: Exploring the boundaries of "executive attention". J Exp Psychol Learn Mem Cogn, 32(4), 749-777. Kastner, S., Pinsk, M. A., De Weerd, P., Desimone, R., & Ungerleider, L. G. (1999). Increased activity in human visual cortex during directed attention in the absence of visual stimulation. Neuron, 22(4), 751-761. Kiesel, A., Steinhauser, M., Wendt, M., Falkenstein, M., Jost, K., Philipp, A. M., et al. (2010). Control and interference in task switching--a review. Psychol Bull, 136(5), 849-874. 104 Kincade, J. M., Abrams, R. A., Astafiev, S. V., Shulman, G. L., & Corbetta, M. (2005). An eventrelated functional magnetic resonance imaging study of voluntary and stimulus-driven orienting of attention. J Neurosci, 25(18), 4593-4604. Kiyonaga, A., & Egner, T. (2012). Working memory as internal attention: toward an integrative account of internal and external selection processes. Psychon Bull Rev, 20(2), 228-242. Kurth, F., Zilles, K., Fox, P. T., Laird, A. R., & Eickhoff, S. B. (2010). A link between the systems: functional differentiation and integration within the human insula revealed by metaanalysis. Brain Struct Funct, 214(5-6), 519-534. Lavie, N., Hirst, A., de Fockert, J. W., & Viding, E. (2004). Load theory of selective attention and cognitive control. J Exp Psychol Gen, 133(3), 339-354. Lee, D., & Chun, M. M. (2001). What are the units of visual short-term memory, objects or spatial locations? Percept Psychophys, 63(2), 253-257. Linden, D. E., Prvulovic, D., Formisano, E., Vollinger, M., Zanella, F. E., Goebel, R., et al. (1999). The functional neuroanatomy of target detection: an fMRI study of visual and auditory oddball tasks. Cereb Cortex, 9(8), 815-823. Liu, T., Slotnick, S. D., Serences, J. T., & Yantis, S. (2003). Cortical mechanisms of feature-based attentional control. Cereb Cortex, 13(12), 1334-1343. Logan, G. D., Schneider, D. W., & Bundesen, C. (2007). Still clever after all these years: searching for the homunculus in explicitly cued task switching. J Exp Psychol Hum Percept Perform, 33(4), 978-994. Logie, R. H., & Brockmole, J. R. (2009). Bound feature combinations in visual short-term memory are fragile but influence long-term learning. Vis cogn, 17(1), 160-179. Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390(6657), 279-281. Lustig, C., May, C. P., & Hasher, L. (2001). Working memory span and the role of proactive interference. J Exp Psychol Gen, 130(2), 199-207. Magen, H., Emmanouil, T. A., McMains, S. A., Kastner, S., & Treisman, A. (2009). Attentional demands predict short-term memory load response in posterior parietal cortex. Neuropsychologia, 47(8-9), 1790-1798. Majerus, S., Attout, L., D'Argembeau, A., Degueldre, C., Fias, W., Maquet, P., et al. (2011). Attention supports verbal short-term memory via competition between dorsal and ventral attention networks. Cereb Cortex, 22(5), 1086-1097. 105 Makovski, T., & Jiang, Y. V. (2008). Proactive interference from items previously stored in visual working memory. Mem Cognit, 36(1), 43-52. Makovski, T., Shim, W. M., & Jiang, Y. V. (2006). Interference from filled delays on visual change detection. J Vis, 6(12), 1459-1470. Mance, I., Becker, M. W., & Liu, T. (2012). Parallel consolidation of simple features into visual short-term memory. J Exp Psychol Hum Percept Perform, 38(2), 429-438. May, C. P., Hasher, L., & Kane, M. J. (1999). The role of interference in memory span. Mem Cognit, 27(5), 759-767. McNab, F., & Klingberg, T. (2008). Prefrontal cortex and basal ganglia control access to working memory. Nat Neurosci, 11(1), 103-107. Meiran, N., Kessler, Y., & Adi-Japha, E. (2008). Control by action representation and input selection (CARIS): a theoretical framework for task switching. Psychol Res, 72(5), 473500. Menon, V., & Uddin, L. Q. (2010). Saliency, switching, attention and control: a network model of insula function. Brain Struct Funct, 214(5-6), 655-667. Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annu Rev Neurosci, 24, 167-202. Mitchell, D. J., & Cusack, R. (2008). Flexible, capacity-limited activity of posterior parietal cortex in perceptual as well as visual short-term memory tasks. Cereb Cortex, 18(8), 1788-1798. Monsell, S. (1978). Recenct, immediate recognition memory, and reaction time. Cogn Psychol, 10(4), 465-501. Nelson, J. K., Reuter-Lorenz, P. A., Persson, J., Sylvester, C. Y., & Jonides, J. (2009). Mapping interference resolution across task domains: a shared control process in left inferior frontal gyrus. Brain Res, 1256, 92-100. Oberauer, K., Suss, H.-M., Wilhelm, O., & Wittman, W. W. (2003). The multiple faces of working memory: Storage, processing, supervision, and coordination. Intelligence, 31(2), 167193. Olivers, C. N., Meijer, F., & Theeuwes, J. (2006). Feature-based memory-driven attentional capture: visual working memory content affects visual attention. J Exp Psychol Hum Percept Perform, 32(5), 1243-1265. Olivers, C. N., Peters, J., Houtkamp, R., & Roelfsema, P. R. (2011). Different states in visual working memory: when it guides attention and when it does not. Trends Cogn Sci, 15(7), 327-334. 106 Olson, I. R., & Jiang, Y. (2002). Is visual short-term memory object based? Rejection of the "strong-object" hypothesis. Percept Psychophys, 64(7), 1055-1067. Owen, A. M., McMillan, K. M., Laird, A. R., & Bullmore, E. (2005). N-back working memory paradigm: a meta-analysis of normative functional neuroimaging studies. Hum Brain Mapp, 25(1), 46-59. Phan, K. L., Wager, T., Taylor, S. F., & Liberzon, I. (2002). Functional neuroanatomy of emotion: a meta-analysis of emotion activation studies in PET and fMRI. Neuroimage, 16(2), 331348. Poole, B. J., & Kane, M. J. (2009). Working-memory capacity predicts the executive control of visual search among distractors: the influences of sustained and selective attention. Q J Exp Psychol (Hove), 62(7), 1430-1454. Posner, M. I., Klein, R., Summers, J., & Buggie, S. (1973). On the selection of signals. Mem Cognit, 1(1), 2-12. Raffone, A., & Wolters, G. (2001). A cortical mechanism for binding in visual working memory. J Cogn Neurosci, 13(6), 766-785. Redick, T. S., & Thomas, S. (2012). Measuring working memory capacity with automated complex span tasks. European J of Psych Ass, 28(3), 164-171. Rouder, J. N., Morey, R. D., Morey, C. C., & Cowan, N. (2011). How to measure working memory capacity in the change detection paradigm. Psychon Bull Rev, 18(2), 324-330. Scholl, B. J. (2001). Objects and attention: the state of the art. Cognition, 80(1-2), 1-46. Seeley, W. W., Menon, V., Schatzberg, A. F., Keller, J., Glover, G. H., Kenna, H., et al. (2007). Dissociable intrinsic connectivity networks for salience processing and executive control. J Neurosci, 27(9), 2349-2356. Serences, J. T., Ester, E. F., Vogel, E. K., & Awh, E. (2009). Stimulus-specific delay activity in human primary visual cortex. Psychol Sci, 20(2), 207-214. Serences, J. T., Schwarzbach, J., Courtney, S. M., Golay, X., & Yantis, S. (2004). Control of objectbased attention in human cortex. Cereb Cortex, 14(12), 1346-1357. Serences, J. T., Shomstein, S., Leber, A. B., Golay, X., Egeth, H. E., & Yantis, S. (2005). Coordination of voluntary and stimulus-driven attentional control in human cortex. Psychol Sci, 16(2), 114-122. Shipstead, Z., Redick, T. S., Hicks, K. L., & Engle, R. W. (2012). The scope and control of attention as separate aspects of working memory. Memory, 20(6), 608-628. 107 Shomstein, S. (2012). Cognitive functions of the posterior parietal cortex: top-down and bottom-up attentional control. Front Integr Neurosci, 6, 38. Shulman, G. L., Astafiev, S. V., Franke, D., Pope, D. L., Snyder, A. Z., McAvoy, M. P., et al. (2009). Interaction of stimulus-driven reorienting and expectation in ventral and dorsal frontoparietal and basal ganglia-cortical networks. J Neurosci, 29(14), 4392-4407. Smith, S. M. (2002). Fast robust automated brain extraction. Hum Brain Mapp, 17(3), 143-155. Smith, S. M., Jenkinson, M., Woolrich, M. W., Beckmann, C. F., Behrens, T. E., Johansen-Berg, H., et al. (2004). Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage, 23 Suppl 1, S208-219. Song, J. H., & Jiang, Y. (2006). Visual working memory for simple and complex features: an fMRI study. Neuroimage, 30(3), 963-972. Soto, D., Hodsoll, J., Rotshtein, P., & Humphreys, G. W. (2008). Automatic guidance of attention from working memory. Trends Cogn Sci, 12(9), 342-348. Soto, D., & Humphreys, G. W. (2009). Automatic selection of irrelevant object features through working memory: evidence for top-down attentional capture. Exp Psychol, 56(3), 165172. Sperling, G. (1960). The information available in breif visual presentations. Psychological Monographs: General and Applied, 74(11), 1-29. Tanaka, K. (1996). Inferotemporal cortex and object vision. Annu Rev Neurosci, 19, 109-139. Taylor, K. S., Seminowicz, D. A., & Davis, K. D. (2009). Two systems of resting state connectivity between the insula and cingulate cortex. Hum Brain Mapp, 30(9), 2731-2745. Theeuwes, J. (1994). Stimulus-driven capture and attentional set: selective search for color and visual abrupt onsets. J Exp Psychol Hum Percept Perform, 20(4), 799-806. Todd, J. J., & Marois, R. (2004). Capacity limit of visual short-term memory in human posterior parietal cortex. Nature, 428(6984), 751-754. Todd, J. J., & Marois, R. (2005). Posterior parietal cortex activity predicts individual differences in visual short-term memory capacity. Cogn Affect Behav Neurosci, 5(2), 144-155. Tootell, R. B., Dale, A. M., Sereno, M. I., & Malach, R. (1996). New images from human visual cortex. Trends Neurosci, 19(11), 481-489. Touroutoglou, A., Hollenbeck, M., Dickerson, B. C., & Feldman Barrett, L. (2012). Dissociable large-scale networks anchored in the right anterior insula subserve affective experience and attention. Neuroimage, 60(4), 1947-1958. 108 Treisman, A. (1980). A feature-integration theory of attention. Cognitive Psychology, 12(1), 97136. Treisman, A. (1996). The binding problem. Curr Opin Neurobiol, 6(2), 171-178. Treisman, A. (1998). Feature binding, attention and object perception. Philos Trans R Soc Lond B Biol Sci, 353(1373), 1295-1306. Treisman, A., & Zhang, W. (2006). Location and binding in visual working memory. Mem Cognit, 34(8), 1704-1719. Ueno, T., Allen, R. J., Baddeley, A. D., Hitch, G. J., & Saito, S. (2011). Disruption of visual feature binding in working memory. Mem Cognit, 39(1), 12-23. Ueno, T., Mate, J., Allen, R. J., Hitch, G. J., & Baddeley, A. D. (2011). What goes through the gate? Exploring interference with visual feature binding. Neuropsychologia, 49(6), 15971604. Unsworth, N., Spillers, G. J., & Brewer, G. A. (2012). The contributions of primary and secondary memory to working memory capacity: an individual differences analysis of immediate free recall. J Exp Psychol Learn Mem Cogn, 36(1), 240-247. Vaquero, J. M., Fiacconi, C., & Milliken, B. (2012). Attention, awareness of contingencies, and control in spatial localization: a qualitative difference approach. J Exp Psychol Hum Percept Perform, 36(6), 1342-1357. Vogel, E. K., & Luck, S. J. (2002). Delayed working memory consolidation during the attentional blink. Psychon Bull Rev, 9(4), 739-743. Vogel, E. K., & Machizawa, M. G. (2004). Neural activity predicts individual differences in visual working memory capacity. Nature, 428(6984), 748-751. Vogel, E. K., McCollough, A. W., & Machizawa, M. G. (2005). Neural measures reveal individual differences in controlling access to working memory. Nature, 438(7067), 500-503. Vogel, E. K., Woodman, G. F., & Luck, S. J. (2001). Storage of features, conjunctions and objects in visual working memory. J Exp Psychol Hum Percept Perform, 27(1), 92-114. Vogel, E. K., Woodman, G. F., & Luck, S. J. (2006). The time course of consolidation in visual working memory. J Exp Psychol Hum Percept Perform, 32(6), 1436-1451. Wheeler, M. E., & Treisman, A. M. (2002). Binding in short-term visual memory. J Exp Psychol Gen, 131(1), 48-64. Wolfe, J. (1994). Guided Search 2.0: a revised model of visual search. Psychon Bull Rev, 1, 202238. 109 Wolfe, J. M., & Bennett, S. C. (1997). Preattentive object files: shapeless bundles of basic features. Vision Res, 37(1), 25-43. Wolfe, J. M., Cave, K. R., & Franzel, S. L. (1989). Guided search: an alternative to the feature integration model for visual search. J Exp Psychol Hum Percept Perform, 15(3), 419-433. Woodman, G. F., & Vogel, E. K. (2005). Fractionating working memory: consolidation and maintenance are independent processes. Psychol Sci, 16(2), 106-113. Woodman, G. F., & Vogel, E. K. (2008). Selective storage and maintenance of an object's features in visual working memory. Psychon Bull Rev, 15(1), 223-229. Woolrich, M. W., Jbabdi, S., Patenaude, B., Chappell, M., Makni, S., Behrens, T., et al. (2009). Bayesian analysis of neuroimaging data in FSL. Neuroimage, 45(1 Suppl), S173-186. Xu, Y., & Chun, M. M. (2006). Dissociable neural mechanisms supporting visual short-term memory for objects. Nature, 440(7080), 91-95. Xu, Y., & Chun, M. M. (2009). Selecting and perceiving multiple visual objects. Trends Cogn Sci, 13(4), 167-174. Yantis, S. (2008). The Neural Basis of Selective Attention: Cortical Sources and Targets of Attentional Modulation. Curr Dir Psychol Sci, 17(2), 86-90. Yantis, S., Schwarzbach, J., Serences, J. T., Carlson, R. L., Steinmetz, M. A., Pekar, J. J., et al. (2002). Transient neural activity in human parietal cortex during spatial attention shifts. Nat Neurosci, 5(10), 995-1002. Yantis, S., & Serences, J. T. (2003). Cortical mechanisms of space-based and object-based attentional control. Curr Opin Neurobiol, 13(2), 187-193. Yeh, Y., Cheng-Ta, T., & Yu-Chin, C. (2005). Binding or prioritization: The role of selective attention in visual short-term memory. Vis cogn, 12(5), 759-799. Yonelinas, A., Zhang, W., & Shapiro, K. L. (2012). Enhanced familiarity with sequential presentations in visual working memory. J Vis, 12(9), 359-359. Zhang, W., & Luck, S. J. (2008). Discrete fixed-resolution representations in visual working memory. Nature, 453(7192), 233-235. Zhang, W., & Luck, S. J. (2011). The number and quality of representations in working memory. Psychol Sci, 22(11), 1434-1441. 110