DRAFT: This module has unpublished changes.

`EMPIRICAL ARTICLE SUMMARY FORM

Reference (citation)

 

       Aronson, J., Jannone, S., McGlone, M., & Johnson-Campbell, T. (2009). The

             Obama effect: An experimental test. Journal of Experimental Social

             Psychology, 45, 957–960. doi:10.1016/j.jesp.2009.05.006

 

Study Frame

(relevance, background, theory)

 

 

 

 

R) To test the hypothesis that students prompted to think about Barack Obama prior to taking a difficult standardized verbal test would improve their performance relative to white students, and to African American students in control conditions that were not prompted to think about Obama

 

B) Past research on stereotype threat and role model effects, as well as a recent quasi-experiment (Marx, Ho, & Freidman) suggested the possibility of an ‘‘Obama effect” on African American’s standardized test performance, whereby the salience of Barack Obama’s stereotype defying success could positively impact performance.

 

T) Stereotype Threat is the experience of anxiety or concern in a situation where a person has the potential to confirm a negative stereotype about their social group, which can impede a person’s ability to perform to their full capacity.

 

 

 

Research questions/ Hypotheses

 

 

 

 

 

 

RQ) What effect might Mr. Obama’s widely publicized success have on the psychological experiences of black American students in the realm of their intellectual abilities, performances, and efforts.

 

H1) African American test-takers would solve more items if prompted to think about Obama than if prompted to think about McCain, an unspecified politician, or no one at all.

 

H2) The salience manipulation would have no effect on the white test-takers.

 

 

 

 

 

 

 

 

 

Methods

 

Design (experimental, between-subjects, etc.; quasi-experimental; correlational)

This was an experimental, one-way, randomized groups, between-subjects, posttest-only design.

 

Participants

Undergraduate students from three different American universities, of mixed race and gender, and participating in a residential summer program sponsored by the American Association of Medical Colleges for medical school aspirants with approximately 80 students per testing session.

 

What is/are the independent (or predictor) variables?

IV) A brief political survey designed to prompt participants to think about either Barack Obama (Obama condition) or John McCain (McCain condition), or upon neither of the two (control condition).

 

 

What is/are the dependent (or outcome) variables?

DV) Resulting test scores after receiving the experimental manipulation.

 

Procedures (how was the study conducted)

The test given was a verbal section of the Medical College Admissions Test (MCAT) and was administered by a white male who ran the testing session in a manner designed to closely simulate typical group testing. All students in the experiment received the same test instructions, and within each testing session, participants were randomly assigned to receive one of three versions of the test booklet, which contained a cover sheet, the experimental manipulation, the test instructions, and the MCAT test. The manipulation (IV) consisted of a brief political survey being conducted by a colleague of the experimenter.

For students in the Obama condition, three small color photos of Obama were printed on the top of the sheet along with quoted statements from speeches that were chosen to highlight Obama’s success. In the McCain condition Photos of himself in similar contexts as well as quotes highlighting his success were utilized.

In each condition, participants were instructed to indicate on a scale how much they agreed with each of the quoted statements. Next, participants were instructed to think of and list two positive things about the prospect of electing Obama (or McCain) to the presidency. In the control condition, participants were asked to generate two positive things about being politically informed. The manipulation (IV) was designed to induce focus not just upon McCain or Obama, but to prompt an explicit focus on their positive qualities. Following the test, participants answered an additional set of questions aimed at gauging their engagement with and attention to the election.

 

Data Analytic Plan (what statistics were used)

Students’ test performance was computed by summing the number of correct answers. A 2 (race: white or black) x 4 (experimental condition: Obama, McCain, Unspecified Politician, or Control) x 2 (sex: male or female) ANOVA was performed on these scores.

For the participants whose SAT scores were known (n = 71), an analysis of covariance (ANCOVA) was conducted using SAT verbal scores as a covariate.

Participants open-ended positive statements about Obama were coded for level of positivity in which two independent coders (r = .97, Cohen’s k = .92) scored each of the two positive statements before the two scores were summed to create a total positivity score.

 

 

Results/ Findings

 

 

For student scores an ANOVA was performed revealing a significant main effect of race, F(1, 104) = 26.83, p < .01, g2p= .205, with whites (M = 18.68) outperforming

blacks (M = 14.20). A race by sex interaction was also detected F(1, 104) = 6.61, p < .05, g2p = .06, such that white males (M = 20.82) outperformed white females (M = 17.61), while black females (M = 14.67) outperformed black males (M = 12.90).

 

There were no significant effects of presidential condition and no interaction between race and presidential condition (F < 1 in both cases) Black students solved no more problems in the Obama condition (M = 14.5) than they did in the McCain (M = 13.1), unnamed politician (M = 14.5), or stereotype threat control condition (M = 16.0). Analysis of the number of items attempted similarly revealed no effects of the Obama manipulation (F < 1).

 

Looking only at blacks in the Obama condition, test score was uncorrelated with positivity   (r = -.01, p = .98), mentioning race (r = .02, p = .91), mentioning intelligence (r = .23, p = .20). The only significant correlation disconfirmed the logic of the Obama effect: students who reported greater engagement in the election earned lower test scores (r =

.35, p = .05). In a situation that reliably produces stereotype threat effects, this experiment used standardized procedures to get students thinking about Obama in a positive way, yet produced no evidence to suggest that doing so improved their test performance; participants primed to think about Obama performed no better than students in three alternative conditions.

 

 

Discussion/ Implications and Limitations

 

 

 

D) No evidence suggested that greater admiration for Obama was meaningfully associated with better performance, which is curious, given not only the effects obtained from laboratory experiments on black role models (e.g., Marx & Goff, 2005), but even more so, given the encouraging results of the Marx et al.’s quasi-experiment that specifically focused on Obama.

 

L) It is conceivable that giving the exam following the nomination but prior to the election, meant that Obama’s success was insufficient to overcome the stereotype threat experienced by students in our testing sessions. Another possibility is that, given his extraordinary talents, Obama may be perceived as perhaps too innately talented to serve as a threat-buffering role model for the typical African American college student; perhaps his abilities are so stellar that the typical student cannot confidently conclude that ‘‘if Obama can succeed, so can I.” perhaps Obama was seen as an outlier— inspirational, but a bit out of reach as a role model.

 

I) possible selection biases in Marx’s quasi-experiment may have facilitated finding an Obama effect. Specifically, participants in their study responded to advertisements to sit at pre-designated times to take difficult verbal ability tests. This may have restricted their sample to students comfortable being thus evaluated, and perhaps such individuals are more responsive to Obama’s example. Aronson’s sample was quite different; students volunteered for a summer enrichment program which included, among many diverse activities, some test preparation activities of which his group testing was an unadvertised part. In other words, participation in his experiment was not predicated on their feeling comfortable about ability evaluation. They think this may make their sample more representative of the typical Black American student than those recruited to participate in Marx et al. study, and thus they believe their experiment to be a more stringent test of the effect.

 

Marx et al. did not randomly assign their participants to condition. Rather, participants voluntarily chose whether to expose themselves to Obama, pay attention to his speeches, the presidential campaign, and so on. Again this raises the specter of selection bias; perhaps students who were attracted to and inspired by Obama’s success were more likely to show the effect, or to experience less stereotype threat to begin with.

 

 

 

 

 

 

DRAFT: This module has unpublished changes.

EMPIRICAL ARTICLE SUMMARY FORM

Reference (citation)

 

Marx, D. M., Ko, S. J., Friedman, R. A. (2009). The ‘‘Obama Effect”: How a salient    

 

     role model reduces race-based performance differences. Journal of Experimental

 

     Social Psychology, 45, 953–956. doi:10.1016/j.jesp.2009.03.012

Study Frame

(relevance, background, theory)

 

 

 

 

R) The current study was conducted to assess whether, beyond simply inspiring hope, this ‘‘Obama Effect” has a concrete positive influence on Black-Americans’ academic performance.

 

B) When Obama’s stereotype-defying accomplishments garnered national attention – just after his convention speech, and election to the presidency– they had a profound beneficial effect on Black-Americans’ exam performance, such that the negative effects of stereotype threat were dramatically reduced.

 

T) That in-group role models (Obama effect) may be capable of buffering stereotyped targets from the adverse effects of stereotype threat when targets attend to these role models’ counter-stereotypic behaviors.

 

 

 

Research questions/ Hypotheses

 

 

 

 

 

 

RQ) whether Obama’s accomplishments and status as a real world role model can help Black-Americans combat the adverse effects of racial stereotypes and consequently perform better on an academic exam, even under stereotype threat conditions.

 

H1) When Obama’s stereotype-defying successes are pronounced he should boost Black-Americans’ exam performance (i.e., Obama Effect) and thus reduce race-based performance differences even when exams are given under stereotype threat conditions.

 

 

 

 

 

 

 

 

 

 

Methods

 

Design (experimental, between-subjects, etc.; quasi-experimental; correlational)

This was a quasi-experimental, matched assignment, pre-test-post-test design.

 

Participants

N=472 participants (Black-American = 84, White-American = 388)

 

What is/are the independent (or predictor) variables?

Time 1: Race

Time 2: Race and whether they watched Obama speech or didn’t watch speech

Time 3: Race

Time 4: Race

 

 

 

What is/are the dependent (or outcome) variables?

Time 1-4: Verbal exam performance

 

Procedures (how was the study conducted)

At each data collection time participants were recruited via automated email prompts. Once participants logged on to the study website they were informed that the purpose of the study was to examine how people with different experiences solve problems. They were then told about the verbal exam.

 

For each data collection time we activated stereotype threat in two ways. First, participants were told that the verbal exam was ‘‘created by the Massachusetts Aptitude Assessment Center, and is used as a diagnostic tool to assess verbal problem-solving ability.” They were further told that ‘‘the exam is accurate in identifying a person’s intellectual strengths and weaknesses”. Second, participants indicated their race before taking the exam.

 

The verbal exam consisted of 20 problems that were drawn from actual Graduate Record Exams (GREs). The format was identical to the GRE and consisted of three sections: reading comprehension, analogies, and sentence completions. Participants were only allowed 15 min to complete the exam. Scores could range from 0 to 20.

 

To assess whether Black-Americans were concerned about confirming the negative stereotype about their group’s intellectual ability, participants responded to three statements (e.g., ‘‘I worry that if I perform poorly on this test, others will attribute my poor performance to my race” on a 1–7 scale

 

After completing the stereotyped concerns measure and demographics questions, participants received a full written debriefing.

 

Data Analytic Plan (what statistics were used)

Time 1: They analyzed verbal exam performance as a function of participant race, controlling for education level (ANOVA)

Time 2: Exam performance was analyzed using a 2 (Participant Race: Black, White) by 2 (Watched Obama Speech: yes, no) between-participants ANCOVA, controlling for education level.

Time 3: They analyzed exam performance as a function of participant race, controlling for education level (ANOVA)

Time 4: They analyzed verbal exam performance as a function of participant race, controlling for education level (ANOVA)

 

 

 

Results/ Findings

 

 

Time 1: They found a performance difference between White- (M =12.14) and Black-Americans (M = 8.79), F(1, 128) = 14.67, p < .05, g = .32

 

Time 2: Results showed a race main effect, such that White-Americans (M =12.65) performed better than Black-Americans (M= 8.56), F(1, 123) = 17.96, p < .05, g = .36. But more central to our reasoning was the significant race by success salience interaction, F(1, 123) = 5.70, p < .05, g = .21. Simple comparisons revealed that

among those participants who did not watch Obama’s speech (success not salient), White-Americans (M = 13.19) outperformed Black-Americans (M = 6.79), F(1, 123) = 16.74, p < .05, g = .35. However, among those participants who did watch Obama’s speech (success salient)

 

Time 3: They found the expected effect of race, such that White- Americans outperformed Black-Americans (M = 12.89 vs. M = 8.37), F (1, 95) = 10.19, p < .05, g = .32.

 

Time 4: White- and Black-Americans’ performance was equivalent (M = 11.19 vs. M = 9.83), F (1, 112) = 1.01, p = .32, g = .09, demonstrating that Obama’s impact as a role model, due to the salience of his concrete success, again reduced race based

performance differences

 

 

 

Discussion/ Implications and Limitations

 

 

 

D) The Obama Effect embodies the ability of Black-American role models to buffer Black-Americans academic performance from the negative effects of racial stereotypes. Our work has wide applicability because it demonstrates the powerful impact role models have in the real world.

 

I) These findings provide strong evidence for the notion that the Obama Effect can reduce the adverse effects of stereotype threat, even when concerns about racial stereotypes continue to exist.

 

L) The study was conducted online which gave the experimenters little control over their sample. Also there was no mention of how many of the participants were female or male. A gender difference could of affected opinions or feelings of whether Obama was seen as a role model or not and could of affected the obtained results.

 

 

 

Notes

 

 

 

 

 

 

 

 

 

 

DRAFT: This module has unpublished changes.

Darius Muniz

Professor Godfrey

Research Methods II

October 15, 2011

 

I) For Aronson et al. 2009:

1. Describe what makes this an experimental study:

            In Aronson’s study students were randomly assigned to one of three conditions either the Obama condition, McCain condition, or control condition. An independent variable was manipulated in the form of a brief political survey designed to prompt participants to think about either: Barack Obama (Obama condition), John McCain (McCain condition), or upon neither of the two (control condition). The dependent variable, which was the participants resulting test scores, was measured after the manipulation of the independent variable. The experimenters also controlled for possible confounds by administering a test resembling the verbal section of the Medical College Admissions Test that was administered by a white male, who ran the testing session in a manner designed to closely simulate typical group testing, and in which all students in the experiment received the same test instructions. Finally, the resulting data was analyzed utilizing ANOVA and ANCOVA statistics to determine if there were any significant main or interaction effects that would either support or refute their hypotheses.

2. What were the authors’ conclusions (brief summary)?

            The authors found no evidence suggesting that greater admiration for Obama was meaningfully associated with better performance. The authors believed that possibly giving the exam following the nomination but prior to the election meant that Obama’s success was insufficient to overcome stereotype threat. Another possible explanation was that due to his extraordinary talents, Obama may have been perceived as perhaps too naturally talented to serve as a threat-buffering role model for the typical African American college student. They also believed that perhaps Obama was seen as inspirational, but a bit out of reach as a role model for students to relate to. To explain Marx’s obtained results the authors cited that possible selection biases in Marx’s quasi-experiment may have facilitated finding an Obama effect. Participants in the Marx study responded to advertisements to sit at pre-designated times to take difficult verbal ability tests. This may have restricted their sample to people comfortable being evaluated as such, and perhaps such individuals are more responsive to Obama’s example. Aronson’s sample consisted of students volunteering for a summer enrichment program which included, among many diverse activities, some test preparation activities of which his group testing was an unadvertised part. The authors thought this may have made their sample more representative of the typical Black American student than those recruited to participate in Marx et al. study, and thus they believed their experiment to be a more stringent test of the effect.

3. Do you believe them – why or why not? (Think in terms of internal and external validity)

            Aronson and his team claim that their process of participant recruitment was done in such a way that reflected more accurately the typical Black American student. Conversely, Aronson claims that the participants in Marx’s study were obtained in such a way that selection bias may have occurred making their participants less representative of the typical Black American student. This is one of Aronson’s arguments for why Marx was able to achieve the results he did and why Aronson was not. I believe both Aronson and Marx’s samples had poor external validity which affected both of the researcher’s results. Aronson’s participants were pre med students participating in a summer enrichment program who obviously already were above average academically and also were more motivated towards their education which would have made his participants unrepresentative of “the typical Black American student”. This fact alone may have contributed to Aronson’s null findings because his participants may have felt that they didn’t need a role model and were already confident about their ability’s. Aronson does have a valid point that Marx’s participants had selection bias and were unrepresentative of the typical Black American student because they were recruited online, limiting their knowledge of the sample, and were offered a reward for participating, which also would of drawn a certain type of person. Also the fact that he had three times as many white participants as he did Black participants and gives no mention to gender Marx’s participants had poor internal validity. These Factors definitely could have contributed to Marx’s results and gives weight to Aronson’s conclusions of why Marx obtained the results he did.                                                     

II) For Marx et al. 2009:

4. Describe what makes this a quasi-experimental study

            This study was quasi-experimental because participants were not randomly assigned to experimental conditions though they were matched based on age and English proficiency. Also there was no manipulation of an independent variable but the presence of predictor variables being the four different points in time that data was collected from participants. This made this quasi experimental study a comparative time series design where the participants scores were checked for variance due to the salience of Obama’s achievements from prior to nomination to winning the actual election.

 

5. What were the authors’ conclusions (brief summary)?

            The Obama Effect confirms the ability of Black-American role models to buffer Black-Americans academic performance from the negative effects of racial stereotypes and demonstrates the powerful impact role models have in the real world. These findings provide strong evidence for the notion that the Obama Effect can reduce the adverse effects of stereotype threat, even when concerns about racial stereotypes continue to exist. The authors also cited that the differences in scores obtained were affected by different timeframes showing that significant results occurred at different salience levels but were most substantial when Obama’s success was most apparent and concrete during his nomination and post election. 

6. Do you believe them? (Think in terms of internal and external validity)

            The authors provide compelling evidence for the Obama effect by showing that there were differences in assessment scores at different time frames but that participant’s performance were even more equal to their white counterparts at concrete stages in the time collection (e.g. time 2 and time 4). I think that since internal validity and external validity for this sample is weak due to its use of online recruitment, offering a reward for participation, unequal Black to White participant ratio, no mention of the gender differences of their participants, and the way that different samples were utilized for different time periods, these threats to the internal validity of the study casts doubts for me on the actual validity of Marx’s obtained results. This also makes me ask myself if indeed all these threats to internal validity were the actual causes of Marx’s supporting evidence for the Obama effect instead of Obama’s actual ability to buffer stereotype threat due to his status as a role model.

7. What’s your take home message about the influence of role models on stereotype threat? What would you tell your friend?

            I believe that role models definitely moderate the motivation of people taking tests under conditions of stereotype threat. I think that for the evidence to be more concrete to the point of us being able to say that having a role model to buffer the effects of stereotype threat causes an increase in test scores and allows members of the stereotyped group to be more comfortable during these tests, more controlled and internally valid studies must be conducted. I would tell my friend that you should never judge a book by its cover. Although you have studies that imply one message or another and are very convincing, you should always read between the lines and analyze the research to ensure that all proper procedures were followed to make the study valid. Only then can we be confident that the results obtained in the study and the message it implies is actually worthy of being taken as fact. Also we know that the theory of stereotype threat is widely accepted as fact and that the Obama affect is being tested as a possible buffer to minimize these feelings of threat. We know that one article (Marx) found that the Obama affect through role models can increase test scores regardless of this threat and that another article (Aronson) refutes this finding by giving weight to its own contradictory findings. You must realize that the conditions tested under each study are very different, the samples were obtained differently, the samples were comprised of different individuals, and the results were analyzed differently. These facts make Aronson’s study a poor replication of Marx’s original study and therefore in my opinion could not be used as a valid refutation of Marx’s findings. 

 

 

DRAFT: This module has unpublished changes.

Darius Muniz

Professor Godfrey

Research Methods II

October 23rd, 2011

 

I. Indicate the title, date and source of the article in APA stlye.

Ojalvo Epstein, H. (2011, October 21). Q and A How facebook use correlates with student     

     outcomes. The New York Times. Retrieved from http://www.NYtimes.com

II. Write a 2-3 sentence summary of the study, including key information about research design and methods.

            This article was about a study that utilized college students to examine the relationship between frequency of Facebook use, participation in Facebook activities, and student engagement in college. The research was a correlational design and utilized an engagement scale, which is a scale that measures overall student engagement in both academic work and co-curricular activities, as an outcome variable.

 

III. State the information that makes it clear to you that it is a correlational study.

1. First there were several predictor variables used including; time spent on FaceBook, times logged into FaceBook, and activity engaged in while on FaceBook.

2. Second there were several outcome variables used including; engagement scale score, time spent studying, and time spent on in campus activities.

3. Lastly it was mentioned in the article that the participants were 64 percent female and 36 percent male and due to the overcompensation of female participants to male participants that the researcher controlled for gender in his analysis which was controlling for a possible third variable or selection bias that could of affected his outcome.

 

IV. Rate the quality/quantity of information provided about the research methods (scale from 1-5 where 1 = poor and 5 = excellent).

                                                                               (Rating)

1. Quality of information:                                        4

2. Quantity of information:                                      2

 

V. Note 1-2 missing pieces of information about research methods that would be important to know.

1. The sample size of the participants would have been useful to determine the generalizability of the findings.

2. The researcher’s data collection methods would have been useful to determine the strength of the study.

3. How student engagement was measured would have been important to know for validity and reliability purposes.

4. Since this was a devised scale by the researcher, information about if the measure was tested for internal validity using Cronbach’s alpha and if the measure had construct validity and if it was deemed reliable would have made me more confident of the measures validity.

5. Whether this was a cross-sectional study collecting data at one point in time or a longitudinal study collecting data on more than one occasion.

 

DRAFT: This module has unpublished changes.

Darius Muniz

Professor Godfrey

Research Methods II

November 24th, 2011

 

                                                      Intervention Program

I) Indicate the name of the intervention program and the organization that runs it.

            The name of the program is PhotoCLUB and it is an empowerment based intervention. The organization that facilitates this program is the Advocacy and Social Action Research Group, which is headed by Dr. Alisha Ali and Dr. Mowry of NYU’s Applied Psychology department.

II) Indicate the target population of the intervention program.

            The PhotoCLUB intervention is being implemented with women and youth in various low-income housing and domestic violence settings.

III) Describe the goals of the intervention program and the main ways in which they seek to achieve these goals (program components).

            For this intervention they have created a manualized, therapeutic approach to creating positive change by using digital photography as a form of empowerment and self-exploration for participants. This intervention is conducted over a period of four sessions for a duration of 90 minutes each. Participants are guided each week in taking photographs that reflect aspects of their personalities. The first assignment is the Here and Now, in which participants take photos that represent who they are right now. The second assignment is Others, in which participants take photographs representing how other people see them. The third assignment is Tomorrow, in which participants take photographs of who they want to become in the future. Through open discussions with facilitators and other members, participants then develop strategies for becoming their desired future selves.

            This intervention aims to examine the mechanisms through which the PhotoCLUB program can positively impact the mental health of participants by using a combination of Possible Selves Theory (Markus & Nurious, 1986), which states that engaging in an imaginative process of envisioning an ideal future self can provide an essential link between the self concept and motivation for positive change, and Empowerment and Mental Health, which is research conducted by Nelson, Lord, and Ochocha (2001), which examined mental health and well being in relation to empowerment in community settings. These theories and research have been integrated to make up the PhotoCLUB intervention as a vehicle for conducting Participatory Action Research or (PAR).

 

 

DRAFT: This module has unpublished changes.