Illinois Physician Ebook Continuing Education

This interactive Illinois Physician Ebook contains 13 hours of continuing education. To complete click the Complete Your CE button at the top right of the screen.

Illinois Continuing Medical Education

2024 Illinois Professional Licensure Program

DEA REQUIREMENT (NEW) 8 HOURS SATISFIES THE DEA’S NEW ONE-TIME MATE REQUIREMENT Substance Use Disorders

Implicit Bias 1 HOUR SATISFIES IMPLICIT BIAS REQUIREMENT

3 HOURS

SATISFIES OPIOID EDUCATION REQUIREMENT Opioids 1 HOUR Sexual Harassment SATISFIES SEXUAL HARASSMENT REQUIREMENT

INCLUDES: DEA’s new one-time MATE requirement

CME FOR:

AMA PRA CATEGORY 1 CREDITS ™ MIPS MOC STATE LICENSURE

IL.CME.EDU

InforMed is Accredited by the Accreditation Council for Continuing Medical Education (ACCME) to provide continuing medical education for physicians.

2024 ILLINOIS

01

IMPLICIT BIAS IMPLICATIONS FOR PHYSICIANS AND HEALTHCARE PROFESSIONALS

COURSE ONE | 1 CREDIT SATISFIES IMPLICIT BIAS REQUIREMENT

10

EFFECTIVE MANAGEMENT OF ACUTE AND CHRONIC PAIN WITH OPIOID ANALGESICS COURSE TWO | 3 CREDITS SATISFIES OPIOIDS REQUIREMENT

38

SEXUAL HARASSMENT IN HEALTHCARE COURSE THREE | 1 CREDIT SATISFIES SEXUAL HARASSMENT REQUIREMENT

46

LEARNER RECORDS: ANSWER SHEET & EVALUATION REQUIRED TO RECEIVE CREDIT SUBSTANCE USE DISORDERS: A DEA REQUIREMENT COURSE FOUR | 8 CREDITS SATISFIES THE DEA’S NEW ONE-TIME MATE REQUIREMENT

80

Program Options

$150 ENTIRE PROGRAM

13 Credits

$80 • Implicit Bias Implications for Physicians and Healthcare Professionals • Effective Management of Acute and Chronic Pain with Opioid Analgesics • Sexual Harassment Awareness and Prevention $130 • Implicit Bias Implications for Physicians and Healthcare Professionals • Effective Management of Acute and Chronic Pain with Opioid Analgesics • Substance Use Disorders: A DEA Requirement

5 Credits

12 Credits

$80 • Substance Use Disorders: A DEA Requirement

8 Credits

CME that counts for MOC Participants can earn MOC points equivalent to the amount of CME credits claimed for designated activities (see page iii for further details). InforMed currently reports to the following specialty boards: the American Board of Internal Medicine (ABIM), the American Board of Anesthesiology (ABA), the American Board of Pediatrics (ABP), the American Board of Otolaryngology–Head and Neck Surgery (ABOHNS), and the American Board of Pathology (ABPath). To be awarded MOC points, you must obtain a passing score, complete the corresponding activity evaluation, and provide required information necessary for reporting. DATA REPORTING: Federal, State, and Regulatory Agencies require disclosure of data reporting to all course participants. InforMed abides by each entity’s requirements for data reporting to attest compliance on your behalf. Reported data is governed by each entity’s confidentiality policy. To report compliance on your behalf, it’s mandatory that you must achieve a passing score and accurately fill out the learner information, activity and program evaluation, and the 90-day follow up survey. Failure to accurately provide this information may result in your data being non-reportable and subject to actions by these entities.

InforMed has joined the Elite Learning family Two of the nation’s top healthcare education providers have joined forces with one goal in mind: to offer physicians a state-of-the-art learning experience that fulfills your state requirements and empowers you with the knowledge you need to provide the best patient care. Here’s what you can expect from our new partnership: • COURSES: In addition to the mandatory courses you need to renew your state license, you’ll now have access to dozens of hours of elective courses and an expanded content library. • ACCOUNTS: You’ll also have access to a personalized learner account. In your account you can add, organize, and track your ongoing and completed courses. For instructions on how to set up your account, email us at office@elitelearning.com. • BOOK CODES: You may notice a book code on the back cover of the latest InforMed program you’ve received in the mail. When entered on our new site, this code will take you directly to the corresponding self-assessment. See more information below. How to complete Please read these instructions before proceeding.

Read and study the enclosed courses and answer the self-assessment questions. To receive credit for your courses, you must provide your customer information and complete the mandatory evaluation. We offer three ways for you to complete. Choose an option below to receive credit and your certificate of completion. Scan this QR code to complete your CE now !

Fastest way to receive your certificate of completion

Online

• Go to BOOK.CME.EDU . Locate the book code ILPA24CME found on the back of your book and enter it in the box then click GO . If you would like to choose a different program option, use the table below and enter the corresponding code in the box. • If you already have an account created, sign in to your account with your username and password. If you do not have an account already created, you will need to create one now. • Follow the online instructions to complete your self-assessment. Complete the purchase process t o receive course credit and your certificate of completion. Please remember to complete the online evaluation.

Enter book code

GO

Example: ILPA24CME

If you need help finding your code, Browse Book Code FAQs

Program Options

Code

Credits

Price

Entire Program

ILPA24CME

13

$150.00

Courses 1, 2 and 3

ILPA24CME-801

5

$80.00

Courses 1, 2 and 4

ILPA24CME-130

12

$130.00

Course 4

ILPA24CME-802 8

$80.00

By mail

By fax

• Fill out the answer sheet and evaluation found in the back of this booklet. Please include a check or credit card information and e-mail address. Mail to InforMed, PO Box 2595, Ormond Beach, FL 32175-2595 . • Completions will be processed within 2 business days from the date it is received and certificates will be e-mailed to the address provided.

• Fill out the answer sheet and evaluation found in the back of this booklet. Please include credit card information and e-mail address. Fax to 1-800-647-1356 . • All completions will be processed within 2 business days of receipt and certificates will be e-mailed to the address provided.

• Submissions without a valid e-mail will be mailed to the address provided.

• Submissions without a valid e-mail will be mailed to the address provided.

BOOK.CME.EDU

BOOK CODE: ILPA24CME

1-800-237-6999

i

INFORMED TRACKS WHAT YOU NEED, WHEN YOU NEED IT

Illinois Professional License Requirements

GENERAL CME REQUIREMENTS Beginning with the March 2, 2020 renewal period, all licensed physician assistants shall complete 50 hours of approved CE per 2-year license renewal cycle. All CE must be completed in the 24 months preceding expiration of the license. CE hours shall be earned as follows: A minimum of 25 hours of required CEs must be earned in Category 1 CMEs as determined by the National Commission on Certification of Physician Assistants; and 25 credit hours of required CEs can be Category 1, Category 2 or a combination of both. American Medical Association (AMA) (providers accredited by the Accreditation Council for Continuing Medical Education (ACCME)) is considered Category 1 CE. For more information, see https://www.ilga.gov/commission/jcar/admincode/068/068013500001160R.html DEA ONE-TIME MATE REQUIREMENT (NEW) Effective June 27, 2023 , renewing DEA-registered practitioners must complete eight (8) hours of one-time training on the treatment and management of patients with opioid or substance use disorders. IMPLICIT BIAS CME REQUIREMENT For license or registration renewals occurring on or after January 1, 2023, a health care professional who has continuing education requirements must complete at least a one-hour course in training on implicit bias awareness per renewal period. A health care professional may count this one hour for completion of this course toward meeting the minimum credit hours required for continuing education. OPIOID CME REQUIREMENT Physician Assistants (PA) who are licensed to prescribe controlled substances must complete three (3) hours of continuing education on safe opioid prescribing practices prior to license renewal. These hours may be counted toward the total continuing education hours required for renewal of a professional license. SEXUAL HARASSMENT CME REQUIREMENT Physician Assistants (PA) licensed by the state of Illinois must complete one (1) hour of sexual harassment prevention training as part of the required continuing education hours.

We are a nationally accredited CME provider. For all board-related inquiries please contact:

Illinois Department of Financial and Professional Regulation 555 West Monroe Street, 5th Floor

LICENSE TYPE: PA

END OF CURRENT LICENSE CYCLE: PA: 3/1/2024

Chicago, IL 60661 P: 1 (888) 473-4858

Disclaimer: The above information is provided by InforMed and is intended to summarize state CE/CME license requirements for informational purposes only. This is not intended as a comprehensive statement of the law on this topic, nor to be relied upon as authoritative. All information should be verified independently.

ii

Table 1. MOC Recognition Statements Successful completion of certain enclosed CME activities, which includes participation in the evaluation component, enables the participant to earn up to the amounts and credit types shown in Table 2 below. It is the CME activity provider’s responsibility to submit participant completion information to ACCME for the purpose of granting MOC credit. Board Programs ABA American Board of Anesthesiology’s redesigned Maintenance of Certification in Anesthesiology TM (MOCA®) program, known as MOCA 2.0® MOC/MIPS CREDIT INFORMATION In addition to awarding AMA PRA Category 1 Credits TM , the successful completion of enclosed activities may award the following MOC points and credit types. To be awarded MOC points, you must obtain a passing score and complete the corresponding activity evaluation.

ABIM

American Board of Internal Medicine’s Maintenance of Certification (MOC) program

American Board of Otolaryngology – Head and Neck Surgery’s Continuing Certification program (formerly known as MOC)

ABOHNS

ABPath

American Board of Pathology’s Continuing Certification Program

ABP

American Board of Pediatrics’ Maintenance of Certification (MOC) program

Table 2. Credits and Type Awarded

AMA PRA Category 1 Credits TM

Activity Title

ABA ABIM ABO ABOHNS ABPath

ABP

Implicit Bias Implications for Physicians and Healthcare Professionals Effective Management of Acute and Chronic Pain with Opioid Analgesics

1 AMA PRA Category 1 Credit T M

1 Credit LL

1 Credit MK

1 Credit LL & SA

1 Credit SA

1 Credit LL

1 Credit LL+SA

3 AMA PRA Category 1 Credits TM

3 Credits LL

3 Credits MK

3 Credits LL & SA

3 Credits SA

3 Credits LL

3 Credits LL+SA

Sexual Harassment in Healthcare

1 AMA PRA Category 1 Credit TM 8 AMA PRA Category 1 Credits T M

1 Credit LL 8 Credits LL

1 Credit MK 8 Credits MK

1 Credit LL & SA 8 Credits LL & SA

1 Credit SA 8 Credits SA

1 Credit LL 8 Credits LL

1 Credit LL+SA

Substance Use Disorders: A DEA Requirement 8 Credits LL+SA Legend: LL = Lifelong Learning, MK = Medical Knowledge, SA = Self-Assessment, LL+SA = Lifelong Learning & Self-Assessment, PS = Patient Safety

Table 3. CME for MIPS Statement Completion of each accredited CME activity meets the expectations of an Accredited Safety or Quality Improvement Program (IA PSPA_28) for the Merit-based Incentive Payment Program (MIPS). Participation in this Clinical Practice Improvement Activity (CPIA) is optional for eligible providers.

iii

IMPLICIT BIAS IMPLICATIONS FOR PHYSICIANS AND HEALTHCARE PROFESSIONALS

COURSE DATES:

MAXIMUM CREDITS:

FORMAT:

Release Date: 1/2022 Exp. Date: 12/2024

1 AMA PRA Category 1 Credit ™

Enduring Material (Self Study)

TARGET AUDIENCE

HOW TO RECEIVE CREDIT:

This course is designed for all physicians and healthcare providers involved in the treatment and monitoring of patients.

• Read the course materials.

• Co mplete the self-assessment questions at the end. A score of 70% is required. • Ret urn your customer information/ answer sheet, evaluation, and payment to InforMed by mail, phone, fax or complete online at program website.

COURSE OBJECTIVE The purpose of this course is to provide a historical context of race and racism and its relationship to the development of racial implicit bias. The development of implicit bias will be discussed along with research demonstrating the impact of implicit bias on the clinical encounter. Recommendations for mitigating implicit bias are offered.

LEARNING OBJECTIVES

Completion of this course will better enable the course participant to: 1. Know how the history of race in America informs the development of racial implicit bias. 2. Explain the definition of implicit bias and how it differs from explicit bias. 3. Describe factors that contribute to the development of implicit bias. 4. Describe research related to the impact of implicit bias on the clinical encounter and patient referrals.

5. Describe the relationship between racial implicit bias and healthcare disparities. 6. Describe strategies to mitigate the impact of implicit bias in decision making.

IMPLICIT BIAS IN HEALTHCARE Implicit bias significantly affects how healthcare professionals perceive and make treatment decisions, ultimately resulting in disparities in health outcomes. These biases, often unconscious and unintentional, can shape behavior and produce differences in medical care along various lines, including race, ethnicity, gender identity, sexual orientation, age, and socioeconomic status. Healthcare disparities stemming from implicit bias can manifest in several ways. For example, a healthcare provider might unconsciously give less attention to a patient or make assumptions about their medical needs based on race, gender, or age. The unconscious assumptions can lead to delayed or inadequate care, misdiagnoses, or inappropriate treatments, all of which can adversely impact health outcomes. Addressing implicit bias in healthcare is crucial for achieving equity in medical treatment. Strategies to combat these biases involve education and awareness programs for healthcare professionals. These programs help individuals recognize and acknowledge their biases, fostering a more empathetic and unbiased approach to patient care. Additionally, implementing policies and procedures prioritizing equitable treatment for all patients can play a pivotal role in reducing healthcare disparities. Ultimately, confronting implicit bias in healthcare is essential to creating a more just and equitable healthcare system where everyone receives fair and equal treatment regardless of their background or characteristics.

ACCREDITATION STATEMENT InforMed is accredited by the Accreditation Council for Continuing Medical Education (ACCME) to provide continuing medical education for physicians.

1

DESIGNATION STATEMENT InforMed designates this enduring material for a maximum of 1 AMA PRA Category 1 Credit ™. Physicians should claim only the credit commensurate with the extent of their participation in the activity.

COURSE SATISFIES

Implicit Bias 1

FACULTY Benjamin Reese, Jr., PsyD Adjunct Professor Department of Psychiatry and Behavioral Sciences Duke University School of Medicine

This course satisfies the new requirement for one (1) hour on implicit bias. SPECIAL DESIGNATION

ACTIVITY PLANNER Michael Brooks CME Director InforMed

Physician assistants (PA) licensed in Illinois must complete one (1) hour of continuing education on implicit bias training.

DISCLOSURE OF INTEREST In accordance with the ACCME Standards for Commercial Support of CME, InforMed implemented mechanisms, prior to the planning and implementation of this CME activity, to identify and resolve conflicts of interest for all individuals in a position to control content of this CME activity.

FACULTY/PLANNING COMMITTEE DISCLOSURE

STAFF AND CONTENT REVIEWERS InforMed staff, input committee and all content validation reviewers involved with this activity have reported no relevant financial relationships with commercial interests. DISCLAIMER *2023. All rights reserved. These materials, except those in the public domain, may not be reproduced without permission from InforMed. This publication is designed to provide general information prepared by professionals in regard to the subject matter covered. It is provided with the understanding that InforMed, Inc is not engaged in rendering legal, medical or other professional services. Although prepared by professionals, this publication should not be utilized as a substitute for professional services in specific situations. If legal advice, medical advice or other expert assistance is required, the service of a professional should be sought. The following faculty and/or planning committee members have indicated they have no relationship(s) with industry to disclose relative to the content of this CME activity: • Benjamin Reese, Jr., PsyD • Michael Brooks

2

Pre-test:

(ANSWER KEY IS DISPLAYED AT THE BOTTOM OF THIS PAGE.)

1. Almost half of all medical students and residents polled in a 2016 study believed that Black people had thicker skin, less sensitive nerve endings, and experienced less pain than Whites. A. True B. False 2. Implicit bias implies a conscious judgment regarding an individual based on a particular characteristic.

A. True B. False 3. Implicit bias may be influenced by: A. Race B. Sexual orientation C. Age D. All of the above 4. The strategy of “pausing” for the purpose of mitigating implicit bias may be described as: A. Pausing to evaluate a patient’s body language before engaging in conversation B. Pausing to fully review the medical record C. Pausing to reflect on the kind of associations that you may be making about the patient D. Pausing to check the amount of time available for your consultation 5. The implicit association test: A. Measures conscious judgment of specific individual characteristics

B. Is available only to health care professionals C. Must be completed for medical licensing D. Measures strength of associations to specific individual characteristics

The ingrained nature of this racist system afforded any White person the right to stop any African American individual, question them, search them, and even physically abuse them. These actions were accepted practice and, in many cases, were part of legal codes and regulations. Thus, the systemic and structural nature of racism in this country was not only built on the economic practice of enslaving African Americans for individual and family profit, but was built on the practices, beliefs, and laws that created and supported the belief that Africans were not fully human. 2 In the 18 th and 19 th century, the support of this belief of inhumanity took the form of scientific racism or pseudoscience. Many people, both professionals and laypersons, thought that African American people had an innate tendency to want to run away from the confinement of the plantation, had thicker skin and skulls, and had fewer nerve endings and therefore could endure more pain. This rationale was often used for the extreme brutalization and whipping experienced by African American men, women, and children. 3 These beliefs contributed to the medical experimentation conducted on African American bodies, sometimes without any attempt to reduce the pain and suffering of African American patients or experimental subjects. 4,5 Although current laws, policies, and accepted medical practices have eliminated the horrendous abuses of African American bodies, it is important to recognize how deeply embedded many of the beliefs and perceptions of African American people are within American culture, consciously and implicitly.

A 2016 study revealed that almost half of the medical students and residents surveyed endorsed notions of pseudoscience, believing that African American people had thicker skin, less sensitive nerve endings, and experienced less pain than Whites. 6,7 In spite of conscious endorsements of equity, fairness, social justice, and providing the highest level of care, there is this parallel process of unconscious or implicit bias. It is not that providers are inherently bad. Rather, they are human and prone to internalize, to a greater or lesser extent, the beliefs and stereotypes resulting from centuries of systemic and structural racism. These beliefs and internalized stereotypes can cause physicians to behave in ways that violate their deeply held values of fairness and equity. The process is unconscious or implicit. Definition Implicit bias can be defined as those attitudes, beliefs, and stereotypes that affect our understanding, behavior, and actions in an unconscious (implicit) manner. A relatively small portion of the information the brain processes is conscious. The majority of information is processed unconsciously, out of awareness. As people process this information, their unconscious association can reinforce stereotypes that most often differ from their conscious assessment of an individual or group. These unconscious associations can contribute to the unequal treatment of people based on their race, ethnicity, gender, gender identity, age, disability, sexual orientation, etc.

Introduction Although implicit or unconscious bias and its impact on healthcare can be understood in relationship to a range of identity characteristics (age, gender, sexual orientation, etc.), implicit bias related to race is particularly salient in the United States. This focus on racial implicit bias can be understood in the context of the history of race and racism in America. The Enslavement of Africans The first Africans were brought to this country forcibly on ships, arriving on the Southern shores of our nation. Packed body-next-to-body in the hull of ships, those that survived disease, malnutrition, and abuse entered this country as cargo…property. White farmers and various businessmen purchased Africans to plant and harvest crops and to cook, clean homes, and care for their children. They were the property of the individuals and families that purchased them. This forced enslavement of Africans was maintained by a system of inhumane physical and psychological abuse, norms, accepted practices, and laws. As property, Africans weren’t considered human. They were property to be purchased, sold, and even named in the will of individuals before they died, like one might leave a house or wagon to a spouse or children. Like the evaluation or assessment of a used car or house, enslaved Africans were evaluated and rated according to their fitness and/or physical defects. They were not viewed as human beings on par with Whites. 1

3

Although implicit bias and unconscious associations can be a subtle influence on cognition and behavior, their impact on decisions can be significant. 8,9 Recently, an increasing number of states have enacted legislation recognizing the criticality of implicit bias in healthcare. The following is an example from California. 10

(e) Implicit gender bias also impacts treatment decisions and outcomes. Women are less likely to survive a heart attack when they are treated by a male physician and surgeon. LGBTQ and gender-nonconforming patients are less likely to seek timely medical care because they experience disrespect and discrimination from healthcare staff, with one out of five transgender patients nationwide reporting that they were outright denied medical care due to bias. (f) The Legislature intends to provide specified healing arts licensees with strategies for understanding and reducing the impact of their biases in order to reduce disparate outcomes and ensure that all patients receive fair treatment and quality healthcare. The process of implicit bias in no way diminishes the importance of conscious, deliberate behavior. Physicians’ conscious endorsements of the values of equity and fairness can be an important element of their service to patients. In addition to deeply held values, creating equitable healthcare environments requires physicians’ conscious actions to build systems and processes that move them towards the elimination of disparities. It is not only the responsibility of those with particular titles, like a chief diversity officer, but all healthcare providers must be “activists” in their own areas of work: in the treatment room, in the laboratory, or in the administrative suite. The focus on implicit bias does not absolve providers of conscious, focused, deliberate action, it simply highlights an important parallel process.

As toddlers and older children begin to spend time online, watch television, and read books, they sense the race or gender of the people who tend to be in leadership roles, those who appear dangerous, those who are police and fire people, those who are doctors, etc. Inequities and biases in the broader society get incorporated into media, which get consumed by children, whether intentionally or not. These portrayals then influence and shape unconscious associations in children of all ages. 13 Children are also influenced by the behavior of parents, caregivers, and other significant adults in their life, noticing the complexion and gender of their close friends, as well as comments and jokes. It’s not only the things that parents and caregivers say to convey fairness and kindness for all people, but it’s also what children observe day-to-day in the behavior of those adult models. 14 The process of unconscious associations can also be seen in the way adults begin to view and interact with children. In a study at the Yale Child Study Center in 2016, preschool teachers viewed videos of African American and White children walking around a classroom, talking, and interacting with each other 15 . The viewing device also recorded who and what the teachers were looking at during the study. When teachers then were told that there might be challenging behavior, the device revealed that they began to look at and track the African American children. Further, their eyes tracked the African American boys more than the other children. The teachers did not have any conscious idea of their viewing behavior. Adult Research Within the racial history of America, the complexion of African American people has always been a complex dynamic. The One Drop Rule, in practice and then in law, has existed since Africans were forcibly brought to this country. Interracial relationships, both forced and voluntary, resulted in bi-racial children and adults. Essentially, any African ancestry (“one drop of African American blood”) classified an individual as African American. 16 In addition, lighter-skinned African Americans were sometimes viewed as slightly higher in status than darker-skinned African Americans. Darker-skinned African Americans often had fewer employment opportunities and were treated more harshly. Although the deliberate disparate treatment of light-skinned and darker-skinned African Americans has significantly diminished, unconscious bias related to the complexion of people still exists in many sectors of American society. Hiring and promotions of African Americans can sometimes be significantly influenced by implicit racial bias related to complexion. 17,18 In one experiment, subjects were sent to one of two rooms. In one room subjects saw the subliminal presentation of the word “ignorant” on a screen, followed by the subliminal presentation of the face of an African American male. In the other room, subjects saw the subliminal presentation of the word “educated” and the subliminal presentation of the face of the same man.

Assembly Bill No. 241 Chapter 417

An act to amend Sections 2190.1 and 3524.5 of, and to add Section 2736.5 to, the Business and Professions Code, relating to healing arts. [Approved by Governor October 02, 2019. Filed with Secretary of State October 02, 2019.] LEGISLATIVE COUNSEL’S DIGEST This bill would require the Board of Registered Nursing, by January 1, 2022, to adopt regulations requiring all continuing education courses for its licensees to contain curriculum that includes specified instruction in the understanding of implicit bias in treatment. Beginning January 1, 2023, the bill would require continuing education providers to comply with these provisions and would require the board to audit education providers for compliance with these provisions, as specified.

THE PEOPLE OF THE STATE OF CALIFORNIA DO ENACT AS FOLLOWS:

SECTION 1. The Legislature finds and declares all of the following: (a) Implicit bias, meaning the attitudes or internalized stereotypes that affect our perceptions, actions, and decisions in an unconscious manner, exists, and often contributes to unequal treatment of people based on race, ethnicity, gender identity, sexual orientation, age, disability, and other characteristics. (b) Implicit bias contributes to health disparities by affecting the behavior of physicians and surgeons, nurses, physician assistants, and other healing arts licensees. (c) Evidence of racial and ethnic disparities in healthcare is remarkably consistent across a range of illnesses and healthcare services. Racial and ethnic disparities remain even after adjusting for socioeconomic differences, insurance status, and other factors influencing access to healthcare. (d) African American women are three to four times more likely than White women to die from pregnancy-related causes nationwide. African American patients often are prescribed less pain medication than White patients who present the same complaints, and African American patients with signs of heart problems are not referred for advanced cardiovascular procedures as often as White patients with the same symptoms.

Early Childhood Research

One of the important areas of implicit bias research focuses on the question, how early in one’s development does implicit bias begin to show up? Recent research suggests that the foundation for what later shows up as implicit bias occurs in infancy. For example, if we track the eyes of infants, at about four weeks of age they will stare longer and more frequently at female faces if a woman has been the primary caregiver. This is clearly not implicit bias, but it appears to reflect a differential association or a preference. This process of differential response in terms of what types of faces infants tend to stare at continues to evolve. 11,12 Although care has to be taken to not infer bias from infant behavior, it appears that responsiveness to faces that are similar to the infant or the individual who provides nurturing/food may be a precursor to preferences later in development. Did you Know? Much of the research on implicit bias has only occurred in the last 40 years. It has provided increasing support for theories related to the development of implicit biases and ways in which unconscious biases impact decision making.

4

In the next phase of the experiment, subjects in both rooms saw seven variations of the same man’s face: three images were lighter-skinned versions, three images were darker-skinned versions, and one image was identical to the first subliminal presentation. From the seven photographs, subjects were asked to select the version that was identical to the subliminal presentation. Researchers found that subjects primed with the word “ignorant” selected a darker-skinned version of the man, while those primed with the word “educated” selected a lighter-skinned version. 19 Judgment about the worth or intellect of the African American man appeared to be unconsciously influenced by his complexion. Although not reported in the research, subjects would probably deny making judgment based on complexion. It should be noted that in recent generations, discrimination based on skin tone appears to be diminishing. 20 Excessive and biased focus on African American youth’s behavior in schools contributes to the disparate suspension rate for African American and White students in primary and secondary schools. 21 This does not appear to be the result of conscious discrimination. It appears that similar behavior is judged differently when occurring by an African American versus a White student. Blake and colleagues went a step further and examined race, complexion, and suspension rates. They found that African American teenage girls with darker complexions are suspended at a higher rate than those with lighter complexions. 22 Again, unconscious bias seems to be a major factor. PLEASE COMPLETE CASE STUDY EXERCISE 1.

As mentioned earlier, implicit bias operates not only for race, given the historical context of race in America, but gender, sexual orientation, height, weight, and even accent can unconsciously influence attitudes and decisions. In one experiment, subjects listened to two separate English speakers reading the same script. When they saw a photograph of an Asian person as the speaker, they rated the accent as being stronger than when the speaker was paired with a photo of a White person. They also rated the understanding of the content as being more difficult to understand when they saw a face of an Asian person. The assessment of the speaker, prompted by the photograph of an Asian individual, appeared to be influenced by unconscious bias. 23 Before the Covid-19 pandemic, it was more common to have online courses with PowerPoint slides and videos, without seeing the actual instructor. MacNell constructed a research design where a male and a female instructor each led two sections of a discussion group. During one section they both used a male name; during the other section they both used a female name. Students couldn’t see the face of the instructor or hear their voice. They tried to teach all four sections similarly. At the end of the semester, the students in all four discussion groups were asked to rate the instructors on 12 different traits, covering characteristics related to their effectiveness and interpersonal skills. The male-named instructors were rated highest on all characteristics, regardless of whether the instructors were actually male or female. Classwork was graded and returned to students at the same time in all four sections. Students who thought that they were being taught by a male instructor gave a promptness rating of 4.35 out of 5. Student gave the female-named instructors a rating of 3.55. 24,25 Again, this points to the powerful influence of unconscious bias.

Healthcare Research What does this have to do with healthcare? In addition to a provider’s conscious adherence to high ethical standards and a commitment to quality care, they are also subject to implicit bias, like the rest of the population. Fitzgerald and Hurst examined 42 peer-reviewed articles. 26 The evidence indicated that healthcare professionals exhibit the same level of implicit bias as the wider population. A couple decades earlier, Shulman and his colleagues published research that many view as a major stimulus for further research regarding implicit bias and healthcare. 27 They presented 720 physicians with videos of patients (actors) who were similar in physical appearance and medical history, differing only by race and sex. All were candidates for cardiac catheterization. After the physicians saw the videos of the patients and reviewed their history, the researchers found that women and African Americans were less likely to be referred for cardiac catheterization than men and Whites. It appeared that, in spite of the conscious commitment to equitable care, unconscious bias was an influence in referral decision making. The national interest in implicit bias in healthcare intensified when the Institute of Medicine delivered their report, Unequal Treatment, in 2003. 28 It concluded that implicit bias against social groups, including racial and ethnic groups, can impact the clinical encounter. Much of the research supporting this report utilized the online Implicit Association Test (IAT). The IAT measures the strength of associations between concepts such as African American or White, old or young, good or bad, desirable or undesirable, and dangerous or friendly. The reaction time (association) to various pairs of words or photographs is a measure of the strength of the association. Millions of people used this website (operated by Harvard University) to take the IAT or one of the other tests.

Instructions: Spend 10 minutes reviewing the case below and considering the questions and commentary that follows. Case Study Exercise 1 You’ve just come from a meeting with a group of African American and Latinx community residents. They presented the committee, which you are a part of, with a list of demands related to what they perceive as a racist hospital environment.

1. Why might the community members perceive a hospital or healthcare system as being racist?

Commentary on Question 1: In addition to the history and present state of a particular hospital or healthcare system, the history of racism in America in general, as well as continuing racial health disparities, may contribute to some African Americans and Latinx community residents perceiving a hospital as being racist. 2. Why might some African American patients question White providers about their background and experience in working with African American patients?

Commentary on Question 2: The history of race relations in American has contributed to many divisions. White providers may not have many close friends who are African American or spend significant amounts of time in predominately African American communities. Stereotypes about casually dressed young African American men may operate for some providers.

5

The racial disparity in the judgment of pain has been studied as an example of implicit bias in healthcare. In research by Mende-Siedlecki and colleagues, White providers demonstrated more stringent thresholds in perceiving pain on African American faces versus White faces, and those with more stringent thresholds for African American patients prescribed fewer non-narcotic pain relievers. 29 This was not true for Asian faces, suggesting that other-face dynamics were not at play. This research did not investigate whether gaps in empathy or perspective taking skills might be a partial explanation for the disparity. PLEASE COMPLETE CASE STUDY EXERCISE 2. Implicit bias has also been shown to impact the quality of the clinical encounter, particularly communication. In an early study, primary care physicians took the IAT and had their clinical encounters recorded. 30 Provider race bias on the IAT was associated with lower quality communication with African American patients, such as more provider verbal dominance, lower patient positive affect, poorer patient ratings of interpersonal care, lower perceptions of respect from clinicians, and lower likelihood of recommending the clinician. The quality of communication is also related to word choice. One hundred and seventeen videotaped racially discordant physician-patient encounters were analyzed using the Linguistic Inquiry and Word Count software. Providers with higher levels of implicit racial bias (based on IAT scores) more frequently used first-person pronouns and anxiety-related words than providers with lower implicit racial bias scores. 32 Communication is also a major factor in a patient’s experience of trust in the clinical encounter. 33

There is abundant evidence that even when controlling for variables such as insurance, socioeconomic status, geography, and even socioeconomic status, implicit bias is an important influence on patient satisfaction and referral for treatment, both contributors to healthcare disparities. 34,35,36 , Even high-status African American patients can experience disparate treatment. When giving birth, tennis star Serena Williams suffered a pulmonary embolism, although thankful for the care she received, she noted that her status likely contributed to her getting a level of care not afforded to all women. 37 Ms. Williams’ experience causes some to reflect on the 2019 CDC report that indicated a racial disparity in pregnancy-related deaths. Implicit bias can not only influence the assessment of pain but can also influence provider decision making in high-discretion situations. 38,39 For example, of the two or three tests that might be available for a particular condition, there is sometimes discretion on the part of the provider in terms of which test is given or how soon a test is recommended. While consciously endorsing values of equity, fairness, high quality care, etc., provider behavior can be influenced by implicit bias.

Given the rapid, unconscious associations that characterize implicit bias, factors that impede the slow, careful refection of multiple factors can be fertile ground for implicit bias. Many providers may see these factors as basically describing their day-to-day work. But recognizing these factors can provide an opportunity to try to make small modifications, where possible, to mitigate the influence of implicit bias.

Mitigating Implicit Bias

There are several strategies that have shown promise in mitigating or reducing the occurrence of implicit bias. Given the brain’s innate rapid processing of large amounts of data every second and the acceleration factors mentioned above, there is no strategy that can completely eliminate unconscious associations or bias. Further, approaches to mitigating or reducing implicit bias are most effective when more than one strategy is utilized. Strategies to consider include the following: 41,42,43,44 Increasing Knowledge It is useful for workshops and presentations to describe how implicit bias develops and its relationship to societal stereotypes (race, gender, sexual orientation, disability, etc.). Presentation of research findings can help providers understand how implicit bias can negatively impact various aspects of the clinical encounter and contribute to healthcare disparities. As a component of workshops, case studies can help providers apply knowledge to realistic, complex situations. Refresher experiences are also useful.

Accelerating Implicit Bias 40

Quick Implicit Bias Facts Several factors can accelerate implicit bias or make it more likely to be a significant influence. • Time constraints (e.g., limited time to see a patient) • Complexity (e.g., multitasking or needing to consider multiple factors quickly) • Physical constraints (e.g., working long hours)

Instructions: Spend 10 minutes reviewing the case below and considering the questions and commentary that follows. Case Study Exercise 2 John is a White nurse caring for an African American pregnant woman in an obstetrics unit in a hospital. The patient is a lesbian and had an in vitro fertilization. Her partner is White and asks to speak to you, the physician, in private and not in the patient’s room. The partner mentions that she recently saw a news story about how African American women are treated unfairly in comparison to White women. You’re sure that her partner will be given the same level of care as other patients, and you give this reassurance consciously in a deliberate manner. However, you know that, in addition to this conscious process, there are potential areas of implicit bias that might occur, both in decision making and in communication.

1. How might the racial difference between the nurse, John, and the patient influence provider-patient communication?

Commentary on Question 1: Given the history of racism in America, racial discordance between provider and patient may negatively impact trust in the clinical encounter. African American patients may have experienced racial insensitivities, bias, or discrimination in the past and may be vigilant for signs of caring and trust from the provider. In addition to racial implicit bias, there can be implicit bias based on other characteristics, such as sexual orientation.

2. How might implicit bias show up in this case?

Commentary on Question 2: Given that there might be a difference between conscious attempts to be fair and equitable and implicit bias, care needs to be taken to ensure that word choice and nonverbal communication (eye contact, smiling, etc.) do not reflect unintentional bias. Further, nonverbal bias includes not touching and standing further away from the patient. 31 Self-reflection and awareness can be useful tools. Reflecting on the question, Would I react differently if the patient was White or heterosexual? can be a useful strategy.

6

Self-Awareness With knowledge of the development and process of implicit bias as a foundation, a pause-and- reflect approach can increase the occasions when providers deliberately take a few moments to reflect on their thoughts and behaviors when interacting with a patient and/or making an important decision. Pausing allows for a few seconds to reflect on the kinds of associations that they may be making as the patient walks through the door based on the way they’re dressed, their gender, their race, or any other characteristic. Discovering personal tendencies or becoming aware of an area of personal bias can help in modifying communication, where necessary. Such self-awareness can prompt providers to focus on seeing a given patient as an individual (individuation) and trying to see things from the patient’s perspective (perspective taking). Organization Systems and Processes Implicit bias can play a key role in fostering and reinforcing systems of inequities in hiring and promotion. Race, gender, accent, weight, etc. are factors that not only impact implicit bias in patient care but can also influence hiring and promotion decisions. Mitigating unconscious bias in rewards and recognition, as well as in selection decisions, is of critical importance in the work to increase the diversity of medical students, clinical providers, researchers, managers, and senior healthcare providers. Cues and Reminders Information about implicit bias as well as motivation to reflect on personal biases can begin to fade months and even weeks after the initial intervention. Strategies to stimulate recall or remind providers about the work to mitigate implicit bias can be useful. Key words or phrases on the treatment room computer screen or even a specially designed screen saver can be used as a reminder. A mobile phone background can contain a photo or a word that serves as a reminder every time the provider uses the phone. Changing the photo or word periodically can help to avoid habituation. The inside cover of a folder containing CVs can list key phrases to remind search committee members to avoid bias in the screening process. Disparities in access to and delivery of health care services As it is quite apparent that disparities in healthcare exist, The Department of Health and Human Services intends to directly address this inequality in the next four years. It has posted a draft of its strategic goals for the fiscal years 2022- 2026, and impartial access to healthcare is of particular interest. The first of the five stated goals are to “Protect and Strengthen Equitable Access to High Quality and Affordable Healthcare”. 45 As part of this goal, one strategic objective specifies an intent to “expand equitable access to comprehensive, community-based, innovative, and culturally- competent healthcare services while addressing social determinants of health”.

They describe improved access to health-related services for an underserved population through the removal of barriers to access, a reduction in disparities in healthcare, and support of community- based services. An increase in healthcare facilities, a more diverse healthcare workforce and collaboration with cultural and community services can all contribute to improvements in access disparities. 46 Chin et al provided specific suggestions for community involvement through school-based care, household outreach, and religious based care delivery. Members of the community may be involved as peer coaches, peer educators and patient care navigators to enhance use of healthcare services. Educational material intended to address specific cultural perspectives can target unique characteristics of the community and, “open door” clinic policies and streamlined referral processes may contribute to an increase in patient participation. Chin et al also suggest that reduced out of pocket costs, or free giveaways can serve as financial incentives to improve participation in healthcare services. Lastly, psychological services and support through family therapy, motivational interviewing and counseling can help to encourage access to additional services and care. 47

Conclusion

The history of race and racism in America is central to the development of racial implicit bias across various sectors of our society and is a major contributor to racial healthcare inequities. However, we must not lose sight of the intersection of implicit bias and gender, sexual orientation, weight, race, and other individual and group characteristics. Our patients bring their unique physical condition, their intersectional identity, and, in many cases, significant time, living within the ingrained structures, attitudes, and beliefs of this nation. It’s our responsibility to not only engage what we are consciously aware of but also work to uncover personal and organizational biases that impede our movement towards a healthcare environment and society of true equity and the highest quality care.

7

References

17. Grant C. What Is Colorism? Verywell Mind. Published November 21, 2020. https://www.verywellmind.com/ what-is-colorism-5077380 18. Louie P, Wilkes R. Representations of race and skin tone in medical textbook imagery. Social Science & Medicine. 2018;202:38-42. doi:10.1016/j. socscimed.2018.02.023 19. Ben-Zeev A, Dennehy TC, Goodrich RI, Kolarik BS, Geisler MW. When an “Educated” Black Man Becomes Lighter in the Mind’s Eye. SAGE Open. 2014;4(1):215824401351677. doi:10.1177/2158244013516770 20. Branigan AR, Freese J, Sidney S, Kiefe CI. The Shifting Salience of Skin Color for Educational Attainment. Socius: Sociological Research for a Dynamic World. 2019;5:237802311988982. doi:10.1177/2378023119889829 21. Erickson JH, Pearson J. Excluding Whom? Race, Gender, and Suspension in High School. Education and Urban Society. Published online June 24, 2021:001312452110275. doi:10.1177/00131245211027510 22. Blake JJ, Keith VM, Luo W, Le H, Salter P. The role of colorism in explaining African American females’ suspension risk. School Psychology Quarterly. 2017;32(1):118-130. doi:10.1037/spq0000173 23. Zheng Y, Samuel AG. Does seeing an Asian face make speech sound more accented? Attention, Perception, & Psychophysics. 2017;79(6):1841- 1859. doi:10.3758/s13414-017-1329-2 24. MacNell L, Driscoll A, Hunt AN. What’s in a Name: Exposing Gender Bias in Student Ratings of Teaching. Innovative Higher Education. 2014;(4):291-303. doi:10.1007/s10755-014-9313-4 25. Mitchell KMW, Martin J. Gender Bias in Student Evaluations. PS: Political Science & Politics. 2018;51(03):648-652. doi:10.1017/ s104909651800001x 26. FitzGerald C, Hurst S. Implicit bias in healthcare professionals: a systematic review. BMC Medical Ethics. 2017;(1). doi:10.1186/s12910-017-0179-8 27. The Effect of Race and Sex on Physicians’ Recommendations for Cardiac Catheterization. New England Journal of Medicine. 1999;340(14):1130- 1130. doi:10.1056/nejm199904083401424 28. ‌Institute of Medicine (US) Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care, Smedley BD, Stith AY, Nelson AR, eds. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. Washington (DC): National Academies Press (US); 2003. 29. Mende-Siedlecki P, Qu-Lee J, Backer R, Van Bavel JJ. Perceptual contributions to racial bias in pain recognition. Journal of Experimental Psychology: General. 2019;148(5):863-889. doi:10.1037/ xge0000600 30. Cooper LA, Roter DL, Carson KA, et al. The Associations of Clinicians’ Implicit Attitudes About Race With Medical Visit Communication and Patient Ratings of Interpersonal Care. American Journal of Public Health. 2012;102(5):979-987. doi:10.2105/ ajph.2011.300558 31. Elliott, A. M., Alexander, S. C., Mescher, C. A., Mohan, D., & Barnato, A. E. Differences in physicians’ verbal and nonverbal communication with black and white patients at the end of life. Journal of pain and symptom management, 2016; 51(1), 1-8. 32. Hagiwara N, Slactcher RB, Eggly S, Penner LA. Physician Racial Bias and Word Use during Racially Discordant Medical Interactions. Health Communication. 2016;32(4):401-408. Doi:10.1080/ 10410236.2016.1138389 33. Cuevas AG, O’Brien K, Saha S. Can patient-centered communication reduce the effects of medical mistrust on patients’ decision making? Health Psychology. 2019;38(4):325-333. doi:10.1037/hea0000721

34. Saluja B, Bryant Z. How Implicit Bias Contributes to Racial Disparities in Maternal Morbidity and Mortality in the United States. Journal of Women’s Health. 2020;30(2). doi:10.1089/jwh.2020.8874 35. Wilson BN, Murase JE, Sliwka D, Botto N. Bridging racial differences in the clinical encounter: How implicit bias and stereotype threat contribute to health care disparities in the dermatology clinic. International Journal of Women’s Dermatology. Published online January 2021. doi:10.1016/j.ijwd.2020.12.013 36. Wilson BN, Murase JE, Sliwka D, Botto N. Bridging racial differences in the clinical encounter: How implicit bias and stereotype threat contribute to health care disparities in the dermatology clinic. International Journal of Women’s Dermatology. Published online January 2021. doi:10.1016/j.ijwd.2020.12.013 37. Salam M. For Serena Williams, Childbirth Was a Harrowing Ordeal. She’s Not Alone. The New York Times. https://www.nytimes.com/2018/01/11/sports/ tennis/serena-williams-baby-vogue.html. Published January 11, 2018. 38. Johnson JD, Asiodu IV, McKenzie CP, et al. Racial and Ethnic Inequities in Postpartum Pain Evaluation and Management. Obstetrics and gynecology. Published online 2019:10.1097/AOG.0000000000003505. doi:10.1097/AOG.0000000000003505 39. Roeder A. America is Failing its Black Mothers. Harvard Public Health Magazine. Published December 21, 2018. https://www.hsph.harvard.edu/magazine/ magazine_article/america-is-failing-its-black- mothers/ 40. Johnson TJ, Hickey RW, Switzer GE, et al. The Impact of Cognitive Stressors in the Emergency Department on Physician Implicit Racial Bias. Gerson L, ed. Academic Emergency Medicine. 2016;23(3):297-305. doi:10.1111/acem.12901 41. Eight tactics to identify and reduce your implicit biases. The Journal of Medical Practice Management : MPM. 2020;35(5):237. https://login.proxy.lib.duke. edu/login?url=https://www.proquest.com/scholarly- journals/eight-tactics-identify-reduce-your-implicit/ docview/2504870922/se-2?accountid=10598 42. Narayan MC. Addressing Implicit Bias in Nursing: A Review. AJN, American Journal of Nursing. 2019;119(7):36-43. doi: 10.1097/01. naj.0000569340.27659.5a 43. IHI Multimedia Team. How to Reduce Implicit Bias. Ihi. org. Published 2019. http://www.ihi.org/communities/ blogs/how-to-reduce-implicit-bias 44. Di Brito SR, Lopez CM, Jones C, Mathur A. Reducing Implicit Bias: Association of Women Surgeons #HeForShe Task Force Best Practice Recommendations. Journal of the American College of Surgeons. 2019;228(3):303-309. doi: 10.1016/j. jamcollsurg.2018.12.011 45. HHS.gov. Strategic Goal 1: Protect and Strengthen Equitable Access to High Quality and Affordable Healthcare. October 6, 2021. https://www.hhs.gov/ about/draft-strategic-plan/goal-1/index.html 46. HHS.gov. Strategic Goal 1.3. Expand equitable access to comprehensive, community-based, innovative, and culturally-competent healthcare services while addressing social determinants of health October 6, 2021. https://www.hhs.gov/about/draft-strategic- plan/goal-1/objective-1-3/index.html 47. Chin, M.H., Clarke, A.R., Nocon, R.S. et al. A Roadmap and Best Practices for Organizations to Reduce Racial and Ethnic Disparities in Health Care. J GEN INTERN MED 27, 992–1000 (2012). https://doi. org/10.1007/s11606-012-2082-9

1.

Franklin JH, Higginbotham EB. From Slavery to Freedom: A History of African Americans. Mcgraw-Hill/ Connect Learn Succeed; 2011. Phrenology: What is it, and how did it contribute to neuroscience? www.medicalnewstoday.com. Published February 2, 2021. https://www.medicalnewstoday. com/articles/phrenology-the-pseudoscience-of-skull- shapes Historical Context: Facts about the Slave Trade and Slavery | Gilder Lehrman Institute of American History. Gilderlehrman.org. https://www.gilderlehrman.org/ history-resources/teaching-resource/historical- context-facts-about-slave-trade-and-slavery. Published 2021. Accessed October 24, 2021. Cohen AS. Harvard’s Eugenics Era. Harvard Magazine. Published February 19, 2016. Accessed October 24, 2021. https://www.harvardmagazine.com/2016/03/ harvards-eugenics-era ‌Dimuro G. Southerners Actually Thought Slaves Escaping Was A Sign Of Mental Illness. All That’s Interesting. Published April 4, 2018. https:// allthatsinteresting.com/drapetomania Hoffman KM, Trawalter S, Axt JR, Oliver MN. Racial bias in pain assessment and treatment recommendations, and false beliefs about biological differences between blacks and whites. Proceedings of the National Academy of Sciences. 2016;113(16):4296-4301. doi:10.1073/pnas.1516047113 ‌Ramin Skibba. The Disturbing Resilience of Scientific Racism. Smithsonian. Published May 20, 2019. https://www.smithsonianmag.com/science-nature/ disturbing-resilience-scientific-racism-180972243/ Edgoose J, Quiogue M, Sidhar K. How to Identify, Understand, and Unlearn Implicit Bias in Patient Care. Family Practice Management. 2019;26(4):29- 33. https://www.aafp.org/fpm/2019/0700/p29.html

2.

3.

4.

5.

6.

7.

8.

9. DeAngelis T. How does implicit bias by physicians affect patients’ health care? https://www.apa.org. https://www.apa.org/monitor/2019/03/ce-corner. Published March 2019. 10. ‌Bill Text - AB-241 Implicit bias: continuing education: requirements. https://leginfo.legislature.ca.gov/ faces/billNavClient.xhtml?bill_id=201920200AB241. Published 2021. Accessed October 24, 2021. 11. Balas B, Westerlund A, Hung K, Nelson III CA. Shape, color and the other-race effect in the infant brain. Developmental Science. 2011;14(4):892-900. doi:10.1111/j.1467-7687.2011.01039.x 12. ‌Lee K, Quinn PC, Pascalis O. Face Race Processing and Racial Bias in Early Development: A Perceptual-Social Linkage. Current Directions in Psychological Science. 2017;26(3):256-262. doi:10.1177/0963721417690276 13. Williams A, Steele JR. Examining Children’s Implicit Racial Attitudes Using Exemplar and Category‐ Based Measures. Child Development. 2017;90(3). doi:10.1111/cdev.12991 14. ‌Pirchio S, Passiatore Y, Panno A, Maricchiolo F, Carrus G. A Chip Off the Old Block: Parents’ Subtle Ethnic Prejudice Predicts Children’s Implicit Prejudice. Frontiers in Psychology. 2018;9. doi:10.3389/fpsyg.2018.00110 15. Gilliam W, Maupin A, Reyes C, Accavitti M, Shic F. Do Early Educators’ Implicit Biases Regarding Sex and Race Relate to Behavior Expectations and Recommendations of Preschool Expulsions and Suspensions. Yale Child Study Center; 2016 16. O’Connell HA, Bratter JL, Casarez RS. One drop on the move: historical legal context, racial classification, and migration. Ethnic and Racial Studies. Published online May 12, 2020:1-20. doi:10.1080/01419870.2020.1 761554

8

Page 1 Page 2 Page 3 Page 4 Page 5 Page 6 Page 7 Page 8 Page 9 Page 10 Page 11 Page 12 Page 13 Page 14 Page 15 Page 16 Page 17 Page 18 Page 19 Page 20 Page 21 Page 22 Page 23 Page 24 Page 25 Page 26 Page 27 Page 28 Page 29 Page 30 Page 31 Page 32 Page 33 Page 34 Page 35 Page 36 Page 37 Page 38 Page 39 Page 40 Page 41 Page 42 Page 43 Page 44 Page 45 Page 46 Page 47 Page 48 Page 49 Page 50 Page 51 Page 52 Page 53 Page 54 Page 55 Page 56 Page 57 Page 58 Page 59 Page 60 Page 61 Page 62 Page 63 Page 64 Page 65 Page 66 Page 67 Page 68 Page 69 Page 70 Page 71 Page 72 Page 73 Page 74 Page 75 Page 76 Page 77 Page 78 Page 79 Page 80 Page 81 Page 82 Page 83 Page 84 Page 85 Page 86 Page 87 Page 88

checkout.elitelearning.com

Powered by