October 14, 2015
Frank Worrell, Ph.D.
Chair, Board of Scientific Affairs
American Psychological Association
Dear Dr. Worrell:
I am writing to you as the Chair of the Board of Scientific Affairs (BSA) of the American Psychological Association to request formally that you place this correspondence and the motion enclosed in it on the agenda of the Board for their meeting at the end of this month. I will be sending a copy of this request directly to all of the other Board Chairs and the Executive Directors of the major Directorates asking them to consider the same motion at their meetings as well. As you may or may not know, prior to the August meeting of APA’s Council of Representatives, I posted an extensive essay on several APA list serves calling attention to the presence of a variety of forms of systematic bias present in what is known as the Hoffman Report. The essay was received positively but had no effect on the outcome of the Council meeting in which they adopted a resolution against APA members being involved in consulting to or other roles in national intelligence interrogations. Shortly following that meeting, I posted a call for scientific validation of the Hoffman Report. I am attaching a copy of that call to this letter.
I am particularly concerned that BPA consider this request because of the unprecedented nature of the action taken by the Board of Directors and the Council of Representatives in August to specifically exclude an area of professional practice in psychology without any prior consideration by or consultation with the full membership of BSA. The Hoffman Report deliberately undermined the credibility of BSA and its staff. It created the impression that there was something inherently wrong and immoral for scientific psychologists to be involved with their colleagues serving in the armed services of our government and who were and still are trying to answer questions with profound scientific as well as public policy and safety considerations. And the findings of the Report are not and cannot claim to have been made on anything approaching scientific grounds, yet many millions of dollars were spent to generate the Report and now to defend it.
- Investigator bias
- Instruction bias
- Anchoring bias and insufficient adjustment
- Overconfidence bias
- Polarized thinking bias
- Confirmation bias
- Attribution bias
- Deliberate introduction and use of pejorative labels and demonization efforts
- Bias toward deontologically determined ethical principles and standards
My specific request is that you place my call for validation studies on the agenda for your Board for careful consideration. Specifically, I propose that your Board vote to support the following motion:
Given that APA’s Board of Directors and Council of Representatives chose to by-pass the normal policy formulation and recommendation processes in accepting the Hoffman Report and making substantial and ground breaking changes in the Association’s approach to the recognition and regulation of practice and the application of psychological science to many matters in the conduct of General Applied Psychology, the Board of...calls upon both the Board of Directors and the Council to:
- Allocate sufficient resources to begin the work of scientifically validating the Hoffman legal team’s efforts and how those activities were used to formulate the findings of the Report.
- Convene a nationally recognized Task Force to oversee the validation work. The Task Force should be comprised of a diverse array of scientists and scholars in psychology. The Task Force should not have members representing any of the groups or individuals associated with, advocating for, or mentioned in the Report. The Task Force should be expected to create a reasonable plan of work that would allow the full governance structure of the Association to determine whether the Hoffman Report is sufficient in and of itself to guide the formulation and implementation of new policies and processes related to the science, practice, and education of various subdomains of psychology, including, but not limited to, what has become known as operational psychology.
- Based on the recommendations of the Task Force, allocate sufficient funds to allow independent groups of psychological scientists and scholars to examine the Hoffman Report, the associated materials, and the associated processes and procedures enacted by the Board of Directors, the Council of Representatives, various advocacy groups, and APA Boards, Committees, Task Forces, State Associations and Divisions to determine whether there are necessary and sufficient scientific grounds to support changes in APA policies, procedures, and policies. The work of these scholars must be made public for the entire membership to consider in depth and in such a fashion so that timely discussions can be had within and between all of the components of the Association.
- APA’s Board of Directors should move with all due speed to create a public repository of all documents associated with the Hoffman Report, including the notes associated with the interviews conducted during the investigation, and other related materials that were outside of the scope of the Hoffman Teams’ activities but clearly related to their findings and the subsequent actions taken by the Board of Directors and the Council of Representatives (see the call for scientific validation for suggestions as to the other materials that should be included in the repository.)
- APA’s Board of Directors be encouraged to suspend any and all activities associated with the implementation of the Council Resolution until the results of these validation efforts provide sufficient additional scientific evidence that either supports or invalidates the findings of the Hoffman Report.
Quite frankly, there are many, many other problems with the Report. However, the entire matter has become so deeply politicized and polarized within the Association, that I believe that any out right challenges to the validity of the document and thus of the actions taken by the Council of Representatives, Board of Directors, and other advocacy groups and organizations within and outside of the Association would be met with a profound combination of open hostility and denial. In light of that, I am asking that your Board specifically review the motion I am forwarding along with this short and focused rationale. I am advocating that you and every member of your Board remember that APA was founded as the first organization in the world to promote psychology as a science distinct and distanced from philosophy, law, and religion. The science of psychology as we all know is based upon a dedication to the scientific method, to the open dialogue between scientists about their rationales, hypotheses, experimental designs, types of data and their reliability and validity, methods of analysis, descriptions of results, and discussion of the implications of their findings. APA’s Board of Directors in 2014 made an explicit policy decision not to trust the science of psychology to help them answer the complex and difficult challenges they faced. Instead of calling together an internationally recognized group of appropriate experts in the discipline and supplying them with the resources necessary to investigate the allegations and issues, they turned to a law firm and a lead attorney with major prosecutorial experience. It has been reported that the Board of Directors spent nearly $5 million dollars on the Hoffman Processes and Report. In line with the rationale provided above, I believe there are serious scientific grounds to challenge the substance of the Report. I would ask you and your colleagues to devote a reasonable measure of your time discussing these issues and what I propose.
I want to further point out that the Board of Directors and the Council of Representatives took explicit and deliberate actions in June, July, and early August of this year to prevent you from engaging in such deliberations. I would also further point out that one of the major criticisms of what occurred with the PENS Report and processes was that the Board of Directors and Council of Representatives at that time also deliberately subverted the normal processes of policy formation in the Association and did not permit the full range of debate and expression of the diverse array of views for which APA governance has become so legendary. If that had happened at that time, our Association might well be in a very different and productive place with regard to these matters. And if you concur that there are any questions that your Board has about the Report or the processes used by our Board of Directors and Council of Representatives, please give careful consideration to supporting the motion above and forwarding your vote on the issues to the Board of Directors and Council of Representatives. And if you should take your own direct action in response to this letter after careful deliberation, you shall have my complete and public support for what you decide to do.
Please let me know if you have any questions or need any additional information in support of this request.
Richard R. Kilburg, Ph.D.
CEO RRK Coaching and Executive Development
Board for the Advancement of Psychology in the Public Interest
Board of Professional Affairs
Board of Scientific Affairs
Board of Educational Affairs
My main rationales for requesting support for the motion above come from a very careful analysis of the Hoffman Report. In my essay on the subject and as a result of additional considerations, I have identified at least 9 forms of systematic bias in the Report and its findings that have gone largely without comment by either the Board of Directors or the Council of Representatives. These include:
- Investigator bias – the scope of the investigation as charged by the Board of Directors was specifically formulated to narrow the vision and therefore the types of questions and the interpretation of evidence the Hoffman Team pursued and provided.
- Instruction bias – similarly, the Board of Directors formally targeted areas and activities they wanted the Hoffman Team to focus upon. In the executive summary and body of the Report, the authors stated that they explicitly followed those instructions and produced exactly what the Board requested instead of conducting the kind of broadly useful investigation that would have informed policy, process, and procedures and not led to the kind of immediate political action taken by the Association’s leaders.
- Anchoring bias and insufficient adjustment – it is clear in the Hoffman Team’s introduction to their report that they made no concerted effort to examine the charges they were given, determine alternative scenarios that could reasonably be hypothesized to explain the situations faced by the Association and adjust their methods accordingly. As was clearly stated in the Report, the Hoffman team was provided with a wide variety of alternative points of view and explanations of events, all of which they interpreted in terms of the charges given them by the Board of Directors. And in virtually every case in which the Hoffman Team interpreted the motives, means and contents of communication, and specific activities, they chose to describe those sources of information in terms of the basic prosecutorial framework within which they chose to operate.
- Overconfidence bias – in their introduction to the Report, the Hoffman Team explicitly cited all of the aspects of the Association with which they were unfamiliar, the challenges of assimilating the huge archives of information they had to study, and the complexities of the situations the organization presented. Nevertheless, they stated with strong assurances that they had found the unqualified truth of the matters that they were asked to examine.
- Polarized thinking bias – repeatedly in the body of the document, the Hoffman Team drove itself into a narrative form that had the critics of the PENS Report and processes on one side of the events and military psychologists and their supporters on the other. Within the first ten pages of the Report, it was clear to any reader within the context of a-d above what the Team would report and that they had determined who were the heroes and who were the villains.
- Confirmation bias – when any investigator starts to collect data and integrate it with biases to form hypotheses, explanations, and findings, they most often become perpetrators of the active process of then looking for evidence in the available data to support those initial expectations rather than carefully screening all data for its relevance to the events and questions under investigation. In my essay on the Report, I pointed out that there were literally dozens of specific places in the document in which the Hoffman team selected specific pieces of only some emails to support their views. Even worse, data were often presented or alluded to as confirming their take on events and their interpretations of motives of various individuals based on the 150+ interviews they conducted with individuals. Despite calls from others and myself, the actual summaries of those interviews were never included in the Report. Nor have they been made public so that the assertions and allegations of the Hoffman team could be correlated with what people actually said during the interviews. Even worse than that, a small survey of 8 of the interviewees by me revealed that no one was invited to have an attorney help them during this extra legal process and there was no court reporter contracted to collect an accurate record of the proceedings. Instead, attorneys from the Hoffman Team “took notes” during the proceedings. Later the Board of Directors was told that the Hoffman Team met and carefully reviewed their collective notes to assure themselves of the accuracy of what the interviewees actually said, despite the fact that they had no objective record of those remarks. In addition, no interviewee that I talked to or corresponded with was allowed to review the notes from their meetings nor were they permitted to correct the record that was used to draw conclusions. They were also not permitted to see a draft of the Report in advance or asked to comment on what the Hoffman Team alleged that they said. When several of the individuals who were interviewed asked whether they should be represented by counsel, the Hoffman Team members explicitly discouraged them from taking advantage of such resources. Several reported to me that they were told “it would look bad if you brought your own attorney to the meeting.” Any such process or procedure introduced into a psychological experiment would be identified immediately by reviewers as unusable because of the inability to attest to the reliability or validity of the information thus obtained. Yet, here the Association stands, defending a contracted Report by lawyers in which not only the accuracy of information contained in the document but also the very existence of such data can legitimately be called into question. This is the very problem of the wholesale fabrication of research data that has repeatedly come to light over time and in many domains of science. When such processes and deliberate fabrications are brought to light, every scientific body takes immediate action to check and double check every aspect of such studies. And when errors and fabrications are discovered, those investigations and findings are rescinded and the professionals involved are sanctioned appropriately. The scientists who are challenged are expected to immediately offer their raw data and every method and process they used in the conduct of their research to public scrutiny. APA has moved systematically in the opposite direction. The data being questioned are being withheld deliberately from the members of the Association, the scientific community, and the public at large.
- Attribution bias – in dozens of places in the Report, the Hoffman team used quotes from these alleged notes or quotes from selected emails to make specific attributions about the motivations of individuals. In the vast majority of these attributions made about the villains identified by the Hoffman team, the worst possible interpretation of events was most often made, and it nearly always was in the direction of supporting the initial charge to the Hoffman Team by the Board of Directors to look for evidence of collusion. A striking number of these attributions were tied to footnotes referencing the interviews described above, yet no specific data were ever reported from any of the interviews. And as stated above in fact, as of this date, the Board of Directors has either not responded to or rejected any requests to examine those records. This flies directly in the face of the APA Ethics Code governing research reports, findings, and record keeping, -Standard 6.01 (2) allowing for the replication of research design and analysis.
- The Hoffman Team deliberately introduced and consistently utilized the most pejorative and negative terminology possible when referring to deliberations by the PENS group and subsequent reviews of various bodies within APA of Ethical Standards 1.02 and 1.03 dealing with conflicts psychologists experience in balancing the requirements of adhering to the Ethical Principles and Standards and those of laws, regulations, and governing legal authorities and organizational demands. Specifically, the Hoffman Team adopted the term “Nuremberg Defense” – a term introduced and consistently advocated by what the Hoffman Team repeatedly referred to as the “Critics” of the PENS Report and processes - in virtually every comment or allusion to the challenges involved, thus invoking a conscious association between those individuals working as operational psychologists and in other roles in the Department of Defense and the Security Agencies of the U.S. Government and Nazi’s tried by the allies after the Second World War as war criminals. The use of that terminology was repeated deliberately and for effect and created in the mind of any reader the impression that the group of individuals who were identified as the villains were every bit as guilty for what they had done as the Nazi’s leading up to and during the Holocaust. And this was done despite specific education of the Hoffman Team by any number of the people they interviewed about the complexity and difficulties faced by virtually every professional psychologist in trying to balance these demands of interpreting and enacting the APA Ethical Principles and Standards in such situations, including those who work as full time private practice psychotherapists, school psychologists, counseling psychologists who practice in institutional settings, consulting psychologists, and health service providers working in private and governmental institutions. It was frankly one of the most remarkable and egregious types of bias introduced by the Hoffman Team.
- In a number of places in the document in which the Hoffman Team dealt further with the allegation that the PENS work group, APA staff, and members of APA governance were involved in detailed, complex, and nuanced considerations of the requirements of managing the commitments to the Ethics Code in national security interrogations, they introduced the terminology of deontologically based ethical principles. This is highly technical language based in the ethics branch of philosophy. Its standard utilization is in the explanation of duties according to the basic parameters of Immanuel Kant who called for increasingly clear and detailed specification of the duties of individuals to God, the Church, and their fellow humans. External ethics consultants were interviewed and cited at several places in the document. Every one of these references and citations called for increased specificity of APA’s Ethical Principles and Standards. In no place in the document could I find even a reasoned explanation of the opposite point of view. In addition, the Hoffman Team failed to explain that the deontological or Kantian School of ethical thought is but one point of view within the philosophy of ethics and duty. The Report thus specifically and deliberately introduced a unique kind of prejudice into their arguments by the explicit exultation of tighter Kantian and Christian oriented standards with the simultaneous demonization of Standards 1.02 and 1.03 by consistently associating those standards with Nazi terminology and imagery.
The Hoffman Report: A Call for Scientific Validation
By Richard R. Kilburg, Ph.D.
APA’s recent publication of the Hoffman Report and its subsequent policy decisions and associated actions by Divisions and other related organizations are all based on what is, in essence, the face validity of the Report itself and the self-reported processes that the Hoffman legal team used to construct its findings. Many have called for wide-ranging, new initiatives based on those findings and some of those calls, including those for revisions in the APA Ethics Code and Standards, could have significant long lasting effects on the science and professional practices of every member of the Association. Before further action is taken based on the Report, it seems reasonable to require that APA’s Board of Directors and Council of Representatives take responsible steps to examine the validity of the Report and its findings. This is fully in keeping with the history and central values of the science of psychology, which require that any findings, particularly those thought to be extremely innovative or sufficiently controversial, be fully examined and challenged before being accepted into the common cannon of psychological theory and practice. In light of this, it seems reasonable to require that the Board of Directors and Council of Representatives take appropriate actions to mobilize the resources of the Association to those ends. Among the steps that must be considered include requests to:
- The Division of History to provide a thorough examination of the Report, the associated materials, including access to the contents of the interviews that were conducted, the emails and personal communications of governance members and involved others – not just what was included in the binders – and any other materials needed to determine:
- The accuracy of the historical narratives provided in the Hoffman Report
- Further explication of associated historical narratives that were touched upon in the Report but not fully delineated including:
- The work of military psychologists and others in the DoD and U.S. Intelligence Services to revise the U. S. Army field manual and to end abusive and torturous practices by U.S. personnel and contractors.
- Insofar as possible, the history of intelligence interrogations conducted by DoD and U.S. Intelligence Services after the adoption of the PENS Report and the revisions to the U.S. Army Field Manual on Intelligence.
- Detailed exploration of the deliberations of the Boards of Directors during 2014 and 2015 leading to the decision to contract for an extra legal review, the choices of how to charge the Sidley team, the choices about what to do with the Report including personnel actions taken, and the active inclusion of only limited individuals in considerations of follow up implementations.
- The construction, presentation, and word-smithing of all of the drafts of the Resolution passed at APA’s Council of Representatives meeting in August of 2015.
- The active lobbying and advocacy undertaken by all parties inside and outside of the Council of Representatives and Board of Directors during 2014 and 2015 regarding these matters.
- In so far as feasible, a review of any and all activities undertaken by members of APA governance and the Association in general outside of the organization’s normal channels to bring public scrutiny and pressure on the enterprise to take action to reverse previous policy positions.
- An examination of the historical records of the organization to determine whether the efforts undertaken to influence and work with DoD during the PENS process were different in kind or scope from those undertaken by the organization routinely to engage the various branches of the U.S. government on other matters of importance to APA and is members.
- A close study of the supervision processes of APA staff members by APA governance to determine whether those processes exercised during the PENS study and associated activities afterwards were different in kind and substance from those used routinely.
- Finally, an examination of the work of APA staff with members of DoD and the security services during the PENS process and related actions afterwards to determine whether those activities were different in kind and scope from those undertaken by all staff and with other members and constituencies on matters of dire importance to them in the past.
- The work of military psychologists and others in the DoD and U.S. Intelligence Services to revise the U. S. Army field manual and to end abusive and torturous practices by U.S. personnel and contractors.
- The accuracy of the historical narratives provided in the Hoffman Report
- Divisions 12, 16, 17, 29, 38, 39, 42, and other clinically oriented units of APA to examine the literature on the quality, effectiveness, and safety of psychotherapy and other health care interventions used by psychologists to provide clarity on APA ethical principles and standards on Beneficence, Doing No Harm. Competence, and effectiveness of services.
- Divisions 13, 14, 18, 19, 27, and other General Applied Practice in Psychology to examine the literature on quality, effectiveness, and safety of non psychotherapeutic, general applied interventions used by those practitioners – see 2 above.
- Divisions of Social Psychology, General, Experimental, Industrial/Organizational, and Consulting Psychology and others interested to examine the political, interpersonal, organizational, and leadership processes, structures, and procedures involved by APA’s Boards of Directors and Council of Representatives from 2002 through 2015 dealing with these issues to derive lessons that can be used by APA governance and staff going forward to avoid similar explosions of conflict.
- Division of Forensic Psychology to create a task force to examine the Hoffman Report closely for its lines of argument, methods of data collection, analysis, and conclusions drawn, and lines of argument and conclusions drawn related to data inclusion and exclusion processes. In short, we need a team of professionals who are trained in law and psychology to study the Hoffman document and documentation to determine whether the levels of professionalism and competence displayed by the Sidley team met the expectations of both the professions of law and psychology and the science of psychology. Although this may be a very high bar for any group of practitioners to get over, the impact of the Hoffman Report, the decision of the Board’s Special Committee to contract with them and charge them the way in which it did, and the actions and activities of the Sidley team subsequent to agreeing to conduct the investigation, all make such a thorough, multi-disciplinary examination necessary. For the Sidley team had to have known that it was working for an Association with the best psychological science and practice expertise, experience, and knowledge in the world. And if it did not make sufficient use of those areas of expertise in the conduct of the investigation and in so doing produced a document of limited value, then the members of the Association should be educated about what mistakes were made, the potential impacts those oversights may have produced, and the lessons that must be drawn about using such methods to assist the organization in addressing such matters of significant internal conflict and external interest to the American public now and in the future.
- Others may well have additional suggestions to make about how APA and its members may need to study and learn from what has transpired. I know I am not the only member of the organization to have expressed such concerns and have such perspectives. I can only hope that APA governance, especially the Board of Directors and the Council of Representatives, can free itself from any and all defensive postures, including those recommended by law firms contracted to help them with what undoubtedly are going to be substantive and painful matters of litigation, to see the need for moving forward in the most productive and scientific fashion possible. Without the rigor of such examinations, the entire membership will be left to take sides based solely on the language and rhetoric of the Hoffman Report itself and the various ways in which those materials can and have been interpreted.
- I am sure it will be argued that the costs and time involved with the conduct of such validation work would prohibit the Association from making such a commitment. But without such studies, I believe the Hoffman Report will stick to all of the members of the Association and the organization to come in ways that will make our collective work and mutual activities extremely difficult to undertake. It seems clear that the past thirteen years have produced a significant rupture in the social and psychological contracts that bind us together as such a diverse community of scholars and practitioners. Despite the size and direction of the vote on the Council of Representatives, these matters are not settled among us and will not be unless we take additional energetic action to do so. Calls for reconciliation, letters of apology, and actions taken to punish those thought to be responsible are not substitutes for serious and sustained efforts to look thoroughly and completely at what has transpired.
- Psychologists are routinely trained and educated to demand and expect more from each other in all arenas of science, education, and practice. We learn very early in our studies in psychology to take no research report at face value and to dig into it with imagination, intensity, critical thinking, and all of the tools our science and profession have developed. So, let us try to cease the natural, human inclination to point fingers, take sides, demand political, personal, and organizational action, and instead get down to the business at hand. We must study what has transpired in the greatest detail possible in order to learn from what everyone involved on the staff and APA governance over the past 13 years have produced. Our Association is well over a century old. The science of psychology started in Germany in 1879. The practice of psychology started in the 1890’s at the University of Pennsylvania. Let us all agree to honor our collective heritage and preserve our mutual respect and dignity by using what we know how to do better than any other group of human beings on the planet. Let us really try to learn everything we can from what has happened before we try to do anything else.