Search for a Paradigm, Part I

Introduction

Even if offices of institutional effectiveness and planning have the resources and the portfolio of institutional research to contribute to scientific research in education, the resources and research must be organized by structures that guide the agenda, inform funding decisions, and monitor the advancement of the work (Design Principle 2 extrapolated from the National Research Council’s proposal for scientific research in education). In this regard, the challenge for institutional research is twofold. On the one hand, the literature on institutional research by scholars of higher education precludes the profession from participation in activities that aspire or conform to the structure of science. On the other hand, the origins of institutional research exemplify the formation of structures of a social science dedicated to the study of higher educational settings. A first step for the future of institutional research as a social science necessarily entails a refutation of the former in order to realize the full potential of the discipline as exemplified in the origins of its administrative practices – a critique of what passes as common sense among scholars of higher education, the Michigan State School of Institutional Research.

For the latter challenge, the National Research Council’s (NRC) statement on scientific research on education expects no less: the research process itself is “highly contested territory,” often producing equivocal and non-durable results, explored via multiple methodologies, and clarified in periodic “syntheses of research findings…” (2) Understandably, the organization and integration of offices of institutional research as full participants in the advancement of scientific research on higher education has been subject to fits and starts, reversals, and ruptures like any other scientific enterprise. Nonetheless, institutional research remains vital, actualized and clarified as a social science in its praxis, and whether fully articulated or not in the literature. The NRC notes, “Every scientific inquiry is linked, either implicitly or explicitly, to some overarching theory or conceptual framework that guides the investigation.”(3) The assertion is no less true of institutional research than any other scientific enterprise. The second step for institutional research as a social science, therefore, is to move from implicit engagements of theory to explicit statements of theory that re-discover the principles which engendered the formation of the original offices and evidence the accumulation of knowledge from their operations – the search for a paradigm of institutional research as social science.

Creative Output on Institutional Research

In his widely-cited work on the structures of science, Thomas Kuhn notes, “Paradigms gain their status because they are more successful than their competitors in solving a few problems in  that the group of practitioners has come to recognize as acute.” Among the problems that paradigms solve, the first is the designation of a “class of facts” that come to represent the “nature of things.” In that respect, the paradigm holds a priority in the practice of science by guiding what passes as fact, what stands as the next problem to solve, how to approach a solution to the problem, etc. The paradigm itself, however, does not need to be explicitly identified, fully interpreted, or provide “discoverable rules” for the practice of the science. The community of scientists organized within the paradigm nonetheless form through the process of professional initiation to the discipline or field of study wherein the theoretical applications of the paradigm are presented with ready-made solutions demonstrating the force of the paradigm in coursework and training. “While paradigms remain secure,” Kuhn concludes, “they can function without agreement over rationalization or without any attempted rationalization at all.”^[1]

The Michigan State School of Institutional Research – first formalized in the works of Paul Dressel and Joe L. Saupe that rooted institutional research to the particularity of institutions for the production of ephemeral, non-generalizable knowledge – represents such a paradigm, one that remains secure with apparent agreement and relatively few attempts to rationalize its direction of institutional research over the past forty years. The most recent Handbook of Institutional Research (2012) begins with a history of the profession that considers a “debate” regarding the scientific potential of institutional research. Four brief paragraphs that reference documents from the early 1960s to the early 1970s provides no substantive review of the “debate” and concludes with reference to Paul Dressel, who “as the founder of the Office of Institutional Research at Michigan State University, took the middle ground…” with his doctrine of institutional particularity. Presumably, as the section ends after a literature review of a “debate” with documents from 40 and more years ago, Dressel is credited with the final word on the scientific or scholarly limitations of institutional research for higher education.^[2]

Given his stature and his school’s influence over the practice of institutional research in the past forty years, Dressel’s ideas perhaps deserve more scrutiny than any other author of his generation. While such is beyond the scope of these writings, Dressel’s definition for institutional research served a specific purpose in his larger body of writings on scholarly and scientific research in higher education. Credentials as an institutional researcher notwithstanding, as a faculty member in the College of Education at Michigan State University at his retirement, Paul Dressel also contributed to the foundation of the Association of Professors of Higher Education, today known as the Association for the Study of Higher Education (ASHE).^[3] At its founding, as the title of the 3rd meeting of the association attests, ASHE members thoroughly considered the path for scholarship and scientific research on higher education to develop into a field of study and the new organization consciously sought “to distinguish between [a higher education department’s] role and that of an office of institutional research…” At the third meeting, Paul Dressel contributed a summary of his forthcoming work, co-written with Lewis B. Mayhew, another member of the new organization for professors of higher education, Higher Education as a Field of Study, a work that became a touchstone to measure progress in the field.^[4]

In a section of the complete work entitled, “Institutional versus Pure Research,” Dressel and Mayhew clearly delineate institutional research in order to promote an agenda for research on higher education as field of study. “Institutional research is directed to problems and decisions within institutions or systems of institutions… Research on higher education, in contrast…. may be conducted for many reasons: individual curiosity, increased personal understanding, theory development or validation, basis for policy formulation, arousal of interest and concern, or influencing the opinions and the decisions of certain groups in the higher education enterprise.”^[5] The imperative to distinguish between institutional research and pure research is readily apparent in these words, but the logic for the distinction is entirely lacking. In the whole of the work, Dressel and Mayhew do not explain why research within an institution or system of institutions may not be directed for the very same reasons as research on higher education. Do not presidents and provosts have a stake in influencing opinions and decisions of certain groups in the higher education enterprise? Does an institution not have a stake in the basis for policy formation? Would not an institution benefit from theory development or validation before taking an action to improve its operations? Is not increased personal understanding or even curiosity the prerequisite of research and development for a college and university?

Rather than a considered analysis, however, the authors invoke institutional research when they wish to define institutional research as “the contrast” to research on higher education by academics. Contrary to the individual curiosity and search for personal understanding by researchers on higher education, “An institutional researcher…. who becomes enamored of doing research for the sake of enhanced personal insight and increased stature among his fellows is not fulfilling his [sic] obligation.” (81) In a lengthy monograph on the requirements for research on higher education to become a field of doctoral-level study in the American academy, the authors note, “Higher education as a field of study is sufficiently complex as to require great strength from a number of [other] fields.” (173) In comparison, “The intelligent institutional researcher can quickly learn on the job most of what he needs to know about higher education problems.” (124) In light of the latter statement on higher education as a field of study, the difference between institutional research and research on higher education is most starkly evident in one line of the work: “Institutional research requires such a range of knowledge and proficiencies that the typical higher education program may be the least desirable preparation.” (124) Consequently, institutional research is wholly foreign to higher education programming and the advancement of institutional research of little concern to scholars of higher education.

When considered in the context of the parallel development of administrative institutional research and higher education as a field of study, Dressel’s final word on the role of institutional research in higher education reflects the solution to a problem for the faculty of higher education programs who sought to legitimize their own field of study. A comparison of statements regarding the differences between institutional research and research on higher education in Dressel’s edited collection, Handbook of Institutional Research (1971), and his co-authored work, Higher Education as a Field of Study (1974), will certainly provide more comprehensive and nuanced insight in to Dressel’s use of the distinctions between institutional research and research on higher education. For our purposes, it is enough to know that, “in the nature of things,” Dressel did not perceive the nature and progress of institutional research as integral to the nature and progress of higher education as a field of study. Quite the opposite, according to Dressel, higher education as a field of study is measured by its difference and distance from institutional research as a practice. For that reason, the “middle ground” or final word of Dressel on the role and function of institutional research, as uncritically adopted and promulgated in the literature and history of institutional research, has been and remains one of the primary barriers for the organization of institutional research offices for the support of scientific research in higher education.

Modern Times for Institutional Research

The solution for institutional researchers interested in scientific practices, presumably, would be as simple as applying Dressel’s terms and criteria for higher education as a field of study to institutional research as a field of study. That position, however, neglects the realization that the Michigan State School of Institutional Research is a foundational precept of the dominant paradigm for scholarship and research in higher education as a field of study. As Thomas Kuhn’s work on scientific paradigms suggest, a paradigm forms or adheres to a community and “the choice… between competing paradigms proves to be a choice between incompatible modes of community life.” (94) If so, the community of scholars that formed around the paradigm that regards institutional research as a contrast or antithesis to research on higher education are likely socialized, through their educational requirements and professional associations, to see and uphold the distinction. In that respect, if evidence or logic suggests that the distinction is unwarranted or refuted by evidence, then scholars working within the dominant paradigm will seek solutions to re-affirm the “class of facts” or “nature of things” in their worldview and conserve the mode of community life, or scholarship, to which they are accustomed. In other words, scholarship on higher education provides a published record of the community which “uses its own paradigm to argue in that paradigm’s defense.” (94)

Considered in that frame, the prodigious output on “the role,” “the function,” and “the nature” of institutional research by researchers on higher education over the past forty years – from Dressel and Saupe to the present – reflects the enduring commitment of higher education scholars to the paradigm embodied in Rourke’s and Brooks’ riposte to “the heart” – institutional research – of the managerial revolution in higher education. In each article, as the recent briefs on this site have endeavored to illustrate, a key passage will linger over the question of how administrative institutional research fundamentally differs from faculty research on high education, often favoring the distinction between applied research and basic research. After 2002, however, the National Research Council’s monograph, Scientific Research in Education, acknowledge that education is an “applied field,” effectively rendering moot the traditional distinction between institutional research and research on higher education. Confronted with this dilemma, higher education scholars likely had several options, but two, in particular: 1) to rethink institutional research as research on higher education or 2) rethink the distinction between institutional research and research in higher education. The former, as suggested in Kuhn’s work, would require an entirely new paradigm for research on higher education that would greatly expand the community of scholars and welcome institutional researchers into the fold of “higher education researchers.” The latter option, merely required a new “class of facts” to uphold the “nature of things” according to the existing paradigm and an affirmation of the rarified domain of research on higher education for the existing community of scholars. Anyone remotely familiar with the profession of institutional research will know that the first option has not occurred, so evidence of the second option likely may be found in the record of higher education scholarship.

In 2015, in a work entitled, Institutional Research and Planning in Higher Education: Global Contexts and Themes, Karen L. Webber (editor) and Victor M. H. Borden dedicate an entire chapter to “compare and contrast” institutional research and educational research in higher education in an effort to buttress the standard paradigm. To supplement the common distinction between basic and applied research, Webber and Borden cite the Organization of Economic Cooperation and Development (OECD) definition of research & development to add a third mode of research: experimental development, “systematic work… which is directed to producing new materials, products or devices, to installing new processes, systems, and services, or to improving substantially those already produced or installed.”^[6] With this tripartite division of research activities, the authors propose a revision to the standard distinction between institutional research and research on higher education:

[Research on higher education] is perhaps best placed on the cusp between basic and applied research. As an applied field of study, [research on higher education] is necessarily geared toward applied research but its standards and practices require theoretical underpinnings and so theory development is part of the realm. [Institutional research] is better positioned on this spectrum between applied research and experimental development, with a focus on developing new systems and services as opposed to products.^[7]

The OECD manual and its guidance for measuring research and development, of course, provides no guidance for such a distinction between research on higher education and institutional research. The introduction of a third mode of inquiry with which to triangulate institutional research and research on higher education services “the priority of the paradigm,” to use Kuhn’s terms (50-51). The authors, ceding that research on higher education is “an applied field,” claim a new “class of facts” that blocks a recursive consideration for institutional research as basic or “pure” research. Instead, the reader is invited to consider a third order of inquiry that explains how institutional research once again differs from research on higher education by the particularity of its questions and purposes.

More notably, however, is the chapter’s unabashed representation of the process of socialization to the dominant paradigm for institutional research and research on higher education in American programs for higher education. The chapter begins with the narration of a classroom exercise, “In a seminar on institutional research taught by one of the co-authors, doctoral students were asked what they thought most distinguishes [institutional research] from [research on higher education].” Clearly, as described, the exercise presumes that institutional research differs from research on higher education. The doctoral students are not asked to consider whether the pre-knowledge – from prior socialization in other courses or statements by their instructors – that they have brought to the discussion should be challenged or refuted. Instead, as if relating a Socratic dialog on inborn knowledge,^[8] the priority of the paradigm guides the authors’ account of the differences cited by the “students,” apparently without prior class preparation or any effort to conform to the expectations of their professors. To summarize the class exercise, the authors cite Dressel’s “formal case” for the distinction from the 1971 Handbook (see here), and conclude, “The informed views of the doctoral students, and the more formal characterization by Dressel and associates, capture well the theoretical distinctions between [institutional research] and [research on higher education].” (18) As with other works in this genre of quasi-scholarship on institutional research, the authors assume uncritically the need to distinguish institutional research from their own research^[9] and, in a work directed to international professionals of higher education, export worldwide a process of socialization to the Michigan State School of Institutional Research.

In 1992, E. Bernadette McKinney and John J. Hindera characterized institutional research as “in a pre-paradigm stage; no body of scientific theory controls the kinds of questions that can be asked or the kinds of answers sought [my emphasis].”^[10] Little has changed in the ensuing years, as asserted in a prior brief, and generally reflected in the literature. On the other hand, Robert M. Hendrickson recently reassessed higher education as a field of study and, taking Dressel’s and Mayhew’s criteria as the basis for measuring advancement of the “core knowledge” of higher education over the past forty years, reaches a positive judgment of his peers and profession: “The discussion of the core knowledge that is the foundation for the study of higher education demonstrates clearly that the study of higher education is close to or has met the criteria of a legitimate field of study in education.”^[11] These two perceived outcomes for the two professions in the past forty years are entirely related by Dressel’s theoretical framework which establishes institutional research as the contrast to research on higher education and the base condition against which higher education scholars distinguish their work and progress. In this sense, institutional research drifts in a pre-paradigmatic “stage” by design, as a conditional priority for the dominant paradigm for higher education as a field of study, and remains a domain of phenomena and methodologies that higher education scholars as a community neglect categorically as unsuited for the advancement of generalizable knowledge about higher education.

Institutional Research and Scientific Research in Higher Education

In effect, the forty year period in which Paul L. Dressel has been lauded as the final word on the practice of institutional research reflects an era of stagnation and restriction for both professions. The confidence that scholars of higher education as a field of study have in themselves and their profession’s past development merely reflects the certitude of their perceived distance from institutional research. As we have endeavored to illustrate in a few brief paragraphs on Dressel’s influence on both institutional research and research on higher education, the dominant paradigm in higher education as a field of study categorically denies the structure of science to institutional research. As a consequence, the dominant paradigm in higher education directs scholars of higher education away from the phenomena (“entities”) directly investigated and studied by institutional researchers, with a few exceptions for the offices of institutional research that are agreeable to their “role” in scholarly research.^[12] As a consequence, a paradigm for institutional research as a social science must, paradoxically and by necessity, dissent with higher education as a field of study and at the same time uphold the principles of scientific research in education. To this end, the final section of Part I of our search defines a few priorities for a new paradigm of scholarship that deploys the practice of institutional research as part and parcel of the structure of social science for higher education.

As Kuhn notes in his chapter on the necessity of scientific revolutions, paradigms exercise a powerful normative function that informs researchers understanding “about the entities that nature does and does not contain…” (109). The scholarship of higher education over the past forty years is characterized by the normativity and influence of a paradigm in exactly this manner. Thus, if institutional researchers are to be recognized as scholars and researchers on higher education, a new paradigm must take hold, for as Kuhn adds, “when paradigms change, the world itself changes with them. Led by a new paradigm, scientists adopt new instruments and look in new places.” (111) In this respect, a new paradigm for institutional research as a social science always already doubles as critical study of higher education as a field of study and the domain of phenomena it routinely ignores or rejects as not having a “natural” or “scientific” bearing on the study of higher education. In prior briefs, we have sought to challenge specific flaws in the paradigm: virtual offices, the conflation of the technical and analytical, a teleological historiography, etc., etc. At this point, as a first priority of a new paradigm, the current paradigm for higher education as a field of study must be regarded as the fatal flaw: research on higher education has been critically misguided in its rejection of administrative institutional research as an essential source of generalizable knowledge about higher educational settings.

The goals of the National Research Council (NRC) as stated in Scientific Research in Education serves as an excellent source for a new paradigm of institutional research on higher education. The NRC takes as its subject scientific research on education in general and its statement, in our analysis, is not compromised by the ideology or vested interests of the current paradigm for higher education as a field of study. The NRC upholds the general idea that “scientific inquiry is the same in all fields.” Rather than endlessly masticate on the “roles” and “maturity” of institutional research as a profession, the NRC promotes the practice of social science for education as a discipline: “Scientific research….. is a continual process of rigorous reasoning supported by a dynamic interplay among methods, theories, and findings.” More importantly, for this “continual process” of scientific enterprise to be successful, there must exist “a healthy community of researchers…. guided by a set of fundamental principles.”^[13] As a second priority of a new paradigm, then, the structure of science for researchers of higher education must provide a basis for the unmediated inclusion of institutional researchers in the community of scholars regarded as directly engaged in the study of higher education. The immense wealth of phenomena at institutions and the the portfolio of research under the direction of institutional research offices can be deployed in a manner consistent with the purpose of scientific research in education.

In addition, the NRC lists six guiding principles that “underlie all scientific inquiry, including education research,” for a healthy community of scholars:

1. Pose significant questions that can be investigated empirically.
2. Link research to relevant theory
3. Use methods that permit direct investigation of the question
4. Provide a coherent and explicit chain of reasoning
5. Replicate and generalize across studies
6. Disclose research to encourage professional scrutiny and critique

All six of these guiding principles for scientific inquiry are recorded in the research and publications of the original offices of institutional research from the mid-twentieth century. As a third priority, therefore, a new paradigm will accord to institutional researchers the tacit ability to pose significant questions and acknowledge the implicit engagement of theory in the standard administrative practices of institutional research. Conversely, the new paradigm will afford institutional research the opportunity to make explicit the theoretical basis of its inquiries, methods, and findings in order to engage the major stakeholders in higher education and facilitate honest scrutiny and critique of research agendas, funding, and work. In short, the new paradigm will not obsess over who asks the questions, the technological complexity of the methods, or the utilization of administrative resources to advance scientific investigation of higher education — a healthy community of scholars will be an academic-industry partnership in higher education that bridges the imagined divide between faculty and administration.

As an applied field, the study of higher education will require a more nuanced understanding of the practice of basic research than one that idealizes the supposedly indifferent academic pursuit of knowledge for knowledge’s sake, if such was ever the case for higher education scholars given “the common propensity of academic man [sic] to look with alarm upon any apparent extension in the power of university administration,” as Rourke and Brook observed. To the contrary, to enlist stakeholders from all six “cultures”^[14] of colleges and universities, a fourth priority of the new paradigm will consider favorably the perspective and interests of the “strategic apex” of higher education as a necessary prism through which researchers formulate significant questions to investigate and define the agenda of institutional research. This is not to say that each institution defines for itself the agenda of institutional research on higher education but, rather, that the strategic apex in general, by virtue of its position in the typical structure of higher education institutions, provides the practical measures for what constitutes settled and unsettled problems in the accumulation of knowledge about higher education.

Lastly, discipline is an important commitment that suggests a collective can be organized into a unit of activity at the level of individual conduct. As long as Institutional research as a profession lacks discipline, the institutional effectiveness of higher education colleges and universities as a whole will also lack discipline. As a fifth priority, a new paradigm for scientific research on higher education will endorse the primacy of social scientific research principles in the investigation of questions of general – and lasting – relevance to institutional leadership and governance. For the institutions of higher education, the fifth priority will require commitment to the study of higher education and the employment of institutional researchers with appropriate credentials and competencies for scientific research. Here also, though, we encounter the call to action for institutional researchers to define and map out the future of institutional research as a discipline. The work must be the work of a community of scholars hungry for debate, tolerant of dissent, open to critique, and in search of the next unknown – for no one scribe or scholar should ever be regarded or lauded as the final word for an entire discipline.

The fifth priority, thus, calls for the structure of discipline for institutional research. Part II of the search for a paradigm will further consider characteristics of the structure of institutional research as a social scientific field of study and as an administrative office for the advancement of higher education and higher education leadership.

Note: This post is subject to unrecorded edits through September 15, 2015.