Site of the Delphi oracle in Greece (Photo creative commons lic. by Lance McCord)
When I started reading the scholarly futures literature, one of the methodologies that always seemed somewhat mysterious to me was the Delphi survey method. First, is the association with the Delphi oracle – the influential prophet/seer of classical Greek mythology. Second, the "Delphi" is always capitalized when the method is mentioned, which led me to assume that this was a protected proprietary method of limited access to a budding scholar such as myself. Third, whenever the Delphi method is mentioned, it is accompanied by a narrative discussing multiple rounds of surveying, reflection, feedback and so forth, which makes it seem almost unrealistically complex, except for major studies with access to extensive resources. Fourth, there seem to be considerable differences in how Delphi surveys are administered in individual cases. All in all, this lent the Delphi method an air of mystique, almost as mysterious as the Greek Delphi oracle herself. Yet, the method has been widely used in futures studies, and with good results. It is, if not an important method to be able to use in futures studies, one that any futurist has to be familiar with.
Like so many futures methods, the Delphi survey method was developed in the 1950s to 1960s by Project RAND, and further developed by the RAND Corporation, which developed out of Project RAND. In short, the Delphi survey method is a technique for structuring communications using anonymous multi-round surveying that is intended to generate consensus among a broad range of subject-area specialists on issues of concern. Generally a Delphi survey takes the form of a questionnaire where specialists are asked to indicate their judgments or rankings on the feasibility of the issues described or, in the case of technology forecasts, to indicate the approximate time period that they expect the technology to become available. The surveys are then collected by the team responsible for administering the survey who analyze and summarize the responses with regards to emerging trends. Participating specialists are then given an opportunity to review an anonymous summary of the outcomes of the previous round of surveying before they are surveyed again. In many Delphi surveys, specialists are asked to justify their responses in cases where they differ considerably from the majority of survey participants. This cycle of survey-reflection-survey continues until participants' responses stabilize, i.e. the difference between survey rounds is insignificant. Usually Delphi surveys require two to three survey rounds to stabilize.
One of the things that fuels the mystique about the Delphi survey method for me is that, although the general description above holds more or less, there is a seemingly endless abundance of variations on the Delphi method. In current literature we encounter, "policy Delphi", "real time Delphi", "dynamic Delphi", and the ever present, non-descript "modified Delphi". Despite this variation, all approaches are consistent in terms of what they aim to achieve. As was previously mentioned, the purpose of Delphi surveys is to generate agreement between specialists on specific matters. Yet, it is not just a mere matter of identifying points of convergence in specialists' thinking. Rather, it is a matter of constructing, from privileged expert knowledge, non-biased, shared agreement on what developments and occurrences in our environment mean. There are two important aspects of the Delphi method that support this aim. The first is the anonymity that aims to avoid the social and psychological factors that influence the formulation of shared meaning in face-to-face settings ensuring that no individuals dominate the process by virtue of their personal traits. The second is the forced self-reflection on one's beliefs inherent in the repeated surveying and review processes. This ensures that all individuals' final responses are based on a collective sense of the state of affairs that impact on the situation being analyzed.
Assuming that I have accurately portrayed the core aims of the Delphi method, it would appear to allow for considerable variation in how the surveys are conducted, as long as any individual approach holds true to those core aims. This is a good thing because initial formulations of the method placed it far out of reach of many organizations and groups that stand to benefit from it. Early Delphi studies involved dozens of experts and the surveys were paper-based, which required surveys and responses to be "snail-mailed" back and forth between facilitators and experts. The result was that Delphi studies were prohibitively costly and time-consuming, and, consequently, only used in very limited instances. Alternative approaches have made the process far more manageable. Many alternative approaches use information and communication technology (ICT) to simplify the management process. The Real Time Delphi and web-based Delphi, for example, use ICT to eliminate the need to mail paper-based surveys back and forth. They also use ICT to give participants immediate feedback on survey results using dynamically generated graphs. The dynamic Delphi and the Real Time Delphi also eliminate the need for distinct multiple rounds of surveying since the web-based systems allow participants to change their rankings, judgments, etc., immediately, in reaction to other participants responses.
Although alternative approaches to the Delphi method make it more manageable, the reliance on large numbers of experts still limits its usability for small and medium-sized organizations. The first hurdle is finding experts that are able or willing to participate and the second is the cost that is likely to be associated with contracting sufficiently large numbers of qualified experts. It seems to me, however, that, at least in some cases, small and medium-sized organizations may be better off involving smaller numbers of experts and, instead, involving members of the communities that they serve. This would especially make sense if we accept that the Delphi method is essentially a meaning-making activity. For many types of organizations the community that they serve is no less an important component of making sense of the issues that they are confronted with than the knowledge of experts. A good example is educational organizations. Yes, experts can be an important source of information regarding management, pedagogy, etc., but educational organizations also reflect the values and culture of the communities that they serve. In fact, according to many scholars and commentators, it is precisely the range of values and cultures in communities served by educational organizations that makes educational change so difficult. So, it seems to me that educational organizations would be prime candidates for a novel Delphi approach that involves experts, along with community members, to generate shared contexts for future organizational and educational development.
Thus we arrive at my current pet project. I am constructing a low-cost web-based system for conducting "quick & dirty" Delphi surveys for educational organizations. The system is based on freely accessible open source software that has been modified to align with the core aims of the Delphi method as I have described them above. The approach used is derived from some of the modified Delphi methods that I have described, particularly the policy Delphi, the dynamic Delphi, and the Real Time Delphi. The system supports the random selection of survey participants from a pre-determined pool of potential participants that includes experts, parents, community members, and, perhaps, even students.
I plan to post a paper describing a prototype of the system, with screenshots, in about a week or so. If anyone is interested in this system and, especially, if someone would like to try it out, let me know.
Bibliography
(some references are not complete – They should be locatable in Google Scholar)
Adler, M., & Ziglio, E. (1996). Gazing into the oracle: The Delphi method and its application to social policy and public health. Jessica Kingsley Publishers.
Brown, B. B. (1968). Delphi process: A methodology used for the elicitation of opinions of experts. Santa Monica, CA: RAND Corporation.
Chou, C. (2002). Developing the e-Delphi system: A web-based forecasting tool for educational research. British Journal of Educational Technology, 33(2), 233-236.
Clayton, M. J. (1997). Delphi: A technique to harness expert opinion for critical decision-making tasks in education. Educational Psychology, 17(4), 373-386.
Colton, S., & Hatcher, T. (2004). The web-based Delphi research technique as a method for content validation in HRD and adult education research.
Gordon, T., & Pease, A. (2006). RT Delphi: An efficient, "round-less" almost real time Delphi method. Technological Forecasting and Social Change, 73(4), 321-333.
Linstone, H. A., & Turoff, M. (2002). The Delphi method: Techniques and applications. (available online)
Pollard, C., & Pollard, R. (2004/2005). Research priorities in educational technology: A Delphi study. Journal of Research on Technology in Education, 37(2), 145-160.
Scheele, D. S. (2002). Reality construction as a product of Delphi Interaction. In M. Turoff & H. A. Linstone (Eds.), The Delphi method: Techniques and applications (pp. 35-67).
Turoff, M. (1970). The design of a policy Delphi. Technological Forecasting and Social Change, 2(2), 149-171.
Wicklein, R. C. (1993). Identifying critical issues and problems in technology education using a modified-Delphi technique. Journal of Technology Education, 5(1), 54-71.
Yao, J., & Liu, W.-N. (n.d.). <em>Web-based dynamic Delphi: A new survey instrument.</em>
2 Responses to The Delphi survey method: Towards a cheap, “quick & dirty” approach for educational organizations