Anna Marie Høstgaard
Department of Planning, Danish Center of Health Informatics, Aalborg University, Denmark
Pernille Bertelsen
Department of Planning, Danish Center of Health Informatics, Aalborg University, Denmark
Lone Stub Petersen
Department of Planning, Danish Center of Health Informatics, Aalborg University, Denmark
Christian Nøhr
Department of Planning, Danish Center of Health Informatics, Aalborg University, Denmark
Download article
Published in: Scandinavian Conference on Health Informatics 2013; Copenhagen; Denmark; August 20; 2013
Linköping Electronic Conference Proceedings 91:8, p. 33-37
Published: 2013-08-21
ISBN: 978-91-7519-518-6
ISSN: 1650-3686 (print), 1650-3740 (online)
Experience and time has shown a need for new evaluation methods for evaluating Health Information Technology (HIT); as summative evaluation methods fail to accommodate the rapid and constant changes in HIT over time and to involve end-users; which has been recognized as an important success factor in HIT development. A new evaluation methodology; including an analytical framework; has been developed specifically for HIT development: Constructive Technology Assessment (CTA) for HIT. It offers solutions to both the problems associated with summative technology evaluation and a way to involve end-users. The CTA methodology is based on a Socio-technical understanding of technological development as an open ended; emergent process. The CTA was used during the EHR development process in the Region of North Jutland where it proved successful inproviding learning and feedback between all relevant groups during all the phases in the process. Thereby anumber of problems were prevented to occur later on.Thus; the CTA method and its framework are useful for evaluators and project-management in order to facilitate and support successful HIT development.
[1] Warfinge JE. EPJ i Danmark før regionsdannelsen i 2007. En undersøgelse af evalueringer af Elektronisk Patientjournal i klinikken. (Electronic Patient Record in Denmark prior to the formation of regions in Denmark 2007. An investigation into clinical evaluation of EPR). 2012;12-2.
[2] Fieschi M; editor. An inventory of evaluation studies of information technology in health care: Trends in evaluation research 1982 - 2002. MEDINFO 2004: IOS Press; 2004.
[3] Douma K; Karsenberg K; Hummel M; Bueno-de-Masquita J; Harten W. Methodology of constructive technology assessment in health care. International Journal of Technology Assessment in Health Care 2007;23(2):162-168.
[4] Dahler-Larsen P; Krogstrup H.K. Evaluering og institutionelle standarder: Nyinstitutionelle betragtninger af evaluering som vor tids ritual. 2000
[5] Sundhedsstyrelsen. Metodehåndbog for Medicinsk Teknologivurdering (Handbook in Methods for Health Technology Assessment). København; DK: Børglum Christensen; F.; Sigmund; H.; 2007.
[6] Müller J; Kjaer-Rasmussen J; Nohr C. Proactive Technology Assessment as a Tool in HIS Development. In: Barber B.; editor. MEDINFO 1989; Singapore North Holland; 1989.
[7] Talmon J; Ammenwerth E; Brender J; De Keizer N; Nykaenen P; Rigby M. STAtement on the Reporting of Evaluation studies in Health Informatics. Int J Med Inf
008;78:1-9.
[8] Ammenwerth E.; Graber S.; Herrmann G.; Burkle T.; Konig J. Evaluation of health information systems - problems and challenges. Int J Med Inf 2003;71(2):125-135.
[9] Bijker WE. How is technology made?—That is the question! Cambridge Journal of Economics 2009;34(1):63-76. [10] Schot J; Rip A. The Past and Future of Constructive Technology Assessment. Technological Forecasting and Social Change 1997;54:251-268.
[11] Müller J; Kjaer-Rasmussen J; Nohr C. Perspektiver for EDB-teknologi i sygehusvaesenet (Perspectives for ITtechnology in the hospital sector). Aalborg: Institut for Samfundsudvikling og Planlaegning; Aalborg Universitetscenter; Aalborg; 1988.
[12] Müller J. Teknologivurdering og trovaerdighed (Technology assessment and credibility). In: Cronberg T.; Friis D.; editors. Metoder i teknologivurdering. Erfaring og fornyelse Kobenhavn: Blytmanns forlag; 1990. p. 267-279.
[13] Scott J.T.; Rundall T.G.; Vogt T.M.; Hsu J. Kaiser permanente’s experience of implementing an electronic medical record: a qualitative study. Information in practice 2005 11/03.
[14] Robertson A.; Takian A.; Petrakaki D.; Crowe S.; Cornford T.; Barber N.; et al. Implementation and adaption of nationwide electronic health records in secondary care in England: qualitative analysis of interim results from a prospective national evaluation. 2010; . Accessed January; 2011.
[15] Kaplan B.; Harris-Salamone K. Health IT Success and Failure: recommendations from Literature and an AMIA Workshop. Journal of American Medical Informatics Association 2009;16(3):291-299.
[16] Bødker K.; Kensing F.; Simonsen J. Participatory IT Design. Designing for Business and Workplace Realities. : MIT Press; 2004.
(17] Pare G.; Sicotte C.; Jacques H. The Effects of Creating Psychological Ownership on Physicians’ Acceptance of Clinical Information Systems. Journal of the American Medical Informatics Association 2006;13(2):197-205.
[18] Kushniruk A.; Turner P. Who’s Users? Participation and Empowerment in Socio-Technical approaches to Health IT development. International Perspectives in Health Informatics 2011:280-285.
[19] The European Union. Redesigning Health in Europe in 2020. 2012.
[20] Cockburn A. Agile software development; the people factor. Computer 2001;34(11):131-133.
[21] Berg M.; Aarts J.; Van der Lei. ICT in health care: Sociotechnical approaches. Methods in Information in Medicine 2003;42(4):297-301.
[22] Edquist C.; Edquist O. Social carriers of Techniques for Development. J Peace Res 1979;16(4):313-331.
[23] Høstgaard AM; Bertelsen P; Nøhr C. Methods to identify; study and understand End-user involvement in HITdevelopment BioMedCentral: Medical Information and Decision Making 2011;11(57).
[24] Høstgaard AM. End-User Participation in Health IT Development: The EUPHIT Method. In: Zakariya Belkhamza SA; editor. Measuring Organizational Information Systems Success : New Technologies and Practices. 1st ed. Hershey; USA: IGI Global; 2012. p. 318-340.
[25] Høstgaard AM; Bertelsen P; Nøhr C. Evaluering af pilottest af Klinisk Proces "Vieweren" (Evaluation of the ‘Clinical Process’ Electronic Health Record (EHR) – Evaluation report). V-CHI tech reports 2010;11-3.
[26] A. M. Høstgaard. Fryder forandring? Casestudie af EPJ udbudsprocessen i Region Nord - belyst gennem en procesevaluering med fokus på lægerne som sociale bærere af den elektroniske patientjournal; EPJ (Change is the spice of life?; process analysis case study of a region in Denmark). Aalborg: Aalborg Universitet; 2009.
[27] Høstgaard AM. Evaluering af Præpilottest af "Klinisk Proces" på Infektionsmedicinsk afd. Aalborg Sygehus Syd (Evaluation of the Pre-pilottest of "Clinical Process" at the Department of Infectious Diseases). 2010.
[28] Nykänen P; Brender J; Talmon JDK;N.; Rigbye M; Beuscart-Zephirf MC; Ammenwerth E. Guideline for good evaluation practice in health informatics (GEP-HI) . International journal of medical informatics 2011;80:815-927.
[29] Mitchell R; Agle B; Wood D. Towards a Therory of Stakeholder Identification and Salience: Defining the Principles of who and what really counts. Academy of Management Review 1997;22(4):853-886.
[30] Kørnøv L. Public Participation. In: Kørnøv L; Thrane M; Remmen A; Lund H; editors. Tools for Sustainable Development Aalborg; Denmark: Aalborg Universitetsforlag; 2007. p. 721-738.
[31] Kensing F.; Blomberg J. Participatory Design; Issues and Concerns. Computer Supported Cooperative Work 1998;7:167-185.
[32] Jacucci G; Kensing F; Wagner I; Blomberg J; editors. Participatory IT-support. Participatory Design Conference; August 5th. 2006; : Association for Computing Machinery; 2006
[33] Falch AM; Jensen F. Erfaringer med udbudsformen ”konkurrencepræget dialog” (The Competitive dialogue process ). 2005.
[34] Kanstrup AM; Bertelsen P. User Innovation Management. Aalborg; DK: Aalborg University Press; 2011.
[35] Nielsen J. Ten Usabiligy Heuristics. 2005; Available at: http://zonecours.hec.ca/documents/H2010-1-2357287.portionOK.pdf. Accessed December; 2012.
[36] J. Kaipio. Usability in Healthcare: Overcoming the Mismatch between Information Systems and Clinical WorkAalto University; School of Science; Department of Computer Science and Engineering; 2011.
