My Practice A rising tide of dissatisfaction with physicians from healthcare consumers, payors and policy makers alike, is beginning to lead to calls for replacement of the long-held privilege of professional self-regulation with regulation enforced externally. Such external scrutiny is already possible through the increasing availability of electronic data, much of which is administrative in nature or generated as a by-product of the clinical encounter. Sophisticated data-mining then allows non-physicians to make inferences about physician behavior and performance that translate into potentially serious consequences for physicians (e.g. P4P). Physicians on the other hand, have either had scant access to such data sets, or have tended to reject them as invalid and unhelpful. This study proposes that physicians actively embrace such data sets and develop them as a key tool to drive and inform self-improvement activities. This could provide an important counterpoint to the uncritical analysis of physicians by non-physicians, and it might allow physicians to retake the high ground of “self-regulation of physicians by physicians”.
Towards a National Learning eFolio for UME
The purpose of this study is to convene a collaborative of medical schools and key national organizations and test the feasibility of fast tracking a consensus-based data sharing specification/standard to enable interoperability of many disparate UME learning systems.
With more than 130 medical schools and a host of commercial vendors, exploring, building or operating different eFolio systems, there is a strong potential for a "Tower of Babel" comparable to that which the Electronic Health Record domain is experiencing, where very little interoperability exists. Creating a stand-alone national eFolio system could provide a centralized solution to assure interoperability and transportability across the continuum of a physician's medical career. But, if we built it, would they come?
Pragmatic considerations suggest to a more decentralized or distributed approach that effectively provides linkage of current and emerging systems. Creating specifications/standards to define data elements and attributes, their provenance and related meta data is necessary to ensure interoperability is necessary. Consensus-based technology standards must be developed through collaboration with invested parties in the medical educational arena along with an ANSI approved standards organization with specialization in the field of medical education.
This project involves the creation of a Standards Group to define the standard and provide strategic oversight and a Working Group to provide technical expertise and build the standard.
Assessment of the Professional Learning and Working Environment
To fully appreciate an individual’s professional behavior, it is necessary to understand the context and environment in which those behaviors are modeled, nurtured and manifested. To this end, the Center has been pursuing development of a survey instrument that offers a snapshot of the local environmental culture as perceived by those working and learning there.
The instrument’s purpose is to evaluate the professional micro setting in which learning occurs. The project combines qualitative research being done on student narratives and work on the hidden curriculum, and is being conducted in two phases. The first phase focuses on research with medical students and residents in the inpatient setting. The second phase will add research with residents in the longitudinal/ambulatory care setting.
Natural Language Processing (NLP)
As regulatory organizations in medicine stimulate new assessment initiatives related to general competencies, Maintenance of Certification (MOC) and Continuing Medical Education (CME), there is increasing need for testing with multiple-choice questions (MCQs). Recent advances in computational linguistics have resulted in NLP technologies that might not only support increased development of MCQ exams and related assessments, but also open doors to new assessment methods. Possible applications of NLP are being explored in collaboration with a computational linguistics group.
The first Pilot testing the feasibility of NLP focused on automated computer generation of MCQs from free medical text. A system was created that produced a number of MCQs that, with minor post-editing, were viewed as potentially usable. A second feasibility Pilot exploring how information contained in vignette-based MCQs might be automatically extracted and used to develop tools to support item development and quality assurance was initiated and is currently underway. The first objective of this pilot led to the creation of a system that, with reasonable accuracy, automatically identifies, extracts and organizes the key atomic features of a vignette-based MCQ (e.g., patient age, gender, chief complaint, etc.). This Item Atomizer continues to be refined, and the next major step will involve rebuilding whole MCQs from the atomic pieces, with a view to creating a tool bench with utility for item writers (e.g., recommender system for distractors, key disease and treatment features) and for editorial staff (e.g., pool analysis, quality control). Several additional pilots are in the planning stages.
Data Sharing Through a Trusted Agent
The first pilot involved a collaborative agreement to link data repositories through a hub that enables electronic compilation of a Common Licensure Application Form in near real time. The pilot went live in three participating states (Kentucky, New Hampshire and Ohio) in late 2006 and early 2007, and rapidly demonstrated initial “proof of concept”. Survey data indicate positive responses on the part of applicants, and satisfaction remains high among the participating State Medical Boards.
The Center continues to explore a variety of other potential Trusted Agent data compilation services that could be provided to the individual throughout the training:practice continuum, to organizations, and eventually to the public. Examples include customized reporting of data to external agencies (e.g., for credentialing purposes), supportive tools for self assessment, portfolios to support and monitor lifelong learning and services in relation to CME.
Portfolios
Portfolios are increasingly in use or under development as educational tools for documenting learner achievement of competence and fostering reflection on practice with the intent of improvement. Simultaneously, electronic communication and dissemination systems are increasingly replacing traditional paper systems. A range of professional organizations are interested in organizing a community of practice to document professional formation across the continuum of education and professional practice. The aim is to develop an electronic portfolio infrastructure that would serve to seamlessly document learner experiences, reflections and outcomes as well as assessments by self and others, across the developmental continuum of medical school, residency and out into practice.
Comments