The MERG Suite: Tools for discovering competencies and associated learning resources
- Ravi Teja Bhupatiraju†1Email author,
- William R Hersh†1,
- Valerie Smothers2,
- Michael Fordis3 and
- Peter S Greene2
© Bhupatiraju et al; licensee BioMed Central Ltd. 2008
Received: 23 October 2007
Accepted: 14 May 2008
Published: 14 May 2008
As the demands for competency-based education grow, the need for standards-based tools to allow for publishing and discovery of competency-based learning content is more pressing. This project focused on developing federated discovery services for competency-based medical e-learning content.
We built a tool suite for authoring and discovery of medical e-learning metadata. The end-user usability of the tool suite was evaluated through a web-based survey.
The suite, implemented as an open-source system, was evaluated to identify areas for improvement.
The MERG suite is a starting point for organizations implementing competency-based e-learning resources.
Competencies in healthcare
Since the Accreditation Council for Graduate Medical Education (ACGME) published the Outcomes Project and the American Board of Medical Specialties (ABMS) endorsed the general or core competencies described in the ACGME project, medical educators across the continuum have sharpened their focus on development of competency-based education . Despite the challenges associated with implementing competency-based education , many specialty societies, boards, residency directors, and associations of medical educators are defining competencies for their specialty or discipline to guide the choice of learning activities and the assessments that learners must complete.
Concurrently, many educators are developing digital repositories of learning resources. These repositories provide rich sources of information but may be difficult to find and access, and the resources within are not always clearly associated with a formal system of competency-based education.
E-learning technology offers a way to deliver education and to assess learning and knowledge application by geographically dispersed and time-constrained physicians. Just as the adoption of electronic health records and their interoperability is being enhanced via adherence to emerging standards , implementation of effective e-learning may be enhanced by adoption of emerging e-learning standards . The health care community is only beginning to make use of standards for e-learning. The standards organization taking the lead in adapting e-learning standards to health care is MedBiquitous , an ANSI-accredited organization which is devoted to advancing healthcare education through technology standards that promote professional competence, collaboration, and improved patient care.
E-learning content creation can be expensive and require considerable investments of time by authors and developers. Early e-learning systems were tightly bound to the presentation, often hard coding the content into the presentation software or more recently dynamically generating learning content from databases.
Historically when Learning Management Systems (LMSs) were used to manage the delivery of e-learning, the learning activities were built on a specific LMS, and if the LMS changed, migration of content could be difficult and costly. In a world of rapidly evolving technology and advancements in learning platforms, consumers demanded the flexibility to migrate between LMSs without sacrificing investments in course development. Further learners and educators were challenged to locate and access relevant learning materials across LMSs or the Web.
The Sharable Content Object Reference Model (SCORM) emerged to address the lack of interoperability, reusability, accessibility, and durability. SCORM is a collection of standards and specifications for packaging, describing, and running e-learning content, as well as tracking user progress through content. In addition, SCORM allows for separation of content from navigation and deployment . One component of the SCORM model, the IEEE Learning Object Metadata (LOM) standard, standardizes the metadata about learning objects in an XML format. Healthcare LOM is an extension of this standard to meet the requirements of healthcare education and was designed by the MedBiquitous Learning Objects Working Group , a collaboration of over 30 individuals from academia, government and industry.
Work has been underway in the education sector to develop specifications for competencies and competency frameworks as well. The IEEE recently approved the Reusable Competency Definition (RCD) standard . This XML standard defines a data model for describing, referencing, and exchanging competency definitions, primarily for online learning. Reusable Competency Definitions provide a way to represent the key characteristics of a competency and enable interoperability among learning systems that use competency information.
The MERG suite
Leveraging e-learning standards, the investigators developed and implemented a prototype search tool called the Medical Education Resources Gateway (MERG). MERG allows users to: a) search the metadata records of competencies and e-learning content, and b) traverse linkages between selected competencies and associated content.
MERG provides a number of tools and services that make competency-linked health-related learning content easier to locate. Content creators and maintainers can easily author metadata, create public repositories to publish the metadata, and register their repositories with MERG. They can also associate metadata with competency frameworks. The suite monitors these registered repositories, harvests the metadata and competency associations as they change, creates metadata indices, analyzes the relationships and provides various search interfaces to discover and navigate the content and competencies based on the inter-relationships of these learning elements. The MERG suite, implemented on Java and Python platforms, is a portable collection of tools for content authors, system administrators, software developers and end users. All the tools are released as open source, enabling user customization.
Architecture of MERG Suite
Standards and Specifications Used by the MERG Tool
Advanced Distributed Learning – Sharable Content Object Reference Model (ADL SCORM) 
A suite of standards and specifications for online education that enables interoperability of learning content across learning management systems.
MedBiquitous Healthcare Learning Object Metadata (Healthcare LOM) 
Extends the IEEE LOM standard and provides a format for describing healthcare learning resources including images and e-learning activities.
IEEE Reusable Competency Definition (RCD) 
Provides a data model for describing and exchanging competency definitions.
Simple Reusable Competency Map 
An IEEE proposed draft prepared by Claude Ostyn that aggregates and defines relationships among competencies.
Competency Associations 
A research specification developed by MedBiquitous that enables educators to create associations between content and competencies.
Technical choices and evolution of MERG
XML database approach
MERG leverages several open source products in its implementation. The original design of MERG envisioned it as an application centred on an XML database since that would have provided both storage as well as search capabilities for the metadata in its native form without the need to maintain any further transformed data representations in synchrony. We used eXist , an open source Java implemented XML database that provided XQuery  capabilities for this purpose. XQuery is a W3C standard for querying across collections of XML documents.
Classical text retrieval approach
We later moved away from that design since the need for emphasis on proper ranking of the search results became evident. We used traditional text search techniques on these mirrors in place of XQuery. Apache Lucene , an open source Java toolkit for developing search engines was used for building the search components. The results were ranked using standard TF/IDF (Term Frequency/Inverse Document Frequency) ranking. This is a commonly used approach which ranks documents by the frequency of occurrence of search terms and their uniqueness in the document collection.
Lucene allows for considerable control over search strategies in an elegant way. We currently do not have enough data for rationally arguing for alternate strategies for this metadata corpus but opportunities exist in the future for the evolution of the product. Moving to Lucene also increased the search performance, but competency-based searching has not yet been implemented. We are now using the file system as mirror storage for the repositories.
There are currently ongoing open source efforts towards providing full text search support for eXist  and the results might be relevant to our implementation. Once these options become mature, the project may be able merge both the search approaches for a more natural design.
We used Karrigell , an open source pure Python based web server for Python based designs and used GlassFish , an open source Java Application Server for the later Java based designs. The early implementation was implemented in Python while the final implementation was on the Java platform with the exception of the simple harvester script, which is implemented Python invoking the GNU Wget  which provides the mirroring functionality.
The RSS and ATOM features of our application were enabled by the library provided by the ROME project .
Addressing various user profiles
The audience of MERG is a diverse group. It consists of authors who create competency-based content and metadata, system administrators who create the local metadata repositories, repository maintainers who update them, content consumers who search the aggregated repositories and software developers who may build on top of the existing infrastructure. Each of these user classes have distinct technical backgrounds and perform very different tasks. Addressing these multiple perspectives is one of the challenges of MERG. The modularity of MERG allows it to direct specific sub-components of the system to the user profiles that directly interact with them. Each of the user profiles, the tasks they perform, the sub-components the users interact with and some of the design decision decisions that went in to address those are described below.
To guide the system administrators who setup the repositories, the MERG suite provides a specification for the repository structure and an interface for registering repositories with MERG. It has a harvester component that visits registered repositories at a specified frequency to update its local copy of repository content. The repository structure and harvester functionality are intentionally kept as simple as possible. Individuals with skills to set up a Web server should have the skills to set up a metadata repository that can be harvested by MERG.
To simplify the task of repository maintainers, MERG repositories are specified with a simple structure intended for use by non-technical audiences, namely content and metadata authors. The repository is maintained by placing all the metadata files into a flat Web exposed directory. Metadata files conform to the Healthcare LOM standard and are assigned filenames with '.xml' extensions. The server's directory listing feature can be used to create a directory of the files if such a feature is available. Alternatively, the listing can be manually created, although this is not usually recommended since more effort or additional tools may be required to synchronize the listing with others in the repository. The metadata files need not be specially named or arranged, but are to be placed in the same directory. Competency definitions, maps, and competency-metadata associations are placed in the Web-exposed directory with the metadata files. MERG's Indexer is able to recognize the file types and processes them accordingly.
To assist the software developers in developing extensions and other value added services, MERG uses the W3C SOAP (Simple Object Access Protocol) 1.2 standard  to provide a remote interface to query the MERG search engine for metadata. This allows software developers to incorporate the end user functions of MERG into their applications without worrying about web user interface changes that may potentially occur in future.
Through a simple web interface, MERG enables users to query across multiple metadata repositories registered with MERG. The users navigate between related competencies and learning objects. Figure 3 shows the MERG search interface after a search has been performed and a LOM result from the search has been selected. The selected LOM record has a single competency associated with it. Figure 4 shows a selected competency record after a search. The selected competency record has no parent or child competencies.
A use case for the MERG Suite
A single coherent hypothetical use case scenario where various MERG Suite components come into play is described as follows.
Dr. X is a gastroenterologist and a medical educator who has prepared an online course for internists in his institution, Springfield Medical Center (SMC), on recent trends in hepatology. SMC uses a SCORM-compliant Learning Management System to deliver its content. The CME co-ordinator, Ms. Y, manages the various courses that either have been purchased or developed in house. Since Dr. X's content has been developed in house and the author has no special knowledge of the workings of Learning Management Systems, Ms. Y helps him by creating the metadata for his content and placing his course in the learning management system. Ms. Y herself has no special technical skills and is only modestly familiar with XML and certainly not with all the rules associated with authoring valid metadata. She uses the Hermes metadata editor to create the metadata for Dr. X's content through a friendly user interface.
A few months later, Dr. X's content has been well received. Through the word of mouth, physicians from other local institutions have requested access to his course. Dr. X and Ms. Y see potential to offer this course to the broader community. Dr. X would like to update the content from time to time as new advances are made in the subject area and Ms. Y sets him up with Hermes to edit the metadata. Ms. Y asks Mr. Z, an SMC network administrator, to set up a metadata and content repository so that SMC's courses can be searched by the broader community. She also asks Mr. Z to work with her to associate the content to a new competency-framework for gastroenterology that is being used by the residency program. Mr. Z is the web master for SMC web site. He downloads the competency map and competency definitions form a national gastroenterology association site. He finds that he simply has to place the metadata and competency files in a folder and enable directory browsing on the folder. He registers the URL of this repository with MERG search engine. MERG harvests and indexes the content and the competency associations. The content itself is placed in a password protected folder.
Dr. W, an internist from rural Pennsylvania, searches MERG for Hepatology content. He finds Dr. X's course and contacts Ms. Y at SMC. He registers for the course and Ms. Y authorizes Dr. W to view the content. Dr. W completes the course and subscribes to the RSS feed from the Hepatology search using his web browser. He will be notified as new Hepatology CME content is registered into MERG.
Ms. Y mentions MERG to her EMR provider, Z-Med. Z-Med looks into MERG and finds that it provides a web services interface to its search. Z-Med has been looking to integrate CME opportunities into its electronic health record system. Z-Med comes up with a plan to integrate links to CME content in its physician portal.
The course, Introduction to Biomedical Informatics, was developed by one of the authors (WRH) and is taught in the biomedical informatics graduate program at Oregon Health & Science University (OHSU) and the AMIA-OHSU 10 × 10 Program .
Content from Contraception Online  was made available by another author (MF) and offers a variety of learning resources for healthcare providers and patients on a range of topics related to contraception. Resources included didactic slide presentations, interactive cases, monographs, and patient education handouts.
MERG Suite underwent internal iterative developmental evaluation towards the design of Hermes and the server side tools. The feasibility of the prototypes were tested using the above two real world metadata collections. Since the web search component of MERG was expected to service most users, we conducted a survey of the potential users of the systems. After receiving approval from the OHSU Institutional Review Board, we administered the survey using Survey Monkey . Participating medical educators were recruited using several email lists (e.g., the MED-ED email list of the Association of American Medical Colleges; the DR-ED email list of Michigan State University; and the MedBiquitous Competencies, Learning Objects, and Virtual Patient Working Groups). The survey consisted of two parts. The first part focused on the use of competencies in medical education. The second part of the survey focused on experience with MERG suite and was administered after participants had an opportunity to apply the tool to one of the two content areas – medical informatics or contraceptive topics in women's health.
The survey instrument included questions about participants' educational roles and responsibilities, their use and the availability of competencies for the discipline(s) in which they teach, and their views on the value of a tool that would enable searching for e-learning content by using competency search terms. Each participant was asked to use our prototype system with its candidate competencies and content and to provide feedback to guide further development.
Professional role(s) of respondents.
Teacher or course faculty member
Educational target audiences of respondents. (Total may exceed 100%.)
Physicians in practice
Nurses, Physician assistants, and other allied health professionals
Frequency of respondents' development of lectures, instructional materials, syllabi, or other scholarly educational products for use in teaching.
Less than once a year
1–2 times a year
3–6 times a year
Most recent use of competencies to develop educational materials.
1 to 2 years ago
6 months to 1 year ago
3 to 6 months ago
Within the last 3 months
Likert scale ratings of opinions concerning competencies.
Neutral (do not know)
I am comfortable in using the available competencies to inform instructional interventions.
Significant barriers exist to using competencies to develop education or educational products.
I have an interest in learning more about the competencies available for my specialty or discipline.
It would be useful to link learning resources to the competencies that those resources address.
I would use a search tool that linked learning resources to competencies and linked competencies to relevant learning resources.
Having access to a tool such as this may change how I go about designing instruction.
Available competency sets do not align with curriculum goals.
There are inadequate numbers of competency sets.
The faculty does not yet understand the concept of competency-based education.
Available competency sets are not practical or are vague or unproven.
Educators have trouble choosing between competing competency sets.
There is difficulty in adopting competency sets within traditional curricula.
Creating competency-based assessment tools is difficult.
Competency sets add to information overload.
Usability of MERG.
How easy or difficult was the tool to use?
How easy or difficult is it to access learning resources through MERG?
Although our intent was to assess respondents use of our search system linking competencies to relevant learning objects using limited data sets for illustrative purposes, survey participants in their comments focused more on the narrowness of available learning objects rather than on any challenges encountered in the use of the search system. This may be that in our instructions we did not communicate the limited nature of learning objects available for testing and thus did not manage expectations and/or the search tool itself was intuitive for most users and the challenges encountered related more to their inability to identify a wide variety of resources. The feedback we gathered from the study guided us in updating the web user interface. Figure 5 shows the updated user interface for MERG. The current user interface is more fully featured and designed as a single page view with a tabbed interface. It is also more responsive and incorporates some cosmetic improvements.
Our study engaged a group of medical educators that understood and endorsed the competency-based approach to learning; although they did indicate that the competency sets specific to their teaching are currently limited. While they agreed in principle with the approach taken through development of the MERG search tool, the participants focused on the comparatively scant content generated from their searches. In retrospect, participants should have been alerted to the narrowness of the current content knowledge-base which was assembled only for usability evaluation purposes. More specific instruction should have been provided to participants explaining our interests in testing the functionality of the search tool and factors related to ease of use, rather than investigations related to the comprehensive nature of search returns.
Notwithstanding the issue of search return volume, the results from this assessment of MERG suite functionality indicate that overall the professionals who tested it found it easy to use and were in agreement with the usefulness of the approach. With the addition of a wider range of content and linked repositories, quite different testing results would likely be realized and greater insights into functionalities of interest might be revealed.
The future utility of MERG will depend on the availability of standardized, competencies that are linked to content.
Barriers to adoption
While the important role of competencies is now recognized in the educational community and standards for authoring competencies are available, only a small portion of content is linked to competencies. Tools such as the MERG suite may lower the barrier of entry in developing and distributing competency-linked content from a technical perspective by providing tools that shield the users from the details of the specification and by offering guidance for setting up the repositories. However, additional research is needed with respect to effective use of competencies. Experiences in domains such as U.S. Public Health Service, which appear to be at the forefront in designing and applying competencies for health professional education, may inform future development in this area.
MERG has been designed as a modular application with loose coupling between components. While this approach has distinct advantages, it does pose challenges in deployment. Although not a substantial service at present, our aim is that the MERG tool kit serve as an evolutionary platform for implementing such services in the future.
Availability and requirements
Project name: MERG Suite
Project home page: http://ir.ohsu.edu/MergSuite/
Operating system(s): Windows 98 or higher, Unix-based OS.
Programming language: Java, Python, Groovy
Other requirements: Java 1.6 or higher, Python 2.5 or higher
License: GNU GPL.
Any restrictions to use by non-academics: None
This work was funded in part by Grant 1G08 LM008235 of the National Library of Medicine
- Batalden P, et al: General competencies and accreditation in graduate medical education. Health Affairs. 2002, 21: 103-10.1377/hlthaff.21.5.103.View ArticlePubMedGoogle Scholar
- Harden RM: Learning outcomes and instructional objectives: is there a difference?. Medical Teacher. 2002, 24: 151-155. 10.1080/0142159022020687.View ArticlePubMedGoogle Scholar
- Brailer DJ: Interoperability: the key to the future health care system. Health Affairs. 2005, 24: 19-21.Google Scholar
- Fallon C, Brown S: E-Learning Standards: A Guide to Purchasing, Developing, and Deploying Standards-Conformant E-Learning. 2003Google Scholar
- MedBiquitous Consortium. [http://www.medbiq.org]
- SCORM®. – Sharable Content Object Reference Model. [http://www.adlnet.gov/scorm/index.aspx]
- Learning Objects Working Group. [http://www.medbiq.org/working_groups/learning_objects/index.html]
- IEEE Reusable Competency Definition or Educational Objective – Information Model. [http://www.ieeeltsc.org/working-groups/wg20Comp/wg20rcdfolder/]
- Hersh WR, et al: Adopting e-Learning Standards in Health Care: Competency-based Learning in the Medical Informatics Domain. AMIA Annu Symp Proc. 2006, 334: 8.Google Scholar
- eXist XML database. [http://exist.sourceforge.net/]
- XQuery. [http://www.w3.org/TR/xquery/]
- Apache Lucene. [http://lucene.apache.org/]
- eXist eXtension to full text search. [http://exist.sourceforge.net/soc-ft.html]
- Karrigell Web Server. [http://karrigell.sourceforge.net/]
- GlassFish Open Source Application Server. [https://glassfish.dev.java.net/]
- GNU Wget. [http://www.gnu.org/software/wget/]
- Direct Web Remoting Library. [http://getahead.org/dwr]
- ROME Project. [https://rome.dev.java.net/]
- dom4j. [http://www.dom4j.org/]
- Apache Velocity. [http://velocity.apache.org/]
- W3C SOAP Specification. [http://www.w3.org/TR/soap/]
- Hersh W, Williamson J: Educating 10,000 informaticians by 2010: the AMIA 10 × 10 program. International journal of medical informatics. 76: 377-82. 10.1016/j.ijmedinf.2007.01.003.Google Scholar
- Contraception Online. [http://www.contraceptiononline.org/]
- Survey Monkey. [http://www.surveymonkey.com/]
- ADL SCORM. [http://www.adlnet.gov/scorm/index.aspx]
- MedBiquitous Healthcare Learning Object Metadata. [http://www.medbiq.org/working_groups/learning_objects/index.html]
- IEEE Reusable Competency Definition. [http://www.ieeeltsc.org/working-groups/wg20Comp/wg20rcdfolder/]
- Proposed Draft Standard for Learning Technology – Simple Reusable Competency Map. [http://www.ostyn.com/standardswork/competency/ReusableCompMapProp20051117.pdf]
- MedBiquitous Competencies Working Group. [http://www.medbiq.org/working_groups/competencies/index.html]
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.