› Home › Teaching Support Programs, Services & Committees › LMS Review › LMS Review Process
LMS Review Process

Given the goal of DePaul’s Vision Twenty-12 Plan for Academic Enrichment to “become a model provider for quality distance learning,” DePaul’s Teaching, Learning and Technology Committee began a process of reviewing Learning Management Systems options in the fall of 2007. These critical systems, which support classroom teaching with 24/7 access to key resources and underlie online instruction, had not been systematically reviewed for many years despite many changes in technology. Many other universities and colleges have changed systems as usage has increased and more user-friendly tools have been developed.

Review process documents:

Key steps in this process were as follows:

  • In the 2007-8 academic year, the Learning Management Systems subcommittee was formed and generated a list of LMS features that would be desirable. These included interconnection with ePortfolio systems, and Content Management Systems.
  • In fall of 2008, the LMS subcommittee identified five leading LMS vendors: Blackboard, Angel, Desire2Learn, eCollege and Moodle (supported by a hosting service provider, RemoteLearners.net). These were identified based on stability, quality of products and services, and
  • TLT requested that the review include ePortfolio and Content Management Systems, due to the advantages of interoperability and the interest in such systems indicated by faculty. The committee developed a process for review of these systems, to be carried out by the subcommittee under the review and auspices of TLT. A set of questions reflecting critical aspects of an LMS, ePortfolio and Content Management systems were developed after repeated review and revision by the subcommittee and all of TLT. These were sent to the five vendors in a request for a proposal.
  • Four of the five vendors responded to the request. One, eCollege, declined to participate. All other vendors were invited to participate in the rest of the review process.
  • The committee developed three sets of assessment rubrics.
    • Two were task-based: one short set for casual faculty users of the LMS, and the other a longer set of tasks for faculty and staff who are intensive users of the systems.
    • The third was a set of criteria for assessing all three systems (LMS, ePortfolio and Content Management systems). The criteria were based on the questions previously developed and which formed the basis of the Request for Proposals.
    • All three sets of assessment rubrics were reviewed by TLT members and shared with the vendors. All asked respondents to prioritize the different aspects.
  • All DePaul faculty were invited to participate in a testing session, in conjunction with the DePaul Teaching and Learning Conference on April 17, 2009. A computer room was set up, with access to all four LMS systems and sample course items to upload. Several of the vendors were present, as well as staff to assist faculty. Many faculty participated in the test. The short task-based rubric was used.
  • All DePaul Faculty were invited to test the LMSes via email for an extended period (through May 10). Access directions and the short task-based rubric were sent to those who responded.
  • All DePaul faculty were invited to attend presentations by the vendors, held on April 23 and 24. All participants were asked to use the standardized criteria to evaluate the tools, including LMSes, ePortfolios and content management systems.
  • Members of TLT and other interested faculty and staff were invited to conduct extended testing of the LMS systems, using the in-depth task-based rubric.

Ongoing research steps:

  • A set of summer courses were selected to be offered in Blackboard 9.0 and Desire2Learn. Faculty and student feedback will be sought about the learning experience.
  • Information Systems teams are conducting an Architecture Review, to ensure that any system chosen by TLT can integrate with DePaul’s PeopleSoft and other information systems.
  • All sources of data will be analyzed and in the summer, and a report created by the LMS Subcommittee. Data sources will include:
    • Vendor responses to the RFPs
    • Faculty testing results (using the short task-based rubric)
    • Extended testing results (using the in-depth task-based rubric)
    • Presentation evaluation results (using the standardized evaluation form)
    • Summer testing results
    • IS Architecture review

The TLT committee will review all sources of data and the analysis and report of the Subcommittee, and make a recommendation in September.

DePaul Teaching & Learning Events DePaul Teaching & Learning Resources DePaul Teaching and Learning Conference