BM 10 Each course includes a clear statement of learning outcomes in respect of both knowledge and skills. There is reasoned coherence between learning outcomes, the strategy for use of e-learning, the scope of the learning materials and the assessment methods used.

Learning outcomes express what students should have achieved when they have completed the course. These outcomes will be specified in terms of knowledge, skills, vocational/professional competencies, personal development, etc. and will usually be a combination of these.
Each course should have a specification which sets out the relationship between learning outcomes and their assessment. The course design team should ensure that the assessment strategy reinforces the teaching strategy.
Decisions about the use of e-learning and e-assessment should be made on the basis of providing the most effective means of achieving and assessing the learning outcomes. There should be a clear rationale for the use of e-learning and the level of support provided.

Course description

Each course includes a clear statement of learning outcomes in competences (terms of knowledge, skills and attitudes). There is a reasoned coherence between learning outcomes, the strategy for use of Onsite Training and Distance Learning, the scope of the learning materials and the assessment methods used [BM 10].

Learning outcomes define the competences that trainees should have achieved when they have completed the Blended Learning course. These outcomes will be specified as competences (in terms of knowledge, skills, vocational/professional competencies, personal development, etc.).

Each course should have a specification which sets out the relationship between learning outcomes and their assessment. The course design team should ensure that the assessment strategy reinforces the teaching strategy.

Decisions about e-assessment should be done on the basis of providing the most effective means for achieving and assessing the learning outcomes. Collaborative assessments can be used as well.

The course description must be communicated with all stakeholders (mainly the trainees) in time (best in advance of the course start.

11. Learning outcomes determine the means used to deliver course content. In a blended-learning context there an explicit rationale for the use of each component in the blend.

Course designers should adopt a logical approach to their use of e-learning. Fitness for purpose should drive decisions on the selection of educational components. The blending should be such that methods and media are well chosen for the purposes they need to fulfil.
Course designers must resolve the tension between:

• the flexibility of access to resources offered by the anywhere, anytime availability of e-learning
• the flexibility of educational interaction offered by face-to-face contact between students and teachers.

If it is not possible to deliver learning outcomes solely through e-learning then some face-to-face techniques may be essential.

Learning Outcomes

This benchmark will be enhanced/altered by:

Blended Learning courses should always care for a well-fitting balance between Distance Learning (mostly realized by e-learning) and face-to-face techniques (in the onsite teaching part).

12. Course design, development and evaluation involve individuals or teams with expertise in both academic and technical aspects.

The course design process should be carried out by staff who are equipped to address both academic and technical aspects of e-learning. Those involved in the course design process require experience of e-learning and its capabilities. In a rapidly evolving field much of this experience is gained through project participation rather than formal qualifications.
The development of high quality e-learning content is dependent on close collaboration and good communication between academic course designers and those responsible for the production of the constituent media components. Engagement between academics and media professionals in the technical design contributes significantly to the effectiveness of course materials.
A key issue that affects working relationships is whether technical design inputs should be integrated with the academic and pedagogic design process or applied to the outputs from this process. Institutions should consider the most appropriate methodologies to use in their specific context.

Technical and Training Expertise

Course design, development and evaluation involve individuals or teams with expertise in training as well as technical aspects (BM 12).

The course design process should be carried out by staff or trainers who are equipped to address training and technical aspects of Blended Learning (which always covers Onsite Training and Distance Learning).

This covers

• The experience of e-learning and its capabilities
• In a rapidly evolving field, most of this experience is gained through project participation rather than formal qualifications.

The development of high-quality content (for both Onsite Training and Distance Learning) is dependent on close collaboration and good communication between course designers (with deep knowledge of the related profession or training subject) and those responsible for the production of the constituent media components (a mainly technical task). Engagement between both groups in the technical design contributes to the effectiveness of course materials significantly.
In SMEs a personal union of technical and training staff may occur.

A key issue that affects working relationships is whether technical design inputs should be integrated with the subject-related and training design process or applied to the outputs from this process. Institutions should consider the most appropriate methodologies to use in their specific context.

13. OER and other third-party material is selected with regard to learning outcome, tailored if necessary for fit to the learning context, and integrated with other learning materials. These materials are subject to the same review processes as other course materials.

Open Educational Resources (OER) are digital materials offered freely and openly for use and reuse by educators and students. OER can be found through the large institutional and collaborative repositories that now exist.
A course designer could develop a course by picking existing OER components, or other third-party resources (and perhaps customising them), rather than developing new material from scratch.
The resulting e-learning material should be judged under the same quality criteria as newly-authored material or bought-in material.
An important benefit of OER is that the licence to freely change material makes it possible to update and improve it, allowing high quality e-learning components to evolve as users improve content and offer it back to the OER community.
Rights must be carefully tracked to ensure that the appropriate level of access is preserved and that authors are credited where appropriate. The intellectual property rights associated with OER (often one of the Creative Commons licences) usually allow material to be used without cost for non-commercial purposes and allow material to be freely reversioned and updated.

Use of third-party material

OER and other third-party material are selected with regard to learning outcome. They are tailored if necessary, in order to fit to the learning context, and integrated with other learning materials. These materials are subject to the same review processes as other course materials (BM13).

Open Educational Resources (OER) are digital materials offered freely and openly for use and reuse by educators and students. OER can be found through the large institutional and collaborative repositories that now exist. An important benefit of OER is that the license to freely change material makes it possible to update and improve it, allowing high quality e-learning components to evolve as users improve content and offer it back to the OER community.

A course designer could develop a course by picking existing OER components, or other third-party resources (and perhaps customizing them), rather than developing new material from scratch.

Use of OER must take into account several criteria

• The resulting e-learning material should be judged under the same quality criteria as newly-authored material or bought-in material (or must be upgraded to the expected level).
• Material in foreign languages must be translated (with some minor exceptions)
• Rights must be tracked carefully to ensure that the appropriate level of access is preserved and that authors are credited when it seems appropriate.

The intellectual property rights associated with OER (often one of the Creative Commons licenses) usually allow materials to be used without cost for non-commercial purposes and allow material to be freely revised and updated.

14. E-learning materials have sufficient interactivity (student-to-content or student-to-student) to encourage active engagement and enable students to test their knowledge, understanding and skills.

E-learning opens up many possibilities for interactivity in learning materials and activities. Interactivity can relate to an individual student using online resources such as simulations, animations and quizzes. It can also relate to students interacting with each other via online communication tools such as forums, wikis or web conferencing.
Individual students will benefit from the active approach to learning and the feedback provided by simulations and interactive formative assessments. These techniques can encourage deeper learning.
Online communications and collaboration tools have great potential in providing support and academic community building for distributed student populations. Their use can be a make or break factor for some students, and can make the learning experience far more engaging and lively for everyone.

Interactivity

E-learning materials and Onsite Training have sufficient interactivity (trainee-to-content, peer activities trainee-to-trainee, and/or trainee-to-trainer) to encourage active engagement to create a sustained learning. This also enables trainees to test their knowledge, understanding and skills (competences) (BM 14)

Blended Learning opens up many possibilities for interactivity in learning materials and activities. This must be enabled in Distance Training as well in an active Onsite Training.

• Interactivity can relate to an individual trainee using online resources such as simulations, animations and quizzes.
• It can also relate to learners interacting with each other (peer activities) via online communication tools such as forums, wikis or web conferencing.

Individual trainees will benefit from the active approach to learning, the option of individualized training and the feedback provided by simulations or interactive formative assessments. These techniques can encourage deeper learning and are able to lead to sustained learning results.

15. Independent learning materials provide learners with regular feedback through self-assessment activities or tests.

Independent learning materials may be designed to serve the needs of several courses or programmes; such packages should therefore be self-contained, have clear learning objectives and measurable outcomes.
When delivered by e-learning the materials should be designed to maximise the use of interactive techniques to provide opportunity for student self-assessment of progress towards learning outcomes.
E-learning offers opportunities for embedded interactive formative assessment with automated feedback. Development of these assessments requires significant academic input and collaboration with experts in the facilities available through the institution’s VLE systems. The benefits to students through rapid feedback are considerable.

Learning Material

Independent learning materials provide learners with regular feedback through self-assessment activities or tests (BM 15).

Independent learning materials may be designed to serve the needs of several courses or programmes; therefore such packages should be self-contained, have clear learning objectives and measurable outcomes (Micro Learning).

When delivered by e-learning (in both settings like Onsite Training and Distance Training) the materials should be designed to maximize the use of interactive techniques to provide opportunities for trainees self-assessment of progress towards learning outcomes.

E-learning offers opportunities for embedded interactive formative assessment with automated feedback. Development of these assessments requires significant training input and collaboration with experts in the facilities available through the institution’s VLE systems. The benefits to trainees through rapid feedback are considerable.

Multimedia-bases and interactive training material is offering some motivating feedback to the trainees.

16. Courses conform to explicit guidelines concerning layout and presentation and are as consistent as possible across a programme.

This benchmark tests aspects of good practice in materials design. A poor score in this area may indicate that the institution should make a greater investment in the contribution of media professionals to course design teams.
It is recommended that institutions implement a framework of technical, accessibility and presentational standards that apply to e-learning materials. This framework should include the following factors:

• Interfaces should conform to usability and accessibility standards.
• materials should be accessible to users with special requirements, for example students with a visual impairment or limited manual dexterity.
• Learning materials should have good graphic design standards.
• Materials should be neutral as to sex, ethnicity, age and so on.
• Software should be up-to-date and platform neutral.
• course developers should take into account download times.
• Learning materials should be accessible and usable via a variety of devices including mobile devices.
• Style sheets and schemas should be used to provide consistency of presentation.
• Course developers should be provided with suitable authoring tools.
• Learning material should use responsible design (to fit best to multiple devices).
  (This item is an enhancement in the frame of the TIBL project.)

Conformity

This benchmark can be used without major changes  but will be enhanced by one item in the list

17. Courses provide both formative and summative assessment. Assessment is explicit, fair, valid and reliable. Appropriate measures are in place to prevent impersonation and/or plagiarism, especially where assessments are conducted online.

Course design should provide opportunities for formative and summative assessment.
Continuous assessment may fulfill both formative and summative functions. Feedback from tutors/mentors on assessment tasks provides the learner with reinforcement or direction for remedial learning. Teachers should be required to provide timely feedback aimed at improvement. Marking criteria need to be uniformly understood and consistently applied.
Peer and self-review can also be used for formative assessment. Clear marking criteria are needed for this to be a valuable exercise.
The development team should aim to exploit the interactivity inherent in e-learning, particularly through formative assessment activity, to confirm and reinforce student learning. Formative assessment may be built into the design of structured teaching materials.
The use of e-learning raises issues of verification of student identity, and measures should be taken to prevent impersonation and plagiarism. These measures may include: checking identities at approved examination centres; using software to detect plagiarism and collusion; cross-referencing and correlation between performance on written examinations and on continuous assessment.

Assessment

This benchmark fits without changes to VET.

18. Course materials, including the intended learning outcomes, are regularly reviewed, up-dated and improved using feedback from stakeholders as appropriate.

The institution should have procedures to ensure effective review and updating processes. Independent evaluation of course design and course materials should be carried out to ensure comparability with national or professional standards
In the case of e-learning courses, the evaluation process should address subject content, modes of delivery and levels of interactivity. For example: 

• external assessors should be engaged to review course design and provide developmental feedback
• the monitoring and evaluation process should provide documented feedback for improvement and redevelopment
• whilst in presentation, data on patterns of student use may be gathered and analysed
• evaluation information should be sought via student surveys.

In an e-learning situation there is the potential for generation of very extensive data on student performance, etc. for quality improvement. The course design team should devise a strategy for exploiting this.

Actuality

This benchmark can be used without major changes in the TIBL context