Organising for eAssessment
Organising e-assessment demands for an articulation between design, implementation, tools and people involved. Setting up the platforms and the tools for e-assessment is a task that should be performed by a team and managed by an administrator. The team must follow a number of e-assessment and organisation guidelines, when using the platforms and choosing the most appropriate e-assessment tools to a specific course. The administrator must guarantee that learners understand the e-assessment process, the ways of communication and feedback, and the type of assessment and certification.
Key areas of responsibility
Prior to the platform course delivery the assessment should be defined and set-up by the trainer (diagnostic, formative, summative and for learning assessment). The trainer should use diverse techniques, in accordance with the platforms and the course learning outcomes and subjects (multiple choice, closed response questionnaires, open exercises, simulations, specific tasks, collaborative exercises, research activities, etc.).
Remember the following principles:
The assessment system should be clear to learners, containing guidelines regarding the objectives and dimension (level of depth), a timeframe with deadlines, criteria, and weighting of the various assessment instruments.
The assessment should promote autonomy and interaction. Individualised or flexible pathways should be provided, allowing learners autonomy, creativity, learning through experience and action. Individualised learning management should be promoted through the creation of portfolios of acquired skills and knowledge.
Learning guidance is provided by introducing analytical feedback and mechanisms for self-training and autonomy according to the specific needs of the learners.
Feedback is given to the learners on the assessment results to systematically support the self-directed learning.
Individualised learning management is promoted through the creation of portfolios of acquired skills and knowledge. acquired knowledge.
Who is involved: Actors
Manager/Administrator, Trainer, Tutor, Learner, Automatic tutoring assistant, helpdesk, etc.
What do they do
Administrators/managers – guarantee that the platforms have all set-up instruments according to the course design, course assessment strategies, instruments and level of certification.
Trainers and Teachers – design of the e-assessment strategy, instruments and timings. Create the e-assessment set of instruments including the database of questions for exams, self-assessments, closed response questionnaires, open exercises, simulations, specific tasks, collaborative exercises, research activities, essays, etc.
IT and Managers – support trainers and administrators for the e-assessment implementation. Train the trainers on the possible instruments to be used, find e-assessment patterns by course topic/subject area, support technically all actors, support co-creation and evaluation of instruments, etc.
Learners – co-create e-assessment instruments, participate in the e-assessment instruments improvement
Coordinating the e-assessment process
1-Develop the training organisation e-assessment guidelines. Include recommendations and examples on strategies and tools. Prepare platform tutorials for teachers in what concerns e-assessment tools. List the type of e-assessment set of instruments including the database of questions for exams, self-assessments, closed response questionnaires, open exercises, simulations, specific tasks, collaborative exercises, research activities, essays, etc.
2- Training the trainers in e-assessment, evaluate the guidelines and improve them according to trainers experience, give examples
3- Support trainers in the development of the e-assessment instruments when they need
4 – Support learners on e-assessment understanding if needed
5 – Control the process of trainers marking following the course administrative deadlines
6- Guarantee storage of versions and backup of assessment results
Diagram based on the original assessment lifecycle by Manchester Metropolitan University in Jisc https://www.jisc.ac.uk/guides/enhancing-assessment-and-feedback-with-technology
The technical support team should include IT and education staff. The team should guarantee that trainers and learners are able to use the platform for e-assessment, according to the guidelines. Technical support can be provided via the e-assessment tutorials, or via phone contact with the help-desk or technician. If needed pedagogical support should also be provided.
The process of marking students using a LMS platform can be done automatically by the platform or can be done by an educator.
Auto-marked assessments – marked automatically by the platform. Question types that can be auto-marked are multiple-choice questions, multiple response, true/false etc. The assessment can be designed to give the student their mark as soon as they finish their test or it can be held back for checking and comments by the teacher/trainer.
Human-marked assessments – are assessments which require marking by the trainer. They are usually essay questions or long calculations. The trainer is able to add comments, feedback and marks to each question in the platform.
The process of e-assessment involves not only an active communication channel between the educator and the learner(s), but also personal data transfer and storage: learner identification, learning analytics of actions in the assessment platform/software, evaluation marks, etc. According to Kiennert et al (2017), “(…) security measures are required to protect the environment against system and network attacks. The issue concerning security is challenging from both educational and technical point of views”. Another challenge with online assessment is making sure that the authentication of the data source is accurate, in order to secure the identity of the person taking the online assessment.
In other words, the security challenges posed to online assessment are from two points of view: a) the security of shared personal data against external intrusions or unauthorised parties, also taking into consideration legal (like the General Data Protection Regulation), social and ethical issues; b), and the security of the assessment process itself, ensuring that the person being assessed is, in fact, the learner, and that he/she undergoes the assessment process without external aids, as they would in a face-to-face/paper-based assessment, preventing cheating.
Kiennert, Christophe et al. (2017) “Security Challenges in e-Assessment and Technical Solutions”. Conference: 2017 21st International Conference Information Visualisation (IV). DOI: 10.1109/iV.2017.70 https://www.researchgate.net/publication/321120609_Security_Challenges_in_e-Assessment_and_Technical_Solutions