A deep dive into multimodal assessments
What is multimodality?
Chalks incessantly scribbling words and numbers on blackboards (and later pens on whiteboards), and students poring over textbooks — that was the traditional approach to teaching and learning. How boring!
Cut to today, knowledge dissemination has transcended old-school methods. Smartboards have found their way into classrooms and training centres. Videos, projects, field trips, and roleplays are taking over the curriculum.
Today’s teaching model believes in engaging multiple senses. The VARK framework is widely used here. It emphasises the use of visual, auditory, text (reading & writing), and kinaesthetic elements for instructing.
Welcome to multimodality, a happy confluence of written, oral, visual, audio, gestural, spatial and tactile expressions! It has definitely evolved learning into something more interactive and enjoyable.
A goodbye to the yawns!
As teaching and learning evolves, assessment methods must also follow suit. They too must go beyond the read-write mode of evaluation.
Multimodal assessments and their significance
Pre-employment assessments are crucial to the hiring process. They evaluate and validate the knowledge and skills candidates claim in their resumes.
Traditionally, the evaluation happened with only written tests and interviews. Though they are still needed, they don’t suffice in gauging the full spectrum of skills required for a role. This is where multimodal assessments offer a valuable advantage.
Multimodal assessments involve using multiple assessment formats. They include written tests, group discussions, presentations, simulations, video interviews, hands-on project exposure, etc. Each format individually helps gauge a specific skill. Together, they form a formidable force that strengthens the assessment process.
Here are compelling reasons to adopt them:
- Attracting Gen-Z: Multimodal assessments appeal to the digital-savvy Gen-Z. Many assessment methods such as live-coding and recorded video interviews are digital, which is right up Gen-Z’s sleeve.
- Holistic evaluation: Each assessment mode provides a distinct insight into a candidate’s abilities. For instance, written examinations evaluate knowledge retention, while group activities or team-based simulations provide insights into a candidate’s collaboration and teamwork. Case studies gauge analytical and decision-making abilities, and oral interviews test spontaneity.
Combining these assessment modes offers a comprehensive, deeper, and well-rounded view of a candidate’s competencies, enabling recruiters to assess a candidate’s potential from multiple angles.
- Validating skills and knowledge: Multimodal assessments enable candidates to demonstrate their skills and knowledge through practical tasks, presentations, or simulations. This ensures that candidates possess the abilities they claim on their resumes, reducing the risk of hiring someone based solely on their self-reported experience.
- Identifying job fit: Different jobs require different skill sets and personalities. Multimodal assessments help match candidates to roles that align with their strengths and preferences, increasing the likelihood of successful job performance and job satisfaction.
- Role-specific tailoring: Different roles require different skill sets and qualifications. With flexibility in assessment methods, recruiters can tailor the evaluation process to suit each role’s specific requirements. For example, technical positions might include coding challenges, while leadership roles could involve situational exercises.
With significant benefits to be gained from multimodal assessments, recruiters must design and implement an effective assessment strategy that incorporates multimodality.
How to set a multimodal assessment?
Setting up a multimodal assessment requires careful planning and consideration. Here’s a step-by-step guide to creating an effective and comprehensive multimodal assessment, with an example of evaluating a marketing candidate.
- Determine objectives: Clearly define the objectives and competencies you want to assess.
Example objective: Evaluate marketing knowledge, creativity, and ability to develop a marketing campaign for a product. - Identify modes: Select the assessment modes that align and provide a holistic evaluation of the objectives.
For the marketing campaign, the modes could be:
- Create an advertisement for the product
- Present a persuasive pitch for the marketing campaign
- Write a detailed marketing plan
- Plan and execute a small-scale promotional event
- Develop tasks: Create tasks or prompts for each mode that aligns with the learning objectives. Ensure that tasks are clear, relevant, and challenging enough to showcase candidates’ skills.
Here is an example:
- Task 1: Design an eye-catching advertisement that effectively communicates the product’s unique selling points.
- Task 2: Present a compelling pitch to a panel of interviewers, explaining the marketing campaign’s strategies, target audience and expected outcomes.
- Task 3: Develop a comprehensive marketing plan detailing the campaign’s objectives, timeline, budget and distribution channels.
- Task 4: Plan and execute a small promotional event to engage potential customers and promote the product
- Establish criteria: Develop scoring guidelines and rubrics that outline the expectations and standards for each mode of assessment. Clearly define what constitutes exemplary, proficient, and developing performance.
Here are some rubric examples for the tasks set in step 3 above:
- Task 1: Creativity, visual appeal, alignment with brand identity, and effectiveness in conveying key features.
- Task 2: Clarity and confidence in presenting the marketing campaign, persuasiveness in capturing the panel’s attention, and ability to handle questions effectively.
- Task 3: Thoroughness of the marketing plan, strategic approach, attention to detail, and alignment with the product’s objectives.
- Task 4: Planning and execution of the promotional event, engagement with the audience, and effectiveness in promoting the product.
- Plan logistics and resources: Determine the technology and resources needed for each mode of assessment. Ensure that necessary access to the required tools and platforms is available.
- For example, for the marketing evaluation, reserve a presentation room for the pitch presentation, and ensure access to the graphic design software, marketing resources
(for copywriting, research, digital and social media marketing, etc.) event planning materials, and so on.Sequence and timeline: Create a well-structured sequence of assessment tasks and establish a realistic timeline. Allow sufficient time for preparation and submission.
Here is an example of timeline-setting:
- Week 1: Introduction to the marketing campaign project and assessment expectations.
- Week 2: Candidates work on their visual advertisements and marketing plans.
- Week 3: Candidates present their marketing pitch to the interview panel.
- Week 4: Execution of a promotional event.
- Communicate clearly: Clearly communicate the purpose, expectations, and timelines. Provide detailed instructions for each task, including submission requirements and guidelines for using technology platforms.
- Assess and evaluate: Evaluate candidates’ performance based on the established rubrics. Take note of strengths, weaknesses, and any additional insights.
- Feedback and follow-up: Offer timely and constructive feedback, either through individual feedback sessions, written feedback, or a combination of both. Proceed with subsequent stages, as applicable.
- Review and refine: After completion, review the effectiveness of the multimodal assessment. Gather feedback from assessors and candidates to identify areas for improvement and make necessary refinements for future assessments.
These steps can help design robust, inclusive and effective multimodal assessments. However, incorporating them in the systems is not a bed of roses.
Challenges and Considerations
Implementing multimodal assessments comes with its share of challenges and practical considerations. Here are some key ones:
- Alignment: Ensuring that the chosen assessment modes align with the required competencies is crucial for effective evaluation. The assessment should accurately reflect the skills and knowledge that candidates are expected to demonstrate.
- Assessment design and sequencing: Designing well-structured and sequenced assessments is essential for a cohesive and meaningful evaluation process. The tasks should flow logically and build upon each other, providing a comprehensive view of candidates’ abilities.
- Resource availability: Ensuring the availability and accessibility of necessary resources (both technical and non-technical) for each assessment is of the utmost importance. This ensures that candidates have everything they require from your end to complete their tasks seamlessly.
Here are the technical and infrastructure requirements you should think of:
- Hardware and software: This includes devices and software platforms for recording, editing, and presentation.
- Data security and privacy: Security measures to protect the confidentiality and privacy of candidates’ assessment data must be in place.
- Backup and recovery: Reliable backup and recovery processes to safeguard assessment materials and data in case of potential technical issues must be established.
Also spare a thought for non-technical resources such as these:
- Stationery: Pens, pencils, markers and paper
- Whiteboards: For brainstorming sessions, group discussions and presentations
- Meeting rooms: For discussions and presentations
- Time management: Candidates need time to prepare and complete the tasks. Striking a balance between the assessment’s complexity and the available time is vital.
- Training and support: Providing adequate training and support on using assessment tools, platforms, and technology is necessary to enhance a smooth assessment process.
- Accessibility and accommodation: Assessments must be made accessible to all candidates, including those with disabilities. Accommodation should be made to enable equal participation and fair evaluation of everyone.
- Coordination: Assessment schedules, submissions, and feedback must be carefully planned and organised.
- Consistency and standardisation: Standardising rubrics and assessment guidelines is essential to ensuring that evaluation metrics yield fair and unbiased results.
Addressing these challenges and considerations is imperative for a smooth assessment process.
Conclusion:
Multimodal assessments have emerged as a dynamic and inclusive approach to evaluating knowledge and skills.
They have proven their effectiveness in providing a comprehensive evaluation of candidates’ capabilities, offering a bias-free hiring process, matching the specific requirements of each position, helping better predict performance in an actual job environment, and more.
By setting clear objectives, designing well-sequenced assessments, providing the necessary resources, and ensuring accessibility, a fair and equitable assessment environment can be created.
Multimodal assessments are refreshing, resilient and insightful, leading to enhanced candidate selection, increased retention rates, and improved overall organisational performance.
Reference: