Sunday, December 14, 2014

How to outguess Multiple Choice Tests

In his book ‘How to predict the unpredictable’ William Poundstone exposes the weakness of human bias and competence in setting MC tests. Over six years ago I wrote this piece All of the Above - how to cheatMultiple Choice questions, so it was heartening (or disheartening) to see some real evidence that backed up those claims. Poundstone chewed up stats on 100 tests from schools, colleges and other professional sources. He found the following weaknesses in serious professional certification exams, SATs and in many educational assessments that matter in terms of selection.
True-False test strategy
1. Go through the whole test getting those you know right first.
2. If the known answers before and after the one you don’t know are the same, choose the opposite, as there’s more alternation of TRUE and FALSE than in a truly random sequence, so to choose a different answer from the last one increases your chance of getting it right from 50% to 63%.
3. If the known answers before and after are different, choose TRUE as there’s more right TRUE (56%) answers than FALSE (44%).
Multiple Choice test strategy
1. Pick ‘none of the above’ or ‘all of the above’ as this gives you up to a 90% increase on random guessing.
2. Pick B on a four choice test, as this gives you an 11% increase, as human testers can’t randomise.
3. Avoid the previous choice gives you an 8% increase, again as human testers can’t randomise.
4. Choose the longest item. Even on the supposedly unbeatable SAT exams.
5. Eliminate outliers.
6. Always guess, as instinct can sometimes kick in to produce the right answer.
Conclusion

We live in an age of constant assessment, that is a shame. However, to live in an age of inefficient assessment is even worse, as smart, coached and tutored students can ‘game’ the system. For assessment designers it may be worth looking at these and other ‘game’ strategies to eliminate the weaknesses of your tests. First randomise the right answers. Humans just cannot randomise. Then look for the common biases. One final point. Randomisation is an algorithm. This is a case where algorithmic power of computers can eliminate human cognitive bias and weak competence. 

 Subscribe to RSS

Tuesday, December 09, 2014

VR is a medium not a gadget: 7 learning principles that work in VR

VR is a medium not a gadget but how appropriate is it for education and training? I’ve spent a lifetime using technology in learning but am no technological determinist. I don’t for example, like the use of tablets in secondary schools or whuiteboards. However, the first time I tried an Oculus Rift, it blew my mind, not just with its total immersion but its possibilities in learning. Before we get carried away with the sheer joy of the toy, what does the psychology of learning tell us about VR?
First, we have an avalanche of research and evidence from flight and military sims that show how powerful simulations can be. You’d be surprised, indeed you wouldn’t step on a plane, if your pilot hadn’t gone through many hours of flight sims. The learning effect with VR promises to be even better.
1. Attention - total focus
Selective attention is a necessary condition for learning, especially with novices, due to the imitations of working memory, and the one thing VR provides is an environment that commands the learners’ undivided attention. Total immersion means that it is difficult NOT to be attentive, as your entire sensory experience is taken over by the world you enter. Artificial cueing can also be introduced with exaggerated features and attention-grabbing cues. More than this, selective attention can also be heightened, even tested. For example. in a driving VR sim, you can hear a mobile phone ring, answer (txt or hands free) and test the reaction time to things that happen on the road. VR grabs your attention, holds it and can guide you towards selective attention for optimal learning.
2. Emotion – intense
I have never seen people react to any learning experience with the same intensity as I’ve seen in VR. We know that we learn more when we have an emotional connect with what we learn, along with other conative factors such as interest, competitiveness, positive feelings etc). This is important in all forms of learning but particularly when the affective components drive attitudinal behavior. Compliance training, sexual harassment, diversity and so on,  are poorly taught in classrooms, page-turning e-learning or with expensive role modeling. VR gives you the opportunity to have real affective impact as it induces conative factors.
3. Learn by doing
So often absent in institutional learning yet we know from Dewey, Kolb and Schank that we learn a lot by ‘doing’. This is the Achille’s heel of school, college, University and organisational learning, yet now we have the ability to place learners in worlds where that appear and feel real. Within these worlds we can get them to do all sorts of real tasks. I’m already involved with programmes in health and construction, and have seen many more that involve not just being there but doing things within these total worlds. If performance matters and the learning needs to produce people who can recall what they learnt and apply it in practice, then learning by doing really does matter and VR delivers.
4. Context – keep it real
A well-known research finding in learning concerns the power of context. To ground learning in the context of real world problems is to move towards real improvements in performance, especially in the world of training. We recall more when we learn in the same context in which that knowledge or skills has to be used.  VR can place us in real, rare, dangerous, even impossible situations to learn. I’ve taught Newton’s three laws through a Spacewalk VR, where the learner dons a spacesuit and flies, under their own control around the International Space Station. For construction VR programmes, the learner learns on a construction site with real equipment, real tasks and real colleagues. In the social acre programme, you learn within a care home and deal with actual residents to learn the competences identified in national frameworks.
5. Transfer – matched tasks
Transfer means the ability to apply real knowledge and skills from the learnt environment to the real world. From Thorndike onwards, we’ve had research and evidence that shows how a match between the two can enhance learning and subsequent performance. VR allows you to enter another simulated world (social care home, construction site, physics lab, operating theatre, hotel, burning building or war situation, etc) and learn and apply knowledge and skills.
6. Cognitive swap – to see ourselves
This is something that it almost unique to VR. To see ourselves as others see us or put ourselves in the shoes of others – that is often a useful learning experience. In VR there has been gender swapping, race swapping and self-observation. There is even an experiment where an artist is spending 28 days living through the eyes of another person, using VR. This is already producing results in terms of awareness of gender issues, such as sexual harassment, racism and so on. In the social care VR programme, mentioned earlier, the learner becomes an older resident and we simulate poor vision, to teach competences such as ‘ saying your name when you enter a room’.
7. Retention - increases
VR learning experiences are focused, vivid, intense, relevant, real, practical, contextualised and even allow the impossible. We know that consolidated long-term memory needs these conditions, along with repeated practice in order to produce competent recall. VR can deliver these pre-conditions for real learning and, in my view, do it faster than other methods of delivery.
Conclusion

Learning theory backs up VR as a learning medium and I haven’t even mentioned cost. VR, especially consumer VR, brings VR learning into the hand of learners with cheap consumer devices and a publishing industry that will sell content. The business model is the same as that of games, with cheap devices and low cost games, sold at volume.

 Subscribe to RSS

Monday, December 08, 2014

Why Online Educa Berlin is just so damn much fun!

With a Christmas market near the hotel, you can nip out for a Gluhwein and Currywurst or try the Florian, a real Berliner restaurant. Behind the venue is the beautiful Tiergarten - you can walk through this to the Brandenberg Gate. I’ve said this before but this is very much a ‘Berlin’ conference. It wouldn’t be the same anywhere else. It has that Berlin vibe. You may, like me, find yourself not going to bed until 2am then 4 am, although I have the ‘it was my birthday excuse’. My son lurched back at 7am after a night of clubbing. Then there’s the museums and art galleries. My own favourite is the Hamburger Hof, with Anselm Kiefer, Joseph Beuys and always a huge contemporary exhibition section.
Party time
Berliners can throw a party and the Thursday night bash is an all you can drink affair with a band and folks of all ages cutting a fair few shapes on the dance floor. This year it was my birthday at midnight and as the band struck up ‘Happy Birthday’ the real drinking began. It got messy.
People not speakers
This year - Howard Rheingold, Stephen Downes, George Siemens and a raft of people, some funny, some highly analytic, some passionate about projects and mostly people who have something new and useful to say. Where else could you get introduced to Howard Rheingold and compared notes on the joys of soldering? The ‘Spotlight Stage’, quick 30 min sessions on juicy topics. I did VR (Virtual Reality) and AR (Augmented Reality), where Todd Revolt and I compared the two with a raft of educational applications.
Good sessions
Sure there’s a lot of parallel sessions but where else would you get sessions on everything from a hard-swearing, Australian comedienne talking about anal beads to something on learning analytics, then a session on the use of educational technology in the Ebola crisis? There’s an entire ‘business’ stream with some hard-nosed sessions on learning and development.
Small village feel
Big but intimate, you bump into people you know (or don’t know) all the time. I don’t know a conference of this size that feels so much like a small village. You can move from room to room with ease. The Marlene bar is like the local saloon, it’s the place for official meetings and just like the local saloon, it gets pretty lively at night.
Education & business
With delegates from education, government and business, there’s good cross-pollination. It’s a private and public affair, not talking past each other, but maybe keeping each other honest. I like this. It’s easy to stay in your institutional comfort zone and get all anti-corporate of you’re in education or government, and all sniffy about education if you’re in the private sector.
Big Debate








Chaired by the charming Harold Elletson, this parliamentary style debate (which means an open bar) usually ends up as a piece of high theatre, both sides bashing each other over the head with arguments and/or jokes. On occasion, it can get edgy. Aric Sigman wouldn’t speak to me after our ‘cage-fight’ in 2009. And one of my favourite sessions of all time, was the debate with Jeff Staes, who turned up with a full-sized, stuffed sheep. We argued that all diplomas and degrees should be banned to an audience who largely taught and sold degrees and diplomas. Believe me when I tell you that their sister debates in Africa are even wilder!
International
With people from over 100 countries, you’ll meet all sorts. I especially like the fact that the same organisers do E-learning Africa, so there’s some grounded stuff that’s not just first-world reflection. It’s easy to get stuck in a national perspective and here you can ask what’s happening elsewhere, as well as explore other markets.
Exhibition
This is a sizeable affair so you can get round a lot of vendors between sessions, the ‘demo’ day is also good. I was showing the Oculus Rift this year and so many turned up I had to give impromptu talks to the people in the queue waiting to try it. You’ll see the big boys alongside the little guys trying to launch their products.
Conclusion

If you want to end the year with an event that will stimulate your weary neurons, fill up your boots with new ideas and get your dancing feet twitching, Online Educa should be in your calendar. See you all next year.

 Subscribe to RSS

Tuesday, December 02, 2014

Oculus Rift: consumer release – what & when?

So, Facebook bought Oculus for $2.3 million - $400 million cash, $1.6 billion Facebook stock and $300 million on Oculus meeting financial targets. What next?
The DK1 blew me (and almost everyone I showed it to) away. The DK2 has had them smiling, gripping, swaying and screaming. The consumer release is imminent. Brandan Iribe claimed that the launch is near and most commentators see it in 2015, May or November. The DK1 was good but was relatively low resolution, with a slow pixel refresh rate giving motion sickness. The DK2 was better with higher resolution and the motion sickness seemed to have gone but a more limited field of view. Then came Crescent Bay, a step up from the DK2.
Kits
Development Kit 1 had a LCD screen with1280×800 (640×800 per eye) and 3DOF (3-axis rotational tracking). Note that a DOF is simple a type of movement, either forward/back, up/down, left/right  (3DOF) or rotations (roll, yaw, pitch) – hence another 3DOF giving 6DOF in total.
Development Kit 2 has an OLED screen with higher resolution 1920×1080 (960×1080 per eye) and 6DOF (3-axis rotational tracking + 3-axis positional tracking). This additional tracking, helped by an extra, external sensor allows you to lean down to pick things up and lean round corners.
The Consumer Version will have an OLED screen but we don’t quite know what the release resolution will be, but it will be AT LEAST 1920×1080.
Sound
Built in headset with 3D sound is what Oculus are after. In demos I’ve seen so far the audio adds lots and massively enhances the experience and emotion. With 3D sound the idea is that sounds can be placed by the designer anywhere in the 3D space you’re in or dynamically in relation to the position of your head. This stuff will be a real headf**k, especially in horror games, which are already shaping up to be THE genre of choice for the Oculus. I can see the music industry responding with almost live experiences. I showed the Oculus to Lawrence Bell (signed Arctic Monkeys) and he sees its poetantial.
Controllers
Oculus bought Carbon Design Group, who created the look and feel of the Xbox 360 controller and original Kinect camera. They had been working together for some time, so the die is now cast, with Oculus likely to come out with their own controller. A games controller works fine and the Xbox 360 is as good as you get. You can zip around at a fair old lick. 
But the next step is to get something like the Razer-Hydra in place, where you can grab, lift and manipulate objects in 3D space. I’ve tried this and it works fine, Trouble is, they’re no longer in business. Something will come along here, especially with some haptic feedback.
Treadmill
For those who want to walk and run through environments, there’s the Virtuix Omni, the omnidirectional (love that word)  treadmill with a bowl shaped platform, special low friction shoes and a safety ring.
Backpack VR
With the kit on your back, you are untethered and can walk around an open space (and play zombie games.) Companies like VR Backpack provide an Oculus Rift headset with a powerful PC and hand controllers/gun peripherals. VR backpack’s latest (Version 3) uses the Oculus Rift DK2, WebVR software platform, and ultrabook laptop. An interesting problem here was getting the computers to work with the lid closed!
Conclusion
My interest is in educational applications but VR is a medium, not a gadget and there will be inroads into entertainment (film & TV), Games, healthcare and art. The Oculus Rift (and Galaxy Gear VR) is only the start of a move from 2D to 3D that will revolutionise media.

 Subscribe to RSS

Monday, December 01, 2014

10 mind-blowing Oculus Rift experiments in education

Marc Zuckerberg tried the Oculus Rift headset, was blown away and bought it for a staggering $2 billion. It’s a pure tech buy as Oculus has no customers. What Zuckerberg saw was more than gaming. What he saw was the breadth of possible applications, one is education.“Imagine…. studying in a classroom of studentsvand teachers all over the world - just by putting on goggles in your home.” Some of us are now working to make this a reality.
1. Sex swap
Hundreds of millions are spent on often useless classroom and e-learning equalities, sexual harassment and diversity training. We now have the chance to allow you to swap gender or race, seeing the world from their point of view. I played around with this using a female avatar in Second Life – it was a harrowing experience! This crazy experiment shows us the way. You will, literally see and feel what is like to be someone else. If you think this is crazy, the artist Mark Faird will spend 28 days wearing the oculus and seeing the world through another person’s eyes. Watch video here.
2. Tank training
The Norwegian Army have rigged up the Oculus Rift to four cameras around the tank to feed in scenarios, and other map data, to allow drivers to drive tanks with the hatch down. This is just one of a number of military training programmes using the Oculus and VR, tradition users of expensive VR will now see prices plummet. Others examples include medical triage. Watch tank video here.
3. Social care
Caspian Learning, funded by the University for Industry, have produced a Social care training programme, where you literally learn within a 3D model of a care home, dealing with the residents (avatars). You also see the world through their eyes (blurred). It trains, assesses and tracks competences with a full set of analytics that shows progress and proven attainment in each competence.
4. Construction
One more solid training programme from Caspian Learning and Ufi, on training people in retail. Caspian are also doing a health & safety programme on construction sites. You enter a site and have to choose the right gear for the right jobs. If you don’t, accidents happen. This stuff allows you to experience the consequences of your action, do the impossible – through safe failure.
5. Physics
Do you know Newton’s three laws? Try the Spacewalk, where you don a spacesuit and navigate, using small jets on your suit around the International Space Station. Squirt your jet and you move forward, that’s newton’s 1st Law – every action has an equal and opposite reaction. . Sounds simple, but most people are surprised that even when you turn the jet off, this is space, so you continue moving forward. That’s Newton’s 2nd law - very object in a state of uniform motion tends to remain in that state of motion unless an external force is applied to it. How dis the Space Station get up here – that’s Newton’s 3rd law – F=ma. Watch video here.
6. Biology
The undersea walk allows you to do the impossible – walk across the sea floor to Teach geography, geology or biology. Teach the food chain as you walk beneath beneath a humpback whale and shark. Encounter a hydrothermal vent shooting out superheated water at 400 degrees centigrade. Watch video here.
7. History
This is a shocker. It’s 1793 in the middle of the Reign of Terror and you’re about to be executed. Experience this for real, as you get your head chopped off on the guillotine. You literally lean forward and see your head drop into the basket. When do you think the last execution using the guillotine took place in Fance? 1977! Watch video here.
8. Field trips
Harvard have been using VR to create field trips and I don’t just mean, the usual trips to the country to look at plants. In this case you can visit the Great Pyramids in Egypt for some real archaeology. Museums, art galleries, geographic locations, battlefields – you name it, the oculus will not only take you there but allow you to explore at will and learn things in context. Watch video here.
9. Dangerous environments
Makemedia has built this Oculus application that allows you to build a nuclear reactor. It uses the Razer-Hydra with Oculus Rift to allow you to do things within different components, lift, insert, attach etc. The Razer-Hydra provides the learner with a pair of virtual hands to pick up and insoect each component of the primary circuit., such as reactor core, steam generator and pressuriser. This manipulation of objects, along with real haptic feedback will greatly improve many vocational training tasks. Watch video here.
10. PTSD
The Virtual Iraq/Afghanistan PTSD Exposure Therapy System allows the patient to replay traumatic experiences as part of therapeutic recovery. This comes out of the MedVR Lab at the University of Southern California Institute for Creative Technologies (ICT). Watch video here.
Conclusion

What we will see is some quite serious ‘learn by doing’ VR education and training. These programmes will have clear learning objectives, with clear competences, tasks and goals. AI driven avatars will provide interaction with people. We’ll also see support for teachers, trainers and tutors. VR is a medium, not a gadget – a learning medium.

 Subscribe to RSS