End user testingJuly 2, 2007
Had a beer and lunch with one of my old bosses yesterday which lead to me writing this…
A few years ago I was lead tester on an internal automation system. It was the longest eighteen months of my life, with my time and duties spread across three sights in London. Without going into the history of it too deeply, deadline over run, budget overspend and feature creep… We were finally at the point that it could be rolled across two sites for a total of seven different shifts, something like 250 operators and a smaller number of admin staff. Virtually everyone in the company was going to be affected by this in one way or another.
Some small amount of user testing had been carried out a few months previously and their findings incorporated where applicable. Now we were faced with the whole company going live simultaneously. I had a couple of guys fully up to speed to assist me and I had completed all the documentation ahead of schedule. There was still a great number of operators that needed training and support.
Each lead operator form each account on each shift along with their equivalent from the admin side would take part in a week of ‘training the trainers’, spending a day using the software in a simulated but real world situation where their normal workflow had been recreated.
They could then go and support their respective teams on the day, thus easing the burden of support. The training went well, everyone who received training was happy and we were pleased with the results.
Roll out day came around and within hours the bug reports were coming in. Text simply wasn’t updating correctly. As this was one of the main features and one of the reasons for the delay in releasing, we were dumbfounded. Our first suspicions centred on ‘special characters’ such as line endings, em dashes, soft returns and so on. These had been a major headache for us. We couldn’t replicate what the user was experiencing.
I went to see what the operators were doing…
Bingo! This part of the software had been written to update entire blocks of text, which it did flawlessly whatever the translation to any European lanaguage. However, the operators were trying to update just a portion of the text. The software couldn’t find the ‘end of line’ it expected and garbage was being written. When the software was used correctly it didn’t happen.
Nowhere in either the Documentation, Training notes or the actual training was it mentioned that you could select just part of a text block. In the documentation, it specifically said NOT to do so.
I wandered off muttering RTFM. I called my boss and explained what had happened. All he said was “effing end users!”