Review - Loop11

I recently used Loop11 on a remote testing project, and thought it would be a good time to add in my two cents worth in terms of review.

For those who don't know, Loop11 is a survey tool, marketed as a remote testing tool. It lets you set up a test and then run it on a site - even if that site isn't your own - by using url links. The test can include tasks and questions, and all data is captured for you, up to a maximum of 1000 participants.

In terms of 'online testing' it's a world improved from a standard survey, but still a long way away from the ideal observed/measured remote testing environment I'd like to see. Having said that, it's offering a great deal for a reasonable price per test.

Okay, first for the good news. And there's a fair bit of it.

Loop11 allows you to configure your test with a number of tasks, and with questions before or after each, and pretty much everywhere else you need. Tasks have a start url and a success url, and these are tracked to measure success rates. That means you can ask people to access a certain page and then track how many succeeded. 

Questions have a great range of types, which allows you to configure open or closed questions with a range of options and feedback levels - although one piece of advice is to avoid open questions with text input, as this gets hard to review/manage once the responses creep up into the hundreds.

You can copy tasks and questions, and juggle them around in the sequence. At any time you can review the test and walk through it, to see how it works - an invaluable tool. Crucially it also doesn't force you to pay for a test credit until and if you want to launch your test, which allows you to become thoroughly happy with the product before you commit financially. 

The in-page support is great, with plenty of help and examples to get you going. Although it can be a bit slow to build questions in this format, overall it's a pretty slick experience.

When it comes to results there's lots to like too. Loop11 creates a very reasonable PDF output report, but also allows you to export the raw data in various formats including Excel and XML as well as viewing the results directly within your browser.

All this means the tool is easy to set up, easy to manage, and provides a great level of detail at the end.

I also have to commend the Loop11 team for providing great support. We unfortunately encountered a number of problems with our previous test, some of which were outside the control of Loop11, but the support team were always helpful and quick to respond - essential when you're looking at a potentially disruptive live survey on your website.

So what doesn't work well?

I think possibly the biggest gripe I have with this tool is that it blanks the screen while asking questions. During a task a small control bar appears at the top of the browser window, reminding you of the task and allowing users to mark the task completed or failed. Once either is selected the task ends and the user sees a blank screen whilst they are asked questions.

That does work fine for some forms of testing, but for more content/marketing driven tests it's far less useful. For example some questions may relate to whether the user can see a certain piece of content, or how they feel about something they can see. When the page is blank the user is then forced to remember what they saw, and try and comment on it.

We're currently configuring a test where the user will be asked which option they would choose in order to find a certain piece of content - and the possible list is pretty much anything on the nav, quite a long list. Whilst we can list them all and ask the user to choose, that's nowhere near as powerful as asking them to view the page and tell us where they'd go.

So if I could change one thing about Loop11 I'd change this - allow the interface to stay visible during questions.

I also find it personally limiting that the results don't tie questions to tasks. Questions appear in the report by number, but there's no immediate way of seeing which task the question relates to. That means flicking back and forth between the output report and the test structure, especially when you may have duplicate questions (for example "How did you feel about that task?").

Finally, we encountered some errors that made using Loop11 a little tricky at times. With our previous survey a number of people reported their browser crashing during the survey, and Loop11 were not able to identify the cause - though to be fair this may well be down to the external site we were testing. We also hit a problem when attempting to view results, due to the logic behind how report output is generated- that's something Loop11 have assured us is in the pipeline for improvement.

Overall I'd recommend Loop11 to anyone looking to run remote tests - it's not perfect, but it's a great extension to our toolkit and the positive support approach means you'll hopefully overcome any minor gripes you encounter.

You can see more at www.loop11.com.

A SIC website?

I've recently had the dubious honour of using the ASIC website to purchase a business name.

Now, before I say anything negative, let me be clear and say that I've worked on numerous Government websites in the past, and I do understand some of the issues they have to face - to name just a few:

• Tremendous amounts of data that need to be available
• Massively complex tasks that need supporting
• Numerous and disparate user profiles
• Legal restrictions around process/assistance and usage
• Internal organisational resistance to change

The list goes on and on, I'm just making it clear that I understand the limitations.

That said, using the ASIC website was something akin to spending a week with mittens superglued to your hands; you know it's going to be frustrating, you know things are going to be slow and painful, but nothing can quite prepare you for the world of frustration you've just stumbled in to.

I won't document the hoops I jumped through, but time and time again I ended up at the same point - seeing that the business name was available, but having absolutely no clue how to purchase it from here! Like a dog watching a cat on TV I uselessly pawed and clicked on things that were never going to work, whilst what I wanted (This name is available!') paraded in front of me with nary a button or link to make it mine.

Trap doors left and right launched me to new websites or opened massively complex help documentation that confused and confounded and invariably led me - eventually - back to where I started.

I did get there in the end, although to be honest I really am not quite sure how. 

So, it's over to you - has anyone else had a similar experience? ASIC, if you're listening, I'd love to help...!

When did helping become so hard?

Recently I was asked by a company who had contacted me to help them with some design. The request was quite simple, they had a product that was technically working well, but had a rushed user interface that was underselling the whole thing.


Not a problem for us, so we asked to see the interface. Fine, the company say, just sign our NDA. We sign them all the time, so they send it over, we read it - and stop.


This NDA was, shall we say, a little thorough in its wording. A touch over-protective. 


Firstly, it protected this company from any potential infringement on their intellectual property, which is fine. Those rights were aggressively defended, to the point where we were, in effect, obliging ourselves to help them sue us, if they so considered their rights infringed. 


Not only that, but at the end of the engagement we were to allow them into our offices, and full access to all PCs, laptops and storage devices, at their discretion and timing, to ensure that all traces of their materials and IP were removed. Now, call me old fashioned, but that raised an eyebrow or two.


The real killer though was the next section of the NDA.


Normally and NDA works both ways, protecting both parties and their IP. This NDA however protected only this company. Not only that, it actually enabled that company to take our IP, use it, sell it, and sub-license it, at will. It was in effect granting a perpetual world-wide license to this company to steal any and all IP we disclosed to them during the engagement.


A touch one-sided. Needless to say we didn't sign it, and the engagement didn't go ahead.


But when did helping someone become so hard?


What are your experiences with NDA's? Are they becoming more and more draconian, or is this just a one-off..?

Usability of the mobile generation

In today's world, it seems crazy that communicating with other people should be problematic. 


We have home phones, mobile phones, Internet phones. There is a call, a text, an Instant Message, a Tweet - via phone, Facebook & a thousand other apps. We all seem to be so connected we should almost be able to read each other's thoughts.


So why can it be so difficult to get in touch?


Part of the reason, it seems, is a selfish choice that leads in turn to frustration, cost and a failed experience.


I'm talking particularly about the trend of avoiding landlines. It's something that seems to be growing, particularly amongst younger audiences, but also in many families. And I can see the attraction, at first. The telecoms companies want a hefty connection fee, and a monthly line rental fee that takes a nice bite out of your finances before you even make a call. And since you're paying your monthly mobile anyway, why not just cut out the landline altogether?


It's something that I'm seeing increasingly, particularly amongst my children's friends. And it annoys me.


First off, you get the expense hit when your child wants to ring a friend. We have an Internet phone with free untimed calls to landlines. so they know they can use the phone to call a friend and chat for ages, for free. But if that friend has no landline, they'll ring the mobile instead. Instead of a few free calls on the bill, I get a $50-70 hit for long chatty mobile calls.


So I have to school my kids on talking quickly, and cut them off after a few minutes.


Next, you get the reverse call. Child's friend sends a text, usually along the lines of "hey, call me".


"Why can't they call you", I ask. "They don't have credit," the child responds. So, now not only do I have elevated bills to call this kid, I now have to pay when he wants to call our house.


And finally, you get the most frustrating issue of all - the inability to get in touch. 


This generally happens when that household has run out of mobile credit for the month. My child wants to organise a trip to the movies with a friend, and rings. 


No answer from their mobile, but no point in leaving a message since they are out of credit and can't pick messages up. So, he has no choice but to ring again later. Then ring again, and again.


Next, he tries ringing that child's mum (also, of course, on a mobile). Mum picks up, and says the child is off with friends later on. Not sure when he's due back - but since we don't have credit right now, call back a bit later.


You also encounter this problem, in a much more expensive format, when non-custodial parents have only a mobile contact. When children want to ring mum or dad every night, or several nights a week, just watch as the dollars fly out the door.


And it's really not just about the cost - the frustration and pain of repeatedly trying to get hold of people, leaving messages that may or may not get picked up, sending texts that may or may not be read, is a royal pain in the rear section.


It's a relatively simple choice to decide not to have a landline, but in effect you are passing the costs on to everyone around you who needs to get in touch. You're also frustrating the crap out of parents who need to get in touch, for one reason or another.


Vent over.

How to lose a customer - Step 1

It never ceases to amaze me how some businesses seem to throw roadblocks willy nilly between the potential customer and the sign-up moment.


This week, I had two perfect examples of this thrust into my face, whilst signing up for a gym membership.


I've been thinking of joining for a while, and had dropped in to a local gym to discuss prices. I'd gone through the sales process - the pre-talk, the requirements review (how much weight are you trying to lose, sir?), the walk-through, the pep talk, the works. They explained the price, and the discount offered for multiple members (I was signing up two of my teenagers, too).


At the last moment though, literally as I was standing with cheque in hand ready to drive in and sign up, I had an attack of price conscience, and realised I hadn't compared prices with any of the other local gyms. So, I picked the closest, and rang them.


I explained that I was signing up with a competitor gym, for myself and two others, and wanted to know what their prices were and what the discount might be, for three of us. The very nice guy at the other end listened, engaged (great choice, and a great dad for encouraging your kids to be healthy!) and then suggested I come in for a tour.


Thank you, I said, but I didn't have time - I was off to sign up elsewhere, I just wanted to see if they were comparable. Again, he suggested I came in for a tour. More firmly, I told him that just wasn't possible - I just needed to know how if they were at least comparable on price, and if so then I might come sign up with them instead. This was repeated three times, as he desperately tried to convince me to come in. 

Finally, we hit the crunch point. "I'm afraid if you can't tell me what your membership cost/discounts are, I'm just going to go to the other gym". 


And that was it, he said "sorry, I really can't give that out unless you come in", and I hung up the phone.


I was astounded - given how expensive gym memberships are, given that I was talking about a definite commitment that day, and that I was talking three sign-ups in one, it seemed beyond belief that the business would rather walk away from the sale, rather than give a simple price over the phone.


Not only that - but the exact same conversation played out when I rang the next gym in the list. Needless to say, they lost the business too.


I understand that businesses like to have a sales process, and that the sales process works best to convert customers when their needs are well understood. That's particularly true for complex products, and for products where final pricing is complex and/or dependant on use and need.


Gym memberships though are not in that category. In my experience they are generally a two or three tier price, with discounts for payment commitment. It's a simple, stock price - and although there are usually some discounts or offers around, the price generally is the price and remains so. Therefore there's no real benefit in 'understanding the user and their needs', it is simply a case of using that time to sell harder.


When you don't have that chance to sell harder - as in the phone call I made - the choice of abandoning the sale entirely seems - well, entirely insane.


Not so many years ago, we had a call from a company who wanted a price - over the phone - on a usability test. Generally, we like to find out as much as possible before we quote, to make sure we get everything covered in the fixed price; this time, the company were unwilling to provide much info at all, other than the basics (number of people, length of test, locations for testing). We quoted there and then, they hung up, and all was done. About two weeks later they called back and booked the test, and we finally talked about detail. During that talk they explained that they had rung three other companies, all of whom had refused to give prices over the phone. They have been a strong and trusting customer of ours since.


Forcing your customers to engage in the only sales approach you'll accept is a great way to lose customers.