User interviews and observations are a great way (my favorite) to gather insightful feedbacks. More than subjective opinions of various stakeholders, observing how people use your interface to complete tasks is the best way to get objective, rational and unbiased insights. As Jakob Nielsen said, no need for thousands: 5 users allow you to identify most of UX issues. Same goes with the questions to ask during the interviews. You can’t come up unprepared, as this will lead to inexploitable, uncomparable data. But preparing a long list of questions is counterproductive, as you need to be careful not to lead the answers. As for me, I always use the same template of questions for my interviews. It is based on Avinash Kaushik’s 4 questions.

Introduction

During this first phase, try to start a conversation with the user. Start with easy questions on his/her role in the company, main missions and perception of the system. I also ask how frequently the user opens the system and what his/her general satisfaction towards it is.

1. What reasons do/will you come to the system for?

Here, I list all main tasks the user might use the system for, and try to put a proportion for each of them. For example, the user will say he/she comes mostly to search for a person (80%), and sometimes to edit the information (20%.) For each of these tasks, I then observe how the user would complete it via the proposed interface. I ask him/her to comment, and then ask the following questions.

For each mentioned task in 1.:

2.  Did you succeed in doing what you wanted to do?

Observing the user conducting each task, and as he/she comments his/her navigation, I write down every mentioned difficulty and what path the user chooses intuitively. If the user is stuck, I try to ask if he/she had noticed this or this element, trying to give clues, but never leading the user. In the end, I write if user managed to complete the task or not.

3. If not, could you comment some more on why?

If the user could not complete the task, we analyze together the reasons of this failure, and try to think of how the interface could be improved to help him/her complete this task better.

4. What is your satisfaction towards the system regarding this task?

Even if the user managed to complete a task, it could mean he/she got used to this process, but not necessarily that it is ideal. So I ask user to evaluate his/her satisfaction at the end of each task.

I usually finish with more specific questions, and end with « What do you like / dislike about this interface? ». All this usually takes between 45 minutes to 1 hour (exceeding 1 hour often leads to a dicrease in attention.) I carefully write everything down on my printed templates, for later reference. Between each session, I take 15 minutes to order my notes and clean my mind a bit. Once all interviews are done, I deliver a conclusions summary with most mentioned difficulties and issues, as well as clues to solving them.