Usability
Testing of a Smart House System Interface
Susanne
van Dongen
Faculty
of Industrial Design Engineering, University of Technology Delft, Netherlands
Marleen
Stijnman
Faculty
of Industrial Design Engineering, University of Technology Delft, Netherlands
Jonathan
Talbot
Industrial
Design, Faculty of the Built Environment, University of New South Wales,
Australia
ABSTRACT
This
paper reports on the evaluation of a ‘smart house’system that has
a variety of functions that may arguably help elderly people with their
daily living. This evaluation sought to determine how appropriate the
system interface would be for older people in its existing form. A modified
interface was also proposed that might make the existing system functions
more usable for older people. Usability testing of the two system interfaces
was conducted with a sample of 10 people ranging in age from 56 to 76
years. The main conclusions were, firstly that use of the existing system
would present many difficulties to older people, and secondly that the
people participating in this trial favoured the usability of the new
design over that of the existing interface.
1.0 INTRODUCTION
The phenomenon
of ‘the aging population’ is occurring in many societies and it
can be argued that an increasing number of older people will be seeking
to live independently well into their later years of life. Despite
common perceptions that large numbers and proportions of older people
live in supported accommodation (such as nursing homes or hostels),
comparatively few of them do so. The vast majority of older people prefer
to remain in their own homes. Relatively few people decide to move into
supported accommodation, although the number that does so increases
after the age of 75 years. Australian Bureau of statistics data for
New South Wales (for 1991) indicate that older people were more likely
to live in separate houses, with about 465,200 people aged 65 years
and over living this way (ABS 1995). The most common living arrangements
(about 65% of people over 65 yrs) were couples living without children
present or lone persons.
There may be
good social and financial reasons to enable older people to continue
living independently in their own homes. However, loss of mobility
and other consequences of aging may restrict the ability of many of
these elderly persons to live without some kind of help. So-called ‘smart
house’ technologies, which enable devices and systems in the home
to be controlled automatically, may offer some support to people who
might otherwise have difficulty coping with independent living.
Various proprietary
systems have been introduced in recent years and they can offer a wide
range of functions. These can be described under the following
headings:
Communication
systems
Technologies
which facilitate easy communication with others to gain access to social
contact and services may assist elderly people living alone. This may
extend from conventional telephone contact to ‘on-line’ services
or new mobile communications.
Alarm Systems
‘Social’
(or ‘duress’) alarms allow users to call for assistance in a variety
of situations. These systems have proved themselves to be very valuable
for the older people who are living alone. More complete alarm systems
can be made available with little extra cost. These can include full
audio communications and features such as remote gas and fire detectors
and burglar alarms (Spit and Nouws 1997).
Monitoring
Systems
These systems
allow a person to be monitored from a remote location. This might include
using motion detectors in the home or monitoring indicators such as
water usage (toilet flushing for example) or appliance usage to sense
activity (Perenboom and Zaal 1995). Health signs can also be monitored,
such as the ambulatory monitoring of heart rate.
Environmental
control systems
These may involve
basic automation of household conditions such as thermal environment
and lighting and security (locks and alarms). More complex systems can
control many aspects of a domestic environment and may adapt to different
behaviours of occupants.
A ‘Smart
House’ can combine the above technologies to provide a range of functions
that may be helpful to the elderly, but there is little evidence of
studies concerned with how older people can interact with such systems.
Since many older people are not very familiar with technical appliances
and their requirements and capabilities may often be different to those
of younger people, ‘smart house’ technologies can only deliver benefits
if they provide appropriate functions via an appropriate interface design.
The aim of
this study was to consider the application of an existing smart house
system in providing functions that would be useful for older people.
This involved investigating the usability of the interface of the system
to determine whether the functions of the system would be reliably accessible
to older users. Further, we aimed to propose an alternative interface
for the system and compare the usability of this proposed new design
with the existing one.
2.0 METHOD
For this study,
two systems were ‘mocked-up’ using Visual Basic software. User trial
participants interacted with the systems via a touch screen. One (system
A) replicated the existing system’s keypad and the software. The other
(system B) was a new design which used an alternative interface structure.
Both systems
supported the same functions. This set of functions and its hierarchy
in the interface was derived from the existing system and other currently
available products. The ‘working’ aspects of the interface mock-ups
were:
Turning on/off different
lights
Changing the settings
of heaters and air-conditioners
Open/close various
doors, windows and blinds/curtains.
Checking the status
of security devices, such as movement detectors and door contacts.
System A
The interface
for system A is shown in Figure 1. While ‘mocked-up’ for a touch
screen, this interface replicates a design with a display screen and
hard keys. The subjects can use the buttons on the right side
of the interface to operate the system. The screen on the left side
only gives the user information. The appearance of the interface is
always the same, only the text on the screen changes.
Figure 1. System
A interface
System B
This system,
shown in Figure 2, is a touch screen based design, since there are no
hard keys, the number of buttons can vary. This depends on where the
user is within the system.
Figure 2. System
B interface
User trials
Thirteen people
were selected in the age group of 55 years and over (6 women and 7 men)
Three people participated in pilot testing of the system which enabled
improvement of the test procedure and correction of problems with the
mock-ups . The remaining ten participants completed several tasks on
both systems. The tasks were the same for every participant and for
both systems. These tasks simulated use of functioning aspects
of the interface mock-ups.
Because of
the equipment needed it was not possible to go to the participants’
own homes. To be able to create a home environment necessary for the
tests they were set up in the house of a volunteer. The tests were video
recorded, that way major problems the subjects had could be viewed again.
If something was not immediately recorded correctly or if there was
some doubt the interview could always be viewed again afterwards. After
an introduction to the project, a training session took place. This
involved a preliminary task where the subjects entered numbers using
the touch screen. This was done to let the subject get used to the feeling
of working with a touch screen. Most participants had never worked with
this type of equipment, so it was good to let them experience with touching
the screen to interact with the system before the actual testing began.
This was the only training the participants received, because it was
decided that no training would be given regarding the structure of the
systems or the terms used in the interfaces. Also no user manual or
help function was available. The subjects had to rely on the interface
only. While it might be argued that training and user guidance would
be available in a real installation, relying only on the interface gives
a ‘worst case’ situation that can provide useful information for
developing an ‘intuitive’ interface.
The actual
testing consisted of 7 tasks. These were: putting the system in the
Home Mode; turn on the reading light; close all doors and windows; changing
the program time of the air-conditioner; putting the system in the Sleep
Mode; opening the curtains in the bedroom and checking the status of
the security devices. Task durations and errors were recorded. The sequence
of testing was varied to avoid order effects during the trials.
A short questionnaire was administered after each pair of trials. The
questionnaire probed the problems experienced while operating the system
and included some general questions about the usability of the two systems
and about whether they were appropriate for domestic application of
this type.
3.0 RESULTS
Graph 1. Total
time
Per participant
Task durations
The total time
taken to complete the tasks using system B was less then the total time
for system A for all participants. This can be seen in graph 1.
The mean of the total time of system A is 14:53 min. and of system B
6:05 min.
Graph 2. Mean
time per task
In system B
participants needed the most time to perform task 4, changing the program
times of the air-conditioner. When one looks at system A subjects needed
a lot of time to carry out tasks 3 and 4, closing all doors/windows
and changing the program times of the air-conditioner. This can be seen
very clearly in graph 2 on the next page. The dark bars of task 3 and
task 4 show the much greater average times for these tasks. To carry
out task 4 the subjects need to perform the most actions. This makes
the task more complicated and that is probably why it takes them more
time.
Mistakes
Every time
the subjects stray from the ideal route when carrying out a task this
is seen as a mistake. When one looks at the number of mistakes made
by the subjects in the sample, the amount of faults they make in system
A is always more or the same than that of system B. This is shown graphically
in graph 3 below.
Graph 3. Total
mistakes / Subject
Looking at system A and the
mistakes made per task (in Graph 4.), it can be seen that the tasks
that took a long time to carry out, tasks 3 and 4 as told before, also
were the tasks where most mistakes were made. In this sample there can
even be seen a relation between the mean time per task and the mean
amount of mistakes per task in system A. When the subjects make more
mistakes in a task they need more time to complete the task.
In system B most mistakes were
made in task 5 as can be seen in graph 4. But task 5 is not the task
that took the subjects the most time, that was task 4. Task 4 however
also has a high mean amount of mistakes made. In system B one can not
detect a relation between the mean time per task and the mean amount
of mistakes per task.
Graph 4. Number
of mistakes per task
Perceived usability
As can be seen
in graph 5 on the previous page the subjects in the sample gave mixed
answers on the question of how they would rate the user friendliness
of system A. The peak however is situated on the negative side of the
rating scale.
Graph 5. Response
to System A
The subjects
were more positive about system B as can be seen in graph 6. No subject
in the sample rated the system lower than neutral.
Graph 6. Response to system
B
All subjects
in the sample were unanimous in answering the question on which system
they liked working with best. All 10 of them preferred system B. The
rating scales for appreciation of the systems (graphs 5 & 6) corresponds
with the overall preference responses.
4.0
DISCUSSION
Overall the
participants in the trial favoured the usability of system B, the new
design, over that of system A,. The main differences between the two
systems are the visibility of the structure the less ambiguous display
feedback. This outcome suggests that a visible structure increases the
usability of an interface. The main conclusion to be drawn from this
study is that the existing system that was examined was unlikely
to offer an effective solution for assisting elderly people to
live independently. It would appear that it would not be sufficient
to merely take an existing system of this type and expect older people
to successfully adapt to using it to manage functions in
their homes. Successful application of ‘smart house’ technology
will require extensive interface development in order to provide usable
systems that will reliably support people in their daily activities.
The results
of the trials do not mean that there are no faults in system B. The
participants in the user trials still had difficulties while carrying
out tasks with this interface. There are a number of improvements needed
offer suitable resolutions to the problems older people face when interacting
with the new, proposed interface. Some of the problems found may be
overcome through training and manuals. Never-the-less, an ideal interface
would allow the user to work with intuition only, so a consumer product’s
aim should be to reduce training requirement to an absolute minimum.
‘Smart house’ systems of the type investigated are complex, supporting
a disparate array of functions. There is great scope for development
of intuitive interfaces that will provide ease of use for the full range
of people who might benefit from such technology. The testing
reported here did not really focus on the subjects responses to working
with a touch screen but we did observe that after getting used to it,
people seemed to like it very much. In the beginning the subjects were
surprised that a touch with their finger could make things work. Because
they did not need a keyboard or a mouse, working with the computer appeared
to be less intimidating and more direct. Other forms of
interface should also be considered. It may, for example, be possible
to embed multiple interfaces within the home environment which each
allow the occupant immediate interaction with the particular ‘smart
house’ function they wish to use. It may thus be possible to
create a responsive, supportive environment. It appears that existing
interface structures fall short of achieving this.
9.0 REFERENCES
Australian
Bureau of Statistics, 1995, Older people in New South Wales: A profile
Commonwealth
of Australia, 1995
(ABS catalogue
number 4108.1)
Perenboom,
J.M. and Zaal, K., 1995, Thuis blijven wonen met Big Brother: Passieve
alarmering dementerenden
Leeftijd;
vol 33, nr 6: p.31-33
In Dutch
Spit, A. and
Nouws, H., 1997, Bekneld tussen wetgeving en betaalbaarheid: prijst
integrale alarmering zichzelf uit de ouderenmarkt?
Leeftijd;
vol 35, nr 3: p. 10-12
In Dutch