We were keen to get simple answers to simple questions. Never mind the tech specs - what’s it like to live with energy saving devices? And in this case: are smart plugs a good idea? Are they useful to most people?
The short answer, for our reviewers at any rate, is "no". Our reviewers struggled to get most of the smart plugs we looked at even set up, let alone making them useful.
But the long answer is more complicated than that.
Are smart devices a good way to save energy?
We know that Bulb members care about the environment, and they care about their energy bills. We've always been interested in finding ways to help members save more energy.
As more and more people show an interest in saving energy, the shops are stuffed with more and more products that claim to help them do that. Most of these products say they're "smart", which is usually a way of saying that they can connect to your home wifi network, and through it to your mobile phone, wherever you are.
Our normal approach with new ideas is to start small and work up from there. So we started as small as we could think of, with a selection of smart plugs.
What is a smart plug?
In the words of one of our volunteers, a smart plug is "a plug for your home electrical devices, which you can control with your phone".
Stick a smart plug into a wall socket, then plug something (anything) into the smart socket, and now that thing can be controlled from anywhere. For example: you might use this feature to control your lights while you're on holiday, to make it look like someone's home. Or you might use it to turn the kettle on exactly two minutes before you get home.
Some (but not all) smart plugs have the ability to measure how much energy the device that's plugged into them is using. This can be a useful way of finding out how much it's costing you to use the tumble drier, or how much energy your washing machine uses for a hot wash compared to a cooler one.
Some smart plugs can work entirely independently - you just plug them in, connect them to wifi, and off you go. Others work better, or only work at all, if they're part of a larger system of devices. Some need to be purchased with a hub, a device that acts as a bridge between the smart plug and your wifi router.
For our experiment, we decided to stick with the lower-cost, independently capable smart plugs. As you'll see if you read on, that was possibly a mistake.
How we did it
After some desk research to explore what's available, we decided to purchase a selection of smart plugs. Then we asked around the Bulb office for volunteers who'd be willing to borrow one each for a fortnight, and report back with their opinions.
This is a good moment to point out that we're not professional gadget reviewers. We looked at websites like The Wirecutter and we admire the work they do. But we're not them. We're an energy supplier, not a media company. We realised from the outset that our efforts to review products like this would not be the same as those done by the professionals, and we shouldn't pretend they are.
We're here to represent the interests of our members. We know them pretty well, and we know that many of them are keen to use new ideas to help save energy. But not many people have time to wade through loads of detailed product reviews. Sometimes, you just want to hear what it’s like to use a new gizmo, from someone else who’s used it already.
That’s what this experiment was all about.
The devices we tested were:
Each was tested in an ordinary home, not a controlled testing environment. The reviewers had little or no previous experience; we didn’t prepare them, train them, or give them much of a brief. We wanted to make this as close as possible to someone buying the device, taking it home, and seeing what happens.
When the fortnight was up, we gathered our volunteers together for a discussion.
We expected to see an even distribution: one or two devices that emerged as "good", one or two that people thought were "bad", and the rest sitting somewhere in between.
But what happened was rather different.
At the beginning of the discussion, we asked the volunteers to give their smart plug device a first-impressions mark out of 10. Our plan was to use the next 2 hours of debate to help everyone moderate their scores, and give them a chance to change score at the end. That way, we hoped to make the process fairer.
To our surprise, only one device scored more than 5/10 at the start.
The initial scores were:
It quickly became clear that the majority of our volunteer reviewers - all but one - were disappointed with their smart plugs. There were a number of reasons for that:
Setting them up was hard work
All but one of the devices was "difficult" or "almost impossible" to set up in the first place, our reviewers said.
Deborah (Efergy Ego) said: "It was almost impossible to set up. The worst experience ever." John Joe said he felt the same about the Heatmiser Neo he tested.
Lettie (TP-Link HS-110) said: "You needed to be really tech-savvy to get this thing up and running."
And Bill (D-Link) said: "It assumed a lot of prior knowledge for the set up phase. My mum would have given up."
The exception was the Belkin Wemo, which Michael L said "was simple to set up, even though there were no instructions."
Apps were a mixed bunch
All of the smart plugs were controlled by accompanying smartphone apps. Lettie said her device was useful to a point: "It showed me how much energy was being used, but didn't have a way to show me changes in consumption over time. So I had to take screenshots of different readings, and calculate the difference myself."
Michael L (Belkin Wemo) said: "The app for this device is ok. It's easy to use, and I was impressed that if I changed some text in the app from 'lamp' to 'kettle', my Amazon Echo responded to the new spoken command and everything continued to work."
John Joe, using the Heatmiser Neo, found it hard to even get hold of the necessary app. "It took ages to find online, but even once I had it installed, it was annoying and difficult to use."
They're not that useful
Michael T (TP-Link Kasa Smart Plug) found his device unreliable. "Wires are solid, reliable technology," he joked. "This smart plug just wasn't very useful to me."
Bill thought his D-Link "might have some uses for home security - perhaps controlling a remote wifi camera." But he struggled to find many other uses for it.
Deborah said the Ego felt like a wasted opportunity. "I wasn't engaged by it. It seemed to me that there were loads of things it could do - like congratulate me for low energy use, or send me alerts when I'm using too much. But it wasn't particularly useful."
Even after set up, every reviewer struggled to justify using the smart plug in their home. Most found some use for it as a remote control, but often for devices it would be just as easy to control with the usual button or switch.
They're not always that smart
One thing that cropped up during the discussion was how "dumb" most of the smart sockets were. They can communicate and be controlled over wifi, but one smart plug only copes with one household gadget plugged into it. To really understand energy use in your home over time, you'd need a lot more smart stuff: smart sockets everywhere, and a smart energy meter, and a way of collating, analysing and presenting all that data in a way that's useful and usable. It's possible to do that, but it's very expensive.
Wemo the winner?
The Belkin Wemo was by far the favourite device of the bunch. Reviewer Michael L said he and his family liked the easy set up and connection with Amazon Alexa for voice-control. He found it useful for remotely switching the kettle on first thing in the morning, or for switching off lights when carrying other things in his hands to go upstairs to bed.
But despite his higher scores, at the end of our debate Michael L was very clear: "I wouldn't recommend people buy one of these," he said. "They're nice to play with but there's really just not much of a need." He was disappointed that this Wemo didn’t offer an energy usage measuring feature, something that its sister product the Wemo Insight does.
Conclusion: think twice
We thought we'd be ending this with a recommendation to buy a particular smart plug, but actually we've concluded something different:
We don’t think smart plugs are that useful to most people. Generally speaking, they’re hard work to set up and most people didn’t really have much use for one. They don’t make much difference to people’s lives, or to their energy bills.
Our advice is: think twice before buying one on a whim. It's best to buy one when you have already identified a specific need in your home.
If you do decide to buy a smart plug, you should carefully research things like:
Will a smart plug work on its own, or need a hub, or need to be part of a network of smart devices?
Will it work with things like Google Home, or Amazon Echo?
Will it tell me about energy consumption - and if so, will it display and share that information usefully?
Will it be worth the cost? We're not convinced that the cost of buying several smart plugs, a hub, and a voice-controlled device such as Amazon Echo, would be offset by the money you'd save from using less energy. Simple common sense - use high energy-consuming devices less - will probably save most people just as much money, with no outlay at all.
If you’re still interested in buying one, these reviews may be helpful:
While all of those articles covers a different set of devices, and they don’t all agree about everything, they do all pick out the Belkin Wemo Insight, one step up from the Wemo Switch we looked at, as their favourite.
Your experience may be different
Our experience was disappointing, but yours might be the opposite.
If you think smart plugs are useful, or if they’ve made a difference to your life or your energy bills, we'd love to hear from you (particularly if you're a part of the Bulb Community). Not just about which devices you've used, but also what for? How much did you spend? How has your energy consumption changed? How much have you saved?
Most importantly: what advice would you have for other people who wanted to follow in your footsteps?
(Thanks to our volunteer reviewers, their families, and housemates.)
What we're doing next
This experiment didn't turn out exactly how we expected, but we still think it's useful for us to understand what products are out there and how they work, and while we're at it, to share what we find with you. We're also going to think about more than just energy saving - 'smart things' do have other uses we should think about.
So, we're going to carry on testing. Next, we're going to look at smart thermostats.
If you're a member of the Bulb community and you'd like to join in as a product tester, please let us know.