For my Master's Thesis I researched the possibilities of using Natural Language processing to make an IoT-system more intelligible for the end user. In order to achieve this, I designed a chatbot that enables the user to ask questions about their system's behavior.

HassTalk ensures the user can determine the (emerging) behavior of his / her smart home. By asking questions, the user can find out whether the system is behaving properly, with the possible questions being

  • Why is something happening
    Eg. Why is the lamp on?
  • When will something happen
    Eg. When will the heater turn on?
  • What will happen if X happens
    Eg. What happens if I leave?
  • What will be the state of a device at a given time
    Eg. What will be the state of the lamp at 8pm?
Why is it necessary?

In my research, I stumbled upon three main reasons that prove the need of such a software package. First of all, there is a technological gap between most of the end users and their systems. Not everyone is as tech-savy as I am, so why would they even consider implementing a Smart Home? Can it be done by people that do not have an extended technological background.
Secondly, after surveying the users, it emerged strongly that a large majority are afraid of losing part of their control over their home. After all, decisions are made that are not necessarily influenced by the user. What if these actions are unwanted or completely unexpected?
The third argument goes hand in hand with the second one: it is proven that making software intelligible empowers the end user to explore it's possibilities. By providing the possibility of questioning the service (and thereby implementing natural language processing), we achieve a greater level of understanding, control and power. In this manner we hope that the end user is more motivated to implement and explore the possibilities of a smart home and home automations.

Explaining behavior of the system

In the video above an example is shown for the use of the why-question. In the exploratory study, this form of questioning the system was determined as the most important one. This question enables the end user to get an explanation about a certain behavior of the system. In this manner, we ensure the end user has an understanding about the decission path the software has taken.

Future prediction

The developped system is able to predict the behavior of particular devices in the system. Thereby, a user can anticipate the actions and decision process of their system. As a follow up question, the end user can ask for some more explanation on why something will happen in the future. This feature is mainly focussed on debugging existing automations that went wrong in a particular situation.

Hypothetical Analysis

In this example you can see a user asking the actions that will be taken when a person leaves the house. This is just a starting point to show that we can predict the decision path of the system with small alternations. Imagine making an automation for your HVAC system and you want to see the behavior in different circumstances in terms of time, temperature and presence of certain people.


HassTalk is a chatbot system that combines natural language processing with a smart home environment. It enables the end user to take back control of the behavior of their house by questioning their specific system. In this manner, the system tries to empower and engage the user to explore the features en possibilities of their smart home and home automations. If you desire a more in depth read, please contact me by using the contact form on this page or by e-mail on [email protected].

Technical details
Used languages
  • Java
  • JavaScript
  • Python

Used frameworks
  • Node.JS
  • Spring