Voice Scene Generator

We developed an advanced, AI-powered system that enables users to create a smart home scene entirely through voice interactions

The problem

Integrating voice-driven scene creating into a smart home platform is a complex yet essential task for enhancing user experience and accessibility.The client`s platform, which operates through a web browser or mobile application, currently allows users to create scenes using a graphical user interface.These scenes are crucial for managing smart home devices across various rooms and configurations. However, creating scenes via the GUI (graphical user interface) can be time-consuming and challenging for users unfamiliar with the interface of those seeking a more intuitive experience. The challenge lies in enabling users to interact with a virtual assistant that understands their specific home configuration – including devices and rooms and uses voice commands to create scenes seamlessly. This interaction must replicate the functionality of the GUI but be simplified into a conversational format. The solution requires not only advanced voice recognition and natural language understanding but also integration with the client`s existing platform to ensure accurate and effective scene creation.

The solution

To address these challenges, we used cutting edge technologies, including Large Language Models (LLMs), a robust Backend Web Framework, an intuitive Front-End Framework, a Non-Relational Database, and a scalable Cloud Platform. Key components of the solution include:

Voice Assistant Integration: The assistant comprehends user commands by utilizing LLMs to interpret natural language and link it with the user`s home configuration. It identifies the user`s devices, their technical capabilities, and specific preferences to generate scenes that align with their expressed requirements.
Seamless Scene Creation: The assistant mirrors the functionality of the GUI by converting voice commands into actionable configurations within the client`s platform. This enables the user to create scenes intuitively and efficiently.

Testing System with UI: A dedicated testing system was built to ensure the reliability of the solution during development. This system tracks user test conversations, monitors configurations, and evaluates results. It provides transparency and simplifies debugging during iterative development.

AI-Driven Flexibility: The solution dynamically adapts to user preferences and device capabilities, ensuring a personalized experience that is both accurate and easy to use.

The final product empowers users to interact with their smart home through natural conversations via a mobile application. By eliminating the need for complex manual configuration, the system provides a streamlined, intuitive, and user-friendly approach to managing smart home devices.

Our team was responsible for

Identifying and gathering requirements
UI/UX
Implementing the web user interface
Integration with Large Language Model (LLM)

The client

Alterco Robotics is a manufacturer of home automation devices and solutions. They offer a proprietary cloud platform similar to Apple Home and Google Home, where users can create custom scenes using a sophisticated user interface. The company has tasked us with finding a solution that enables users to effortlessly create scenes using voice commands, facilitating intelligent home management. This solution is accessible through Amazon Alexa’s voice assistant.

Frontend technologies

web UI – Angular

How we used ML

The project leverages the capabilities of a Large Language Model (LLM) to power an intelligent assistant that facilitates smart home automation. The LLM is employed to understand and interpret user inputs, which include complex descriptions of device configurations, room setups, and intended actions. By processing large text batches and maintaining contextual awareness across conversations, the LLM can:

– Understand Contextual Information: It interprets device details, room layouts, and the user’s specific intents to create tailored solutions.

– Enable Intent Recognition: The LLM identifies user goals, such as creating scenes or automations, by analyzing the sequence of conditions and actions specified in the input.

– Generate Scene Configurations: Using its ability to process detailed prompts, the model generates sequential instructions for smart devices, enabling users to automate tasks like lighting adjustments, climate control, and security protocols.

– Maintain Conversational Flow: The LLM keeps track of prior interactions, enabling it to provide coherent responses, refine previous scenes, or adapt to new requirements.

This approach capitalizes on the LLM’s strengths in language understanding, contextual reasoning, and large-scale text processing to create a seamless and dynamic assistant for smart home management.