Detecting Anomalies in IoT Telemetry

Transforming Raw Streams into Intelligent Action with Microsoft Fabric

The Architecture: An Intelligent Ecosystem

A unified data intelligence ecosystem built on Microsoft Fabric that transforms high-velocity IoT telemetry into real-time insights.

High-Velocity Ingestion

Captures telemetry at scale via Azure IoT Hub, Event Hubs, and Stream Analytics with zero data loss.

Serverless Orchestration

Azure Functions serve as serverless microservices to pre-process, clean, and validate raw JSON streams.

The Unified Lakehouse (OneLake)

Implements a Medallion Architecture (Bronze/Silver/Gold) on OneLake using the open Delta format.

Advanced AI & Engineering

Synapse Data Engineering and Data Science deploy large-scale transformations and ML anomaly detection models directly on the Lakehouse.

Intelligent Serving & Action

Power BI provides live dashboards while Data Activator triggers instant Microsoft Teams alerts on threshold breaches.

Enterprise-Grade Governance

Governed by Microsoft Purview, secured via Azure Active Directory and Key Vault.

Architectural Pattern: Layered Architecture (N-tier)

The system is built on a clean Layered Architecture (N-tier), ensuring separation of concerns:

• Data Ingestion Layer: Captures high-velocity IoT streams via Azure IoT Hub and Event Hubs.
• Processing Layer: Handles real-time parsing with Azure Functions and AI modeling via Synapse.
• Storage & Serving Layer: Microsoft Fabric OneLake Lakehouse coupled with Power BI and Data Activator.

Functional Requirements

• IoT Telemetry Ingestion: Capture and stream high-velocity sensor data with zero loss.
• Anomaly Detection: Apply Machine Learning models to identify real-time anomalies.
• Automated Alerting: Trigger instant notifications via Microsoft Teams when thresholds are breached.
• Data Lakehouse Management: Organize data into Bronze, Silver, and Gold tiers using Delta Lake.

Non-Functional Requirements

• Performance: speed and responsiveness via Azure Stream Analytics and serverless Azure Functions.
• Security: protection against unauthorized access enforced by Azure Active Directory and Key Vault.
• Usability: ease of use provided by automated Data Activator triggers and Power BI.
• Reliability: system stability and availability maintained by unified Microsoft Fabric infrastructure.
• Scalability: ability to handle growth via limitless OneLake storage and scalable Synapse engineering.
• Maintainability: ease of updates and fixes using centralized Microsoft Purview governance.
• Portability: ability to run in different environments due to the open Delta storage format.