Skip to main content

How to Manage Data Quality

Flag invalid sensor readings to maintain data integrity for compliance reporting, ESG audits, and accurate analytics.

Quick Summary

View sensor data history, identify suspicious readings, flag them as Invalid with a reason, then use Valid Only filter when exporting for reports.

Before You Begin​

Requirements
  • Device must have historical sensor data
  • You need permission to manage IoT devices
  • Understanding of normal sensor reading ranges

Why Data Quality Matters​

IoT sensors occasionally produce erroneous readings due to:

CauseExample
Sensor malfunctionTemporary hardware glitch
Maintenance activitySensor disconnected during calibration
Environmental interferenceExtreme conditions affecting readings
Communication errorsCorrupted data during transmission
Power issuesLow battery causing erratic behavior

Bad data in reports can lead to:

  • Incorrect compliance filings (ESG, GRI)
  • False alarms and alert fatigue
  • Misleading trend analysis
  • Poor operational decisions

Viewing Data Quality​

Step 1: Open the Device​

  1. Click the Settings gear icon in the sidebar
  2. Select IoT Management from the dropdown
  3. Click on the device name to open details
  4. Navigate to the Data Overview tab

Step 2: Review Property History​

  1. Find the Property History section
  2. Select the property to review (e.g., Temperature)
  3. Choose a time range from the dropdown:
    • Last 24 Hours (default)
    • Last 7 Days
    • This Week / Last Week
    • This Month / Last Month
    • Last Quarter
    • Custom Range (up to 120 days)

Step 3: Identify Anomalies​

Look for readings that:

  • Spike or drop unexpectedly
  • Fall outside normal operating range
  • Occur during known maintenance windows
  • Don't match patterns from similar sensors

Flagging Invalid Data​

Step 1: Locate the Reading​

  1. In Property History, hover over the chart
  2. Find the suspicious data point
  3. Note the timestamp and value

Step 2: Flag as Invalid​

  1. Click on the data point
  2. Select Flag as Invalid
  3. Enter a reason for flagging:
    • "Sensor glitch during maintenance"
    • "Erroneous spike - sensor malfunction"
    • "Calibration in progress"
    • "Battery replacement caused false reading"
  4. Confirm the flag

Step 3: Verify the Flag​

Flagged readings:

  • Appear marked in the chart
  • Are excluded when using Valid Only filter
  • Include the reason in exports

Filtering Data by Quality​

Use data quality filters to control what data you see and export.

Available Filters​

FilterShowsBest For
All DataEverything, valid and invalidComplete audit trail
Valid OnlyExcludes flagged readingsReports and analytics
Invalid OnlyOnly flagged dataQuality review and correction

Applying Filters​

  1. In Property History, find the filter dropdown
  2. Select Valid Only for clean data
  3. The chart and export will exclude flagged readings

Exporting Quality-Filtered Data​

Step 1: Set Your Filters​

  1. Choose the time range
  2. Select Valid Only to exclude bad data
  3. Select the properties to include

Step 2: Export CSV​

  1. Click Export CSV
  2. File downloads with columns:
    • Timestamp
    • Property name
    • Value
    • Unit
    • Data quality flag

Step 3: Use in Reports​

The exported CSV contains only validated data, ready for:

  • ESG and sustainability reports
  • GRI compliance filings
  • Energy audits
  • Regulatory submissions

Real-World Examples​

Example 1: Clean Data for ESG Report​

Situation: Your sustainability team needs quarterly energy data for the company's ESG report. Some readings during a power outage show zero consumption incorrectly.

Solution:

  1. Click the Settings gear → IoT Management
  2. Open the building's energy meter
  3. Navigate to Data Overview
  4. Set time range to Last Quarter
  5. Review the Property History chart
  6. Find the zero readings during the outage date
  7. Flag each as Invalid: "Power outage - meter offline"
  8. Set filter to Valid Only
  9. Export CSV
  10. Provide to sustainability team

Result: The ESG report contains accurate energy consumption without the false zero readings during the outage.

Example 2: Identify Sensor Malfunction Pattern​

Situation: A cold storage temperature sensor occasionally reports -999°C, which triggers false critical alerts.

Solution:

  1. Go to Settings gear → IoT Management and find the sensor
  2. Open device details → Data Overview
  3. Set time range to Last Month
  4. Filter by Invalid Only to see previously flagged data
  5. Note the pattern — spikes occur every few days
  6. Flag all -999°C readings as Invalid: "Sensor malfunction - hardware issue"
  7. Create a work order to replace the sensor
  8. Meanwhile, use Valid Only filter for dashboards

Result: False alerts stop appearing in reports while maintenance schedules the sensor replacement.

Example 3: Pre-Audit Data Cleanup​

Situation: An energy auditor is visiting next week. You need to ensure all meter data is accurate and any known issues are documented.

Solution:

  1. Click the Settings gear → IoT Management
  2. Open each energy meter one by one
  3. Review Property History for the audit period
  4. Flag any anomalies with clear reasons:
    • "Meter calibration 2024-03-15"
    • "Building evacuation drill - HVAC shutdown"
    • "Sensor replaced - gap in readings"
  5. Export data with Valid Only filter
  6. Keep a copy with All Data for audit trail

Result: Clean data ready for the auditor, with documented explanations for any flagged readings.

Example 4: Humidity Sensor Calibration​

Situation: After calibrating a humidity sensor, it reported 100% for two hours before stabilizing. These readings affect your monthly average calculations.

Solution:

  1. Go to Settings gear → IoT Management and open the humidity sensor
  2. Go to Data Overview
  3. Find readings from the calibration date
  4. Select the 2-hour window of 100% readings
  5. Flag as Invalid: "Post-calibration stabilization period"
  6. Verify the readings after that period look normal
  7. Use Valid Only for any reports covering this period

Result: Monthly humidity averages now reflect actual conditions without the calibration anomaly.

Data Quality Best Practices​

Regular Reviews​

FrequencyAction
DailyCheck for offline devices and recent anomalies
WeeklyReview flagged data and validate reasons
MonthlyAudit data quality across all sensors
QuarterlyExport and archive validated data for compliance

Documentation Standards​

When flagging data, include:

  • What happened: "Sensor reported -40°C"
  • Why it's invalid: "Sensor was disconnected for maintenance"
  • When it occurred: "During scheduled HVAC maintenance 2pm-4pm"

Quality Indicators to Monitor​

IndicatorHealthyInvestigate
Device uptime>99%<95%
Flagged readings<1% of total>5% of total
Alert frequencyWithin expected rangeSudden increase
Data gapsRareFrequent

Troubleshooting​

Too Many Invalid Readings​

SymptomPossible CauseSolution
Frequent flagging neededFailing sensorSchedule replacement
Pattern of bad dataEnvironmental issueRelocate sensor
Intermittent errorsPower or connectivityCheck installation

Data Not Appearing in Reports​

IssueCauseSolution
Zero readings shownFilter set to Valid OnlyCheck if data was flagged
Gaps in timelineDevice offlineReview connection status
Wrong time rangeDate filter incorrectAdjust start/end dates

Cannot Flag Data​

IssueCauseSolution
Flag option not availableInsufficient permissionsContact administrator
Historical data missingBeyond retention periodData automatically deleted

Data Retention​

Data is available based on your plan:

PlanRetention Period
Lite7 days
Basic30 days
Professional90 days
Enterprise365 days
Export Before Expiry

Data beyond your retention period is automatically deleted. Export important data regularly to maintain long-term records.

Integration with Dashboards​

Data quality flags affect dashboard widgets:

Widget TypeBehavior
Current ValueShows latest valid reading
Trend ChartsCan filter by data quality
AveragesCalculate from valid data only
AlertsTrigger on valid readings only

Need help? Contact Infodeck Support

Was this page helpful?