Investigating lost surveys

Guide for Preventing and Investigating Lost Surveys

Rule #1 at mWater is Never Lose Data. We take your data collection activity seriously. Therefore we have built the platform with multiple failsafes to prevent the loss of data. As of this writing, mWater users have submitted over 10 million surveys, averaging 400,000 per month. Of this vast amount of data collected, there has not been a loss of a survey due to a technological error in three years. Over time, we have only ever recorded two lost survey events that were from a technological error. Virtually all survey loss events are from user error. And yet, we so value your data that we investigate every claim of lost data with 100% of our effort.

Since user error is the cause of almost all survey loss events, this guide is written to prevent this source of data loss. It is a detailed approach to rigorous and disciplined management of enumerations. Following this guide will result in cleaner data collection, less noise in your results, and most of all, preventing data loss.

Investigating Lost Surveys

The top risk to survey data is unsynced data. If the device is lost, broken, stolen, or suffers a catastrophic software error, the unsynced data cannot be saved in any way. mWater tries to sync your data every few seconds. If there is no signal from wifi or mobile data sim, mWater stores the data offline until it can sync. 

Train to Sync Often. When training your enumerators, spend a lot of time on the importance of syncing data at least twice a day. It is best to provide an active sim card for each user so that they can always sync. While the expense of this prevents many managers from doing this, comparing the expense of lost data for a day’s work to the sim usually shows a cost-benefit relationship that favors a sim on every device. Re-collecting lost data introduces a bias into your survey data that cannot be tracked or resolved.
Train your enumerators to look for “three green syncs” on the app homepage. The device is only synced when all three of these are green. To read more about the types of sync and what they are used for, read this guide. The guide also has advice on compressing your photos if the image sync is having more trouble than the others. If your enumerators tell you they are synced but you suspect missing data, ask them to text or email you a screenshot of the homepage and the tasks page to verify this sync status. 
If you verify all three green syncs, ask your enumerator if they have used any other device since the work began. They then need to check the sync status on all devices they’ve logged into. 
Ask your enumerator if they have shared this device with another user who logged in with their own account? Every user account needs its own sync. 
If surveys are still missing, it may be that the enumerator deleted them from the app before they were submitted and before the sync event. Ask your enumerator if they have deleted any drafts. Sometimes they do this in a self-cleaning activity if they see cluttered drafts. 
Once you have undergone all of these steps on your side, if you still have missing surveys, you need to involve mWater Staff. Ask the user to submit a problem report from the device they were using to collect their data. This is done in the Settings tab of the app. 
In their problem report, they need to be as detailed as possible. An ideal problem report has three steps: 
  1. What was the user doing before the problem happened? 
  2. What were they trying to do?
  3. What happened instead? In the case of missing surveys, the name of the survey and the contact information of their administrator is also needed. Then, they should check their email, where mWater Staff will contact them to help resolve the issue. 
In the case we can detect deleted surveys, we can often retrieve them for a fee if the user synced before deleting.

Preventing Lost Surveys

Best Practices for User Accounts

Every enumerator should have a unique user account. Sharing accounts is a major way data is confused and lost. In mWater, there is no cost for each account so there is no reason to skip this important step. 

User account names should reflect the enumerator's real name. This ensures the data is trackable and accountable to the user. Do not create a series of accounts that have generic names such as, “Enumerator #31,” or, “WaterHelpers 5.” 
Manage your enumerators and their permissions with the Organization Chart feature in the mWater Portal. When you deploy surveys through the Org Chart, it is easy to keep track with turnover through the branches of the chart. Updating your branches will automatically update all other permissions so long as you are deploying from the Org Chart view, as opposed to the Deploy view in the Survey editor. For more information on using Organization Charts, read this guide and watch this video. 
If the information for your enumerators needs to be updated to comply with this guide for any reason, users can change their email address, username, and personal information in the Surveyor app by:
  1. Clicking the Settings tab
  2. Clicking My Account
  3. Changing any of the user fields
Or via the Portal by:
  1. Clicking their username in the top right corner
  2. Selecting My Account
  3. Changing any of the user fields

Using Multiple Devices per User or Sharing Devices

Device Management. One common way that users feel they have lost surveys is if they see three green syncs on their device, forgetting that they used more than one device in data capture. If a user begins a workday on their smartphone and finishes their workday with surveys on another phone or their laptop, they create a situation where some data is attached to their account on one device and some data is attached to their account on another device. Seeing the three green syncs on one device gives them the impression they are fully synced, when in fact the data is not synced on one of their devices. 

Shared Devices. The app only syncs when it is open, never in the background while another app is in use. It also does not sync when the user is logged out. If two users are sharing one smartphone, tablet, or laptop, swapping between each other’s accounts, the device will only sync the open app logged into one account. This situation can result in users reporting three green syncs for their app when in fact, one app has not synced in the background at all. 

Multiple accounts for one enumerator. Occasionally, an enumerator will accidentally create two user accounts. This is not recommended because it often results in one of the accounts not syncing because the user is signed in with the other account. Before submitting a problem report please check to make sure the user is signed in under the correct account. If a user has mistakenly created multiple accounts, they can delete one of them via the My Account page of the mWater Portal. Please make sure to sync all data before deleting an account.

Do not update the App from the App Store without Syncs

If there is a problem with the app and the app has not been synced, updating the app from the Google Play or iOS Store will result in permanently lost data. 
Note: This is different than the in-app update mWater often offers with a restart now button. The in-app update will not lose data. If you have a catastrophic problem with the app and have data stuck on it, please email info@mwater.co for help. 
Last updated: 10/2021