Data Validation. Simplified.
My Role: Co-founder, Principal Product Designer
As a founding member of a product startup, I and my founding team built a SaaS data validation tool from ground up. The tool delivers efficiency and cost saving to BI teams and data testing teams. The product has a proven track record of saving up to 60% of time in the data verification process.
User Research
​The purpose of this user research study is to understand the user needs and preferences for a self-serve data validation tool. The goal of the study is to identify pain points and opportunities for improvement in the current tool and to inform the design of a new tool that will better meet the needs of users.
Methodology
The study was conducted using a combination of qualitative and quantitative research methods, including surveys, interviews, and usability testing. Participants were recruited from a pool of existing users of the current tool, as well as potential users who had expressed interest in such a tool. The study was conducted over a period of three weeks, with participants engaging in various activities and providing feedback at each stage.
Findings
The findings of the study revealed several key insights into the user needs and preferences for a self-serve data validation tool:
-
Time-Saving: The majority of participants expressed a desire for a tool that would save them time in the data validation process. They were looking for a tool that could automate the process and minimize the need for manual validation.
-
User-Friendly: Participants wanted a tool that was easy to use, with a simple and intuitive interface that could be easily navigated. They preferred a tool that did not require extensive training or technical knowledge.
-
Customization: Users expressed a desire for a tool that would allow them to customize the validation rules to meet their specific needs. They wanted a tool that was flexible and adaptable to different data sets.
-
Feedback: Participants wanted clear and concise feedback on the validation results, with easy-to-understand error messages that would help them quickly identify and resolve issues.
-
Collaboration: Some users expressed a desire for collaboration features, such as the ability to share validation rules with team members and to collaborate on the validation process.
Conclusion
Based on the findings of the study, we recommend the following design features for the new self-serve data validation tool:
-
Automation of the validation process to save time
-
Simple and intuitive interface with easy navigation
-
Customization of validation rules to meet user needs
-
Clear and concise feedback on validation results
-
Collaboration features to support team-based validation
By incorporating these features, we believe that the new tool will better meet the needs and preferences of users, resulting in a more efficient and effective data validation process.
Problem Statement
Organizations rely on data to make critical business decisions, but the accuracy and reliability of that data are often compromised by errors and inconsistencies. The process of validating data can be time-consuming and complex, requiring specialized technical expertise and resources. As a result, organizations may struggle to ensure the quality of their data, leading to costly mistakes and missed opportunities.
A self-serve data validation tool can address this problem by empowering users to validate their own data without the need for specialized technical expertise, saving time and resources, and improving the accuracy and reliability of data used in decision-making. However, current self-serve data validation tools may not meet the needs and preferences of users, resulting in low adoption and limited effectiveness. Therefore, the solution is to design and develop a self-serve data validation tool that is user-friendly, customizable, and provides clear feedback to ensure accurate and reliable data validation.
User Groups
-
Business Analysts
-
Data Scientists
-
Data Quality Testing teams
-
Business Intelligence Experts
User Persona 1
Name: Sarah Lee
Age: 32
Occupation: Business Analyst
Education: Bachelor's degree in Business Administration
​
Background: Sarah works for a large retail company that has multiple data sources for sales, inventory, and customer information. She is responsible for analyzing this data to identify trends and make recommendations to help the company optimize its operations. Sarah spends a lot of time cleaning and validating the data to ensure that her analysis is accurate and reliable.
​
Goals:
-
To find a data validation tool that can automate the process of cleaning and validating data to save time and increase efficiency
-
To improve the accuracy and reliability of her data analysis to make more informed recommendations to the company
​
Challenges:
-
Dealing with large volumes of data from multiple sources
-
Ensuring the accuracy and reliability of the data to avoid making incorrect recommendations
-
Time-consuming process of manually cleaning and validating data
​
​
Sarah is a busy business analyst who needs to process large volumes of data on a regular basis. She is proficient in using Excel and other data analysis tools, but finds the process of manually cleaning and validating data to be time-consuming and tedious. Sarah is looking for a data validation tool that can automate this process and ensure the accuracy and reliability of her analysis. She is willing to invest in a tool that can save her time and increase efficiency, as well as provide more accurate results. Sarah is a team player and values collaboration with her colleagues to ensure that the company's goals are met.
User Persona 2
Name: Michael Chen
Age: 40
Occupation: Business Intelligence Expert
Education: Master's degree in Computer Science
​
Background: Michael works for a large financial institution and is responsible for developing and maintaining the company's business intelligence reporting. He collects data from various sources, analyzes it, and creates visual reports that are used by senior management to make strategic decisions. Michael's role is critical to the success of the company, as any inaccuracies in his reporting could have significant financial consequences.
​
Goals:
-
To find a data validation tool that can ensure the accuracy of his reporting and reduce the risk of errors
-
To improve the efficiency of the data validation process, allowing him to spend more time on analysis and reporting
-
To maintain the highest standards of data integrity and compliance with regulatory requirements
​
Challenges:
-
Managing large volumes of complex data from multiple sources
-
Ensuring the accuracy and completeness of the data used in reporting
-
Meeting tight deadlines for reporting while maintaining high standards of quality
​
Michael is a highly skilled and experienced business intelligence expert who is constantly seeking ways to improve the accuracy and efficiency of his reporting. He is detail-oriented and meticulous in his work, with a strong focus on data integrity and compliance. Michael is an expert in data analysis and reporting tools, but recognizes that a data validation tool can save him significant time and reduce the risk of errors in his reporting. He is willing to invest in a tool that meets his high standards and supports his role in the company. Michael is a team player and values collaboration with his colleagues to ensure that the company's reporting is accurate and reliable.
Initial Concepts
Design Explorations
Outcome
A self-serve data validation tool that revolves around the user behaviour and need.
Smart rule engine and its ease of use empowers the users with 100% flexibility in managing the business rules.
An intuitive design that drastically increases user engagement.
Sneak peek of the final product
User Testing
During the user testing phase for our self-serve data validation tool, we conducted extensive sessions with a diverse group of users. These tests aimed to evaluate the tool's usability, effectiveness, and overall user experience. We observed users navigating the tool, completing tasks, and providing valuable feedback. The insights gained from these tests allowed us to identify pain points, validate design decisions, and make data-driven improvements. User testing played a crucial role in refining our data validation tool to ensure it meets the needs of our users, promotes self-serve capabilities, and enhances collaboration throughout the data validation process.
That unique thing we did
Our goal when developing this tool was to simplify the process of writing and managing business rules for both Data Scientists and Business Analysts. To achieve this, we prioritized making the tool user-friendly and accessible to a wide audience. To ensure ease of use for all users, we even tested the basic business rules page with 8th graders, incorporating their mental model into each iteration of the tool's design.
​
The outcome of our efforts was phenomenal. Our tests were a resounding success, and we found that our primary user groups were able to use the tool without requiring any training. The tool's language speaks directly to users in a way they understand, driving greater efficiency, user adoption, and retention.