From an SLCP expert: Quality Assurance with Artificial Intelligence
- Ivan Simsic-Babic, Data Scientist at SLCP
Assessment quality oversight is one of the biggest challenges facing the social auditing industry today. With social compliance assessments providing a snapshot in time of millions of workers in thousands of factories worldwide, it is not realistic to manually analyze all of them in depth.
This leaves the industry heavily relying on complaints and random testing to flag audits that need further review.
The disproportionately large volume of assessments to the quality assurance resources available brings with it an interesting opportunity in the form of Artificial Intelligence. While it’s difficult and often impractical for a human investigator to deduce patterns across tens of thousands of assessments, Artificial Intelligence is designed to work with large quantities of data and can learn the patterns that define them efficiently. This allows investigators to focus their time and effort where it matters most, with AI helping them navigate the scale and complexity of the data.
Here at SLCP, we recognized this opportunity and have spent the last 2.5 years developing a first-of-its kind AI model to help in our quality assurance program. This model has been running successfully since 2024 and has allowed us to turn the overwhelming quantity of assessment information into an advantage, learning patterns that help us spot assessments in need of further investigation and helping us learn about the industry as a whole.
How does it work?
The information in SLCP assessments is hard to summarize with a few charts or figures unlike, e.g., financial assessments, where an investigator can see at a glance whether the big picture adds up before analyzing details.
As we record specific information ranging from the number of buildings at a factory, to the labeling of hazardous chemicals and the presence of underage workers, it is hard to get a view of the general quality of an assessment at a glance. Even small errors could misrepresent critical information and hide serious risks for workers. Having a method to analyze all incoming assessments and highlight those at highest risk of quality issues is groundbreaking. It allows us to direct attention where it is most necessary and apply the same standard to every assessment, no matter where they were conducted or by whom.
“This model has been running successfully since 2024 and has allowed us to turn the overwhelming quantity of assessment information into an advantage, learning patterns that help us spot assessments in need of further investigation and helping us learn about the industry as a whole. ”
The AI model that we have developed to support our quality assurance program consists of three separate models that are trained on different aspects of the assessments, and each provides their own analysis of quality, ending up with a balanced and comprehensive metric:
One model analyzes questions individually,
one analyzes the assessments in their entirety, and
one scores assessments within clusters of similar factories.
The models learn from existing assessment data, which is important because they essentially learn to understand and predict the way SLCP approved Verifiers typically perform the assessment verifications on-site. By understanding the patterns that the Verifiers define through their work, we can highlight those verifications that fall outside those patterns to the highest degree, meaning we use the Verifiers’ behavior to effectively define their own standard. This keeps the model purely objective, as it doesn’t learn to prioritize certain countries or kinds of factories, instead learning how the Verifiers do their job and what factors might indicate a verification of poor quality.
Why is it useful?
When the quality assurance model runs on a new batch of assessments, every assessment gets assigned a score from each of the three models, which are then put together to give a final ranking. This helps identify which assessments are most likely to contain quality issues. The model then generates a report for each of these assessments which gives an overview of each assessment, and details the reasons it was selected, highlighting the questions that have the highest likelihood of containing some issue and providing general details that help our quality reviewers understand the assessment. This allows our Data Quality and Integrity team to focus on the most highly ranked reports and confirm the presence of issues. By using the model as a supporting tool for our quality reviewers, we significantly cut down the amount of time and resources necessary to find quality issues in assessments.
Nearly every assessment identified by the model has resulted in corrective action requests, helping us spot issues in a highly efficient and effective way. Also, the ability of the model to analyze large-scale patterns helps us gain insights about the global supply chain industry itself, highlighting connections between issues that previously seemed unconnected and now serve as indicators of quality.
Conclusion
The global supply chain industry concerns many millions of workers worldwide who supply most of the world with its daily needs. Despite this, the working conditions in these places are often poorly understood and recorded as a result of their international nature and the quantity of data involved. By applying modern techniques such as AI, we turn this huge amount of data into an advantage by letting it speak for itself and leading us to new insights about the industry that were previously not possible to achieve. Our model is the first of its kind - quality assurance in social/labor assessments is a challenging field because of the variety and quantity of data. At SLCP, we believe we’ve solved a part of this challenge by designing new technology that is specifically tailored to social/labor audits. We will continue to implement and refine our model as part of our ongoing efforts to ensure high quality social and labor data for our customers.
———
Ivan is a data scientist with a specialization in Artificial Intelligence. He has experience in Machine Learning engineering and data analysis. At SLCP, Ivan is working on predictive analytics to identify quality and performance issues and create efficiencies in the data quality and integrity program.
