In many projects, “Supplier Quality Surveillance (SQS)” still means chasing NCRs and reacting to problems. Teams do inspections, write reports and close punch lists – but management rarely gets a clear, numeric picture of which vendors are really performing well.
This guide shows you how to:
- Understand SQS as more than just inspections
- Pick 5 categories of KPIs that actually matter
- Combine them into a Supplier Quality Score (SQS)
- Build a practical vendor scorecard for QA/QC, procurement and project management
Quick Answer – What Is SQS and Which KPIs Really Matter?
In this article we use two related terms:
- Supplier Quality Surveillance (SQS) – activities used to control and monitor supplier quality: audits, ITP review, shop inspections, NCR follow-up and dossier review.
- Supplier Quality Score (SQS) – a composite score built from a small set of KPIs that summarises a supplier’s quality performance over a period (for example, the last 12 months).
Instead of tracking 25 random metrics, focus on five KPI categories:
- Quality performance – defects, NCRs, right-first-time
- Delivery & schedule – on-time delivery and inspection readiness
- Documentation & traceability – certificates, MTRs, nameplates, heat numbers
- Responsiveness & corrective action – how fast and how effectively the supplier responds
- Risk, HSSE & audits – underlying risk level and behaviour of the supplier
With 2–3 well-designed KPIs in each category, you can calculate a clear Supplier Quality Score (SQS) and use it in vendor scorecards, supplier review meetings and allocation of inspection effort.
SQS in Industrial Projects – From Surveillance to Score
Supplier Quality Surveillance (SQS) in EPC / Oil & Gas / Power
In industrial projects, SQS typically includes:
- Supplier qualification and audits
- Review and approval of ITPs and QCPs
- Shop inspections and FATs
- Review of MTRs, certificates and test reports
- Handling of IRs and NCRs
- Final dossier / data book review
All these activities generate data: NCR counts, inspection dates, certificate issues, audit findings and so on. Too often, this data stays buried in PDFs and spreadsheets.
Turning SQS into a KPI-driven process means:
- Standardising how you capture inspection and NCR data
- Selecting a small number of meaningful KPIs
- Using those KPIs to steer supplier development and inspection planning – not just to report history
If you need a refresher on the underlying activities and roles, see
What Is Vendor Inspection? Roles & Responsibilities.
Supplier Quality Score (SQS) as a Composite KPI
A Supplier Quality Score (SQS) is a single number (for example 0–100) that summarises supplier performance over a defined period. Many organisations call this a “Supplier Quality Index” or “Vendor Rating”. The idea is always the same:
- Combine 5–10 KPIs into 5 categories
- Reflect both short-term performance (defects, OTD) and structural risk (audits, HSSE)
- Calculate the score consistently across suppliers, projects and years
Typical stakeholders:
- QA/QC & SQS teams – decide where to focus inspection man-days
- Procurement – decide which suppliers are preferred or at risk
- Project management – see quality risk at a glance when selecting vendors
The 5 KPI Categories for a Strong Vendor Scorecard
Instead of building a scorecard with 20+ metrics, build around these five buckets and pick 2–3 KPIs in each.
1) Quality Performance KPIs
This is the core of supplier quality.
Useful KPIs for industrial vendor inspection:
- NCR rate per PO / per value
Number of nonconformity reports (NCRs) per purchase order or per spend. Shows how often you need to formally raise quality issues.
- Repeat NCR rate
Percentage of NCRs that relate to failure modes already seen on previous orders. A high repeat rate indicates weak root cause analysis and corrective action.
- Right-First-Time (RFT) / First Pass Yield
Percentage of lots or inspections accepted without rework or retest. Captures “hidden” quality issues even when no formal NCR is raised.
- Defect rate (PPM or % accepted lots)
Especially useful for repetitive components or high-volume items.
These KPIs depend on consistent IR and NCR logging, aligned with
Vendor Inspection Reporting: IR, NCR & Final Dossier — Best Practices & Templates.
2) Delivery & Schedule KPIs
Almost every supplier scorecard includes On-Time Delivery (OTD), but for SQS you can refine it.
Good KPIs:
- On-Time Delivery (OTD)
Percentage of shipments delivered within the agreed window (for example –5 to +3 days from contractual delivery date).
- On-Time Inspection Readiness
Percentage of planned inspections (FAT, stage inspections) where the supplier is actually ready at the agreed date (ITP milestone readiness). Late readiness consumes SQS man-days and can push schedule.
- On-Time Document Submission
Percentage of ITPs, QCPs, procedures, MTRs and test reports submitted by the due date.
These KPIs are easier to measure when you plan your work using the
Vendor Inspection ITP Template — Free Excel & Guide.
3) Documentation & Traceability KPIs
In regulated industries, the quality of documentation and traceability is as important as physical quality.
Practical KPIs:
- Certificate completeness & correctness
Percentage of deliveries with complete and correct EN 10204 3.1/3.2 certificates, matching the PO and drawing. The article
EN 10204 3.1 vs 3.2: Differences Explained (Free Checklist)
defines the basics of what “complete and correct” means.
- Nameplate & marking conformity
Percentage of valves/equipment where nameplates and body markings fully match the datasheet, certificates and standards.
- Heat-number & certificate matching accuracy
Percentage of items where heat numbers on the parts match the MTC and traceability records. The
MTC Interpretation Guide + Checklist (PDF)
is a good reference for what to look for in MTCs.
These KPIs push suppliers to improve internal traceability and document control, not just visual and dimensional quality.
4) Responsiveness & Corrective Action KPIs
Quality problems will never be zero. The key question is how suppliers react when problems appear.
Useful KPIs:
- Average response time to NCR / SCAR
Time from NCR/SCAR issuance to initial response (acknowledgement and containment actions).
- Corrective action closure time
Time from NCR/SCAR issuance to final closure (including implemented actions and verification of effectiveness).
- Reduction in repeat nonconformities
Year-on-year reduction in recurring issues attributed to the same root causes.
These KPIs show how seriously a supplier treats your feedback. They rely on clear reporting processes as described in
Vendor Inspection Reporting: IR, NCR & Final Dossier — Best Practices & Templates.
5) Risk, HSSE & Audit KPIs
Many large operators use Supplier Qualification Systems with risk ratings based on product criticality, HSSE performance, financials and quality systems.
You can integrate this into your SQS scorecard with:
- HSSE performance indicator
High-level metrics of recordable incidents, lost-time injuries or serious near misses at supplier facilities, where data is available.
- Supplier audit score & closure rate
Latest audit score (0–100 or A/B/C) and the percentage of audit findings closed by the due date.
- Supplier risk rating
Qualitative rating (for example High / Medium / Low) derived from your internal qualification system, considering criticality, complexity and HSSE.
These metrics help you decide how much surveillance is appropriate: more inspection man-days for high-risk suppliers or packages, less for low-risk ones.
How to Calculate a Supplier Quality Score (SQS) from These KPIs
Once you have selected your KPIs, you can combine them into one Supplier Quality Score (SQS) that summarises performance for each supplier and period.
Example Weighting Model for Industrial Suppliers
A simple weighting model for capital equipment and engineered items might look like this:
| Category |
Example KPIs |
Weight |
| Quality performance |
NCR rate, RFT, repeat NCRs |
40% |
| Delivery & schedule |
OTD, inspection readiness, document timing |
25% |
| Documentation & traceability |
Certificate/nameplate conformity, traceability |
15% |
| Responsiveness & corrective action |
NCR/SCAR response & closure, repeat reduction |
10% |
| Risk, HSSE & audits |
HSSE indicator, audit score, risk rating |
10% |
You can adjust these weights based on your sector and risk profile. Safety-critical projects might assign more weight to HSSE & risk.
Step-by-Step SQS Calculation (Simple Example)
- Normalise each KPI to a 0–100 score
- For “higher is better” metrics (e.g. OTD), use the percentage directly.
- For “lower is better” metrics (e.g. NCR rate), invert them into a 0–100 scale (for example, define internal bands where zero NCRs get 100 points, and high NCRs get lower scores).
- Calculate a category score
- If you have three KPIs in “Quality performance”, average them to get one quality score out of 100.
- Apply weights
- Multiply each category score by its weight (as a fraction).
- Example: Quality score 80 × 0.40 = 32 points.
- Add them up to get SQS
- Sum all weighted category scores to get a final Supplier Quality Score between 0 and 100.
- Repeat per period
- Calculate SQS per supplier per year (and optionally per project) to see trends and compare vendors.
This gives management a simple, comparable number while preserving the detail behind each KPI for root-cause analysis and supplier development.
Example Vendor Scorecard – How It Looks in Practice
Sample Scorecard Layout
A basic supplier scorecard can be implemented in Excel or a BI tool. A simple layout for annual SQS could look like this:
| Supplier |
Quality (40%) |
Delivery (25%) |
Documentation (15%) |
Responsiveness (10%) |
HSSE/Risk (10%) |
SQS (0–100) |
Trend vs last year |
| Vendor A |
82 |
90 |
95 |
85 |
80 |
86.6 |
+4.2 |
| Vendor B |
60 |
70 |
75 |
55 |
70 |
65.5 |
–3.0 |
| Vendor C |
92 |
88 |
90 |
92 |
85 |
90.0 |
+1.5 |
You can add colour coding (green / yellow / red) to highlight preferred, development and at-risk suppliers.
Interpreting the Scorecard
When you review the scorecard:
- QA/QC & SQS can focus limited inspection man-days on suppliers with lower SQS or sharply negative trends.
- Procurement can factor SQS into sourcing decisions, frame agreements and allocation of orders.
- Project management can identify suppliers that need closer monitoring on critical packages.
A natural place to host and standardise such a scorecard is alongside other tools in the
Master QA/QC Templates Pack (ITP, Checklists & Forms).
Designing SQS KPIs & Vendor Scorecards – Step-by-Step
Step 1 – Segment Suppliers by Criticality
Not all suppliers need the same level of KPI tracking. Start by segmenting:
- Critical suppliers – safety-critical, long lead time, high impact on project schedule
- Standard suppliers – important but with manageable risk
- Low-risk suppliers – non-critical, easily replaceable items
For critical suppliers, track the full set of KPIs and update SQS regularly. For low-risk suppliers, you may only track basic quality and delivery.
Step 2 – Select 5–10 KPIs That Actually Matter
Avoid KPI overload. Use the five categories above and pick:
- 2–3 KPIs per category for critical suppliers
- 1–2 KPIs per category for standard suppliers
Make sure each KPI:
- Is clearly defined (formula, units, period)
- Has a clear link to risk or cost
- Has a realistic target
Step 3 – Define Targets, Data Sources and Owners
For each KPI, document:
- Target level – for example “OTD ≥ 95%”, “≤ 2 NCRs per €1 million spend”
- Data source – NCR system, ERP, inspection reports, vendor portal, HSSE reports
- Owner – who is responsible for maintaining the data (QA, procurement, SQS team)
Good IR and NCR practices from
Vendor Inspection Reporting: IR, NCR & Final Dossier — Best Practices & Templates
make this step much easier.
Step 4 – Build and Maintain the Scorecard
Start simple:
- Build the first version in Excel using the table format above
- Calculate SQS quarterly or yearly
- Share draft scorecards internally for feedback before using them with suppliers
Later you can move to a BI tool or integrate the scorecard into your supplier portal. The principles remain the same: few KPIs, clear weights, consistent data.
Step 5 – Use the Scorecard in Real Decisions
The scorecard should influence:
- Vendor selection and allocation – for similar prices, prefer suppliers with higher SQS
- Inspection planning – allocate more inspection days and surveillance to low-SQS or high-risk suppliers
- Vendor development plans – for key suppliers, agree improvement actions based on weak categories (for example documentation or responsiveness)
Without this last step, KPIs stay as “nice reporting” instead of becoming a management tool.
Common Mistakes with SQS KPIs & Vendor Scorecards (And How to Avoid Them)
Too Many KPIs, No Clear Story
One of the most common problems in supplier scorecards is overloading with KPIs – tracking every possible metric so no one knows what actually matters.
Focus on:
- 8–12 KPIs across the five categories
- One composite SQS
- A short narrative explaining the key movements
KPIs That Do Not Link to Risk or Business Impact
Avoid metrics that are easy to measure but do not change decisions (for example, counting emails). Instead, choose KPIs that:
- Reflect risk (NCR rates, HSSE incidents, audit scores)
- Reflect cost (defects, rework, delays)
- Reflect client satisfaction (documentation quality, compliance, on-time delivery)
Poor Data Quality (NCRs & IRs Not Consistently Logged)
If inspectors and project teams do not log NCRs, IRs and document issues consistently, KPIs will be misleading. Improve:
- Templates and training for IRs/NCRs
- A culture where minor issues are recorded, not ignored
- Periodic checks on data quality
Scorecard Used Only Internally, Not Shared with Suppliers
A scorecard that is never shared with suppliers is a missed opportunity.
To avoid this:
- Share the SQS logic and weights with key suppliers
- Review their score and trends in periodic meetings
- Use the scorecard as a starting point for joint improvement plans
How SQS KPIs Connect to the NTIA Vendor Inspection Toolkit
SQS KPIs do not exist in isolation. They sit on top of the processes and tools you already use in vendor inspection:
With these building blocks in place, SQS KPIs become a natural extension of your existing vendor inspection practice, not an extra admin task.
Use SQS KPIs to Steer Vendor Inspection, Not Just Measure It
SQS teams, QA/QC engineers and procurement specialists already collect a huge amount of data: NCRs, inspection reports, certificates, audits, HSSE incidents. The value of that data appears when you:
- Turn it into a small set of clear KPIs
- Combine them into a Supplier Quality Score (SQS)
- Use that score to steer inspection effort, sourcing decisions and supplier development
You can start small:
- Pick 2–3 critical suppliers.
- Implement the five KPI categories on a simple Excel scorecard.
- Calculate SQS for the last 12 months.
- Share and discuss results internally and with the suppliers.
As you refine the model, you can integrate it with templates from the
Master QA/QC Templates Pack (ITP, Checklists & Forms)
and reinforce it through training such as Industrial Valve Inspection & Testing Training, so that both your technical practices and your performance management move in the same direction.
Over time, SQS KPIs and vendor scorecards will help you move from reactive inspection and firefighting to planned, risk-based supplier quality surveillance that visibly protects project quality, schedule and budget.
