Non-Metallic Materials Testing (NMMT) Audit Insights

Article Links

• Interlaboratory Proficiency Test (IPT)
• Common Non-conformances in NMMT Audits
• Initial Audit AC7122-I Top NCRs
• Reaccreditation Audit AC7122-I Top NCRs

 

---

 

Full Article 

The Nadcap Non-Metallic Materials Testing (NMMT) Task Group conducted its first audit in 2006 and is currently led by Chairperson Tara Campbell of Rolls-Royce, and Vice Chairperson, Randy Armstrong of Raytheon. The NMMT Task Group audits aerospace laboratories conducting non-metallic material testing, such as composites, plastics, elastomers, and more. The Task Group is comprised of 65 industry representatives – 45 Nadcap Subscriber representatives from 25 companies and 20 Supplier representatives from 14 companies who actively participate in the technical discussions and decision making. These discussions mostly take place at the three Nadcap meetings held around the globe each year. The Task Group is fully aware that not all industry stakeholders are able to participate and benefit from the opportunities that the meetings represent, such as learning, debating and networking. Consequently, this article is intended to assist to some degree, by providing insights and sharing lessons learned regarding the Nadcap NMMT audit experience. The Nadcap NMMT Task Group differentiates between two types of laboratories, which are defined below:

• Captive Laboratory: a laboratory that belongs to a material, parts or subassembly Supplier, with systems that are dependent on those of the Supplier, and with testing capabilities that are limited to those required by the Supplier’s material. A captive laboratory does not accept work from an outside source.
• Independent Laboratory: a laboratory whose systems are not dependent on those of specific material, part or subassembly Supplier (ownership by a material Supplier does not exclude a laboratory from being considered “independent”). An independent laboratory accepts work from an outside source.

It should be noted that captive laboratories may be audited as a part of Nadcap Composites (COMP), Elastomer Seals (SEAL), or Non Metallic Material Manufacturing (NMMM) audits utilizing either the Audit Criteria (AC) AC7122-P (utilized for COMP, SEAL; the “P” stands for “Part Manufacturer”) or AC7122-R (utilized for NMMM; the “R” connotes for “Raw Material Manufacturer”) and the associated slash sheets (AC7122/1-5) for the test methods performed. Stand-alone NMMT audits utilize AC7122-I (the “I” means “Independent”) and the associated slash sheets (AC7122/1-5) for the test methods performed for both captive and independent labs. For captive laboratories, the Nadcap NMMT Task Group recognizes 9100 certifications by Registrars that are approved and listed in the International Aerospace Quality Group (IAQG) OASIS database, which can be found at www.iaqg.org/oasis. The NMMT Task Group also recognizes existing quality systems approvals in the form of 17025 certifications by an International Laboratory Accreditation Cooperation (ILAC) approved source – full list available on www.ilac.org. It is important to note that independent laboratories must have 17025 certification by an International Laboratory Accreditation Cooperation (ILAC) approved source (www.ilac.org). In addition to the baseline checklists, there are separate Audit Criteria and slash sheets for each of the individual processes covered by the NMMT Task Group. While the “generic” AC7122 does exist, it serves primarily as a reference document as it contains no questions and  defines all three variations of AC7122 that are: AC7122-I, AC7122-P, AC7122-R. In addition, AC7122 defines job audit requirements and contains reference information for creating internal proficiency tests. The complete list of NMMT Audit Criteria is as follows and can be found in eAuditNet under Resources / Documents / Audit Criteria / Non Metallic Materials Testing (AC7122):

• • • • • • • • • AC7122 – Defines AC7122-I, P, R, as well as job audit requirements
                • AC7122-I – General requirements for stand-alone audits (captive and independent laboratories)
                • AC7122-P – General requirements for captive laboratories audited in conjunction with AC7118 (COMP) or AC7115 (SEAL) audits
                • AC7122-R – General requirements for captive laboratories audited in conjunction with AC7124 (NMMM) audits
                • AC7122/1 – Mechanical testing
                • AC7122/2 – Physical testing
                • AC7122/3 – Chemical testing
                • AC7122/4 – Thermal testing
                • AC7122/5 – Flammability testing

Additional information on the Audit Criteria and slash sheets requirements, question intent, acceptable objective evidence, and helpful hints are included in the Audit Handbooks that are available in eAuditNet under Resources / Documents / Audit Criteria / Non Metallic Materials Testing / Handbooks & Guides as shown below along with all the Audit Criteria.

 

Interlaboratory Proficiency Testing

In the past, the NMMT Task Group ran an Interlaboratory Proficiency Test (IPT) in which AC7122-I and AC7122-R accredited laboratories were required to participate. As the NMMT Task Group grew bigger, Auditees requested that participation in the third-party proficiency test mandated by Airbus and SAFRAN be allowed to fulfill the AC7122-I and AC7122-R proficiency testing requirement. The Task Group decided to move forward with the request, and the third part proficiency test was allowed as a means of fulfilling the proficiency testing requirement. After seven IPTs, it has become challenging to recruit additional Nadcap Subscribers willing to administer this free program, as well as to enroll companies willing to donate test specimens. Starting in 2021 (when the next IPT would normally occur), Nadcap IPT participation for AC7122-I and AC7122-R accredited labs will be voluntary, assuming that volunteers and donors can be found. Participation in customer-mandated proficiency testing such as that mandated by Airbus and SAFRAN will continue to be verified.

 

Common Non-Conformances in NMMT Audits

In common with many other Nadcap Task Groups, the NMMT Task Group analyzes and publishes common non-conformances (NCRs) identified during Nadcap audits on a regular basis. The intent is to help Auditees avoid some common pitfalls and strengthen their internal process control. The most recent list of common NCRs can be found in eAuditNet, under Resources / Documents / Public Documents / Task Groups / NMMT / Top NCRs as shown.

Given the small number of audits and the low number of NCRs typically identified during each laboratory audit, the data doesn’t provide meaningful results when conducted for each individual checklist. As a result, the NMMT Task Group has chosen to only compile top NCRs for AC7122-I for a 2-year period. This data is then analyzed separately for both Initial and Reaccreditation audits. This makes sense as the majority of NCRs come from AC7122-I, as opposed to the slash sheets.  

 

Initial Audit AC7122-I Top NCRs

• 23% of initial audits included an NCR against paragraph 11.1, “Is there a procedure addressing the calibration and usage of “employee-owned” equipment used for acceptance purposes?“. Typically, laboratories prohibit the use of all employee-owned equipment and no procedure specifically addresses this, although it is required by the Nadcap Audit Criteria.
• 23% of initial audits had an NCR against paragraph 19.4, “Does the laboratory notify all customers to whom the laboratory sent incorrect or suspect test results within five (5) working days, if affected material is not contained and correctness of results verified, unless otherwise specified by the customer?”. This is only a procedural issue, as laboratories are notifying customers in a timely manner, but procedure simply fails to specify that this should occur within five working days.
• 23% of initial audits received an NCR for non-compliance with paragraph 8.1, “Are calibration status labels posted on each calibrated test machine and measuring device?“. Typically, the equipment has been calibrated, but either the label has fallen off or it has not been updated, or the calibration is only tracked via a database.
• 15% of initial audits included an NCR against paragraph 18.1, “Do the laboratory”s procedures concerning replacement testing require that invalid tests, subject to replacement testing, be assigned a probable cause?“. A straightforward non-conformance, the laboratory fails to assign a probable cause when invalidating a test.
• 15% of initial audits saw an NCR written against paragraph 4.4, “If problems are found by internal audit, does the Q.A. function:

1. 1. 1. Use a central log to record the discrepancies and corrective actions
      2. Investigate to determine probable cause
      3. Participate in the development of corrective actions
      4. Control the application of corrective actions”.

This is again a paperwork issue. Quality Assurance (QA) investigates the probable cause, participates in the development of corrective actions, and controls the application of corrective actions. The problem leading to this NCR is a failure to document this and maintain a log.

• 8% of initial audits received a finding against paragraph 15.1, “Is there a written policy for specimen replacement testing and retesting (in accordance with customer requirements), established and referenced through the Quality System?“. This NCR is raised simply due to the absence of a written policy, not a failure to comply with customer requirements.
• An NCR was raised for 8% of initial audits against paragraph 15.2, “When an outlier is identified in a set of specimen results, is there an assignable cause investigation?". Similar to question 18.1, this NCR is written when outlier results are not investigated for an assignable cause.
• 8% of initial audits saw an NCR written up to paragraph 26.14.8, “Are control procedures in place describing shelf life and out time controls for tabbing adhesive?“. This is written because the laboratory is not tracking shelf life and out time for tabbing adhesive.
• 8% of initial audits included an NCR against paragraph 26.7.2.18, “Is the specimen tested, for at least one iteration, in each of the four loading configurations (end-to end and side-to side)?“. This NCR concerns grip alignment, but will no longer occur, as the question has been deleted in the most recent Audit Criteria revision. The AC7122-I and AC7122-R Appendix A method were revised and no longer require four loading configurations. They now utilize only three configurations. Due to this asymmetry, the ASTM E1012, Annex A compensation method, cannot be applied to alignment data generated using the AC7122-I and AC7122-R Appendix A alignment method.

 

Reaccreditation Audit AC7122-I Top NCRs

• 13% of reaccreditation audits included an NCR against paragraph 26.4.3, “Are chemicals and adhesives used for strain gauging properly identified and stored?”, making it the most frequent reaccreditation audit NCR. This is usually written either because the adhesive is expired, or lacks a label indicating shelf life.
• 9% of reaccreditation audits received an NCR against paragraph 2.6, “Did the Auditee upload a copy of their completed self-audit available to eAuditNet at least 30 days prior to the audit- utilizing the version of the checklist(s) applicable to this audit?”. The self-audit was a new requirement a few years ago.
• 9% of reaccreditation audits had an NCR written against paragraph 2.7, “For each question in the checklist, has the supplier identified where the means of compliance or evidence* of compliance may be found? (*= procedure, checklist, physical location of evidence, etc.)”. If the self-audit is not completed, it is assumed that objective evidence of compliance to the self-audit requirement would not be provided either.
• 9% of reaccreditation audits saw an NCR raised against paragraph 3.4, “Do test procedures include specimen preparation, and are they sufficiently detailed so that the test can be consistently repeated in that laboratory?”. The test procedure typically lacks sufficient detail to be consistently repeatable.
• 6% of reaccreditation audits received an NCR against paragraph 2.8, “Does the self-audit include job audits as required by the TG?”. This NCR was written because, while the self-audit was performed, it did not include the required number of job audits.
• 6% of reaccreditation audits had an NCR raised against paragraph 26.4.1, “Is there a written strain gauge application procedure available which includes:
  1. 1. Specimen Surface Preparation
     2. Strain Gage Alignment and Location
     3. Bonding of strain gage to the specimen
     4. Compatibility of the adhesives and gages to the material and test temperature”.

This NCR was raised because no written procedure exists, or because it lacks all the required information.

• 6% of reaccreditation audits saw an NCR written against paragraph 26.7.2.2, “Are strains at zero load (un-gripped and gripped) and at an average of 1000 microstrain recorded?”. This is a failure to properly perform grip alignment.
• 6% of reaccreditation audits had an NCR written against paragraph 8.1, “Are calibration status labels posted on each calibrated test machine and measuring device?”. This NCR is also on the Initial audit list, and the cause is the same: the equipment has been calibrated, but either the label has fallen off or has not been updated, or the calibration is only tracked via a database.
• 6% of reaccreditation audits received an NCR against paragraph 19.4, “Does the laboratory notify all customers to whom the laboratory sent incorrect or suspect test results within five (5) working days, if affected material is not contained and correctness of results verified, unless otherwise specified by the customer?”. This is another NCR that also shows up on the Initial audits list. It is only a procedural issue as laboratories are notifying customers in a timely manner, but procedure simply fails to specify that this should occur within five working days.

Conclusion

I would caution Auditees not to focus solely on the top NCRs listed above. While these are useful as one part of audit preparation, it should be noted that even the most frequently written NCR described in this article is “only” raised during 23% of audits. A thorough gap analysis and self-audit (utilizing the Audit Handbooks in addition to the Audit Criteria) will identify these issues and allow them to be corrected prior to the actual audit. In addition, several of these top NCRs concern inadequate detail in test procedures or work instructions. This issue has become so common in recent years that the Task Group has developed a detailed Audit Handbook clarification for AC7122-I paragraph 3.4, AC7122-P paragraph 3.3, and AC7122-R paragraph 5.3. The most important sections of this Audit Handbook are reproduced below. “Auditees are required to have test procedures that detail specimen preparation and test methods. This can be accomplished with one or more procedures, and are sometimes referred to as process implementing procedures. Many test methods, such as tensile testing, may be defined by similar test standards, such as ASTM D638, EN6035, & prime unique standards. The importance of creating test procedures is to define how the multiple engineering standards and requirements are applied to a specific Auditee or laboratory. The process implementing procedures must show configuration control. For example, test labs must be able to show how to get from the Prime’s material specification to the referenced test standard to the specific test option and finally to the test procedures. The test method must be:

• Specific to the activities used in the specimen preparation areas and test laboratories
• Specific to the engineering and Auditee options used
• Provide instructions, not just requirements
• For each requirement specify what is to be accomplished, by who, when, & how
• Provide the basis for the Auditee’s/laboratory’s internal audits

A good strategy for writing process implementing procedures is to consider the people involved, not just the process steps. People should easily find their roles and understand how to accomplish the task. Items the procedures could contain are authority documents (such as ASTMs), definitions, materials, facility/equipment, flow chart/standard work, process data, verification processes, what to do with non-conforming results, personnel requirements, records, and forms. The actual test methods may only need to contain some of these items, with the rest defined in other procedures. Auditor verification of “sufficiently detailed” procedures will ensure tribal knowledge and nonstandard work are not the foundation of the quality management system for specimen preparation and testing.” Hopefully, this article is insightful and helps to some degree the Nadcap NMMT community better prepare for their audits. You can contact me at +1 (724) 772 7148 or via email at jtibma@p-r-i.org if you have any questions. Author: John Tibma, Staff Engineer NMMT