Assessing the Construct Validity of Object-Oriented, Class-Level Code Quality Metrics

RESUMO

Background: Code quality metrics are intended to measure latent properties of software source code. Although numerous code metrics have been proposed and used, their construct validity is rarely evaluated. Thus, the extent to which code metrics actually measure what they claim to often unclear. Aim: Drawing from modern measurement theory, we investigate the common class-level, object-oriented code metrics by identifying their factor structure using Exploratory Factor Analysis (EFA). The metrics were extracted from the Apache Maven project by three software tools: Designite, JHawk, Understand. The factor was later verified using Confirmatory Factor Analysis (CFA) on 22 randomly selected open source projects meeting a predetermined eligibility criteria. Results: 24 code metrics that correspond to six constructs: Cohesion, In-Coupling, Out-Coupling, Size, Sub-Inheritance (related to subclasses), and Sup-Inheritance (related to superclasses) were revealed in the underlying factor structure. Ten additional metrics did not correspond to any known dimension of software quality and were removed from the EFA. Ten additional metrics exhibited low loadings in CFA, suggesting their removal from the final measurement model. Cohesion, Size, Inheritance, and Coupling constructs were retained, with subcategories identified for Inheritance and Coupling. Conclusions: Our results strongly support code metrics as multidimensional constructs, since they require measuring two different concepts, revealing sub-categories in our analysis, and Complexity may be better explored in a multilevel model. Overall, our study demonstrates the value of applying modern measurement theory in validating software code metrics.