Constraint aggregation is the key for efficient structural optimization when using the adjoint method for sensitivity analysis. The most widely used constraint aggregation approach, the Kreisselmeier-Steinhauser function, can reduce the number of constraints and returns a conservative estimate. However, this conservative nature reduces the accuracy of the optimization results. This inaccuracy is the most prominent when constraints are active and proportional to the number of active constraints. Using an adaptive approach, this undesirable characteristic is avoided by preserving the exact feasible region. The aggregation parameter is updated according to the constraint sensitivity, and a more accurate result is obtained without additional solver evaluations. This new approach significantly improves the accuracy of the optimization when a large number of constraints are active at the optimum. This improvement is illustrated by the weight optimization of a wing box structure and demonstrated promising computational cost reduction for large-scale design optimization.