Vehicle classification is used in many transportation applications, e.g., infrastructure management and planning. Typical of most developed countries, every state in the US maintains a network of vehicle classification stations to explicitly sort vehicles into several classes based on observable features, e.g., length, number of axles, axle spacing, etc. Periodic performance monitoring is necessary to ensure the quality of collected data; however, such testing has been prohibitively labor intensive to do as thoroughly as needed. To address these challenges, this study examined three interrelated facets of vehicle classification performance monitoring. First, we manually evaluate the performance of vehicle classification stations on a per-vehicle basis, second we develop a portable LIDAR (light detection and ranging) based vehicle classification system that can be rapidly deployed, and third we use the LIDAR based system to automate the manual validation done in the first part using the tools from the second part. In the first part we examined over 18,000 vehicles, at several stations and found good performance overall, but performance for trucks was far worse than passenger vehicles. About a third of the errors were fixed by modifying the classification decision tree, the remaining two thirds of the errors are unavoidable because different classes have overlapping axle spacings or lengths (e.g., passenger vehicles and trucks, or commuter cars and motorcycles). All subsequent uses of the classification data must accommodate this unavoidable blurring. Next, we develop a side-fire LIDAR based classification system that does not require any calibration in the field. Finally, we develop a process to use the LIDAR system (or another temporary vehicle classification system) deployed concurrent to a permanent classification station to semi-automate the manual validation. The automated process does the bulk of the work, typically taking a user only a few minutes to validate all of the exceptions from all lanes over an hour of data. We found wide variance in performance from one station to the next. Since these errors are a function of the specific station, there would be benefit in the short term to leverage the LIDAR based system to evaluate the performance of many other classification stations to catch systematic errors that bias classification performance.