OBJECTIVE: To develop criteria for testing the validity of indicators of prescribing cost, efficiency or quality. METHODS: Our methodology was based on techniques used to validate psychological tests. Face validity was used to test the construct of the prescribing indicator (i.e. what is it trying to measure?). Two tests of external validity were used to relate the performance of each indicator to other measures of the same construct (concurrent validity and predictive validity). In addition we used a published method to test the robustness of ranking of General Practices by each prescribing indicator. We obtained data for all 78 General Practices in Tayside for the years 1993-97. RESULTS: We identified five prescribing indicators that had been previously tested for face validity in a two-stage Delphi consultation study. These were: 1) the ratio of bendrofluazide 2.5mg to total bendrofluazide items; 2) the ratio of generic items to all items; 3) the ratio of trimethoprim to trimethoprim/sulphamethoxazole; 4) the ratio of antibiotics from the primary care formulary to all antibiotics; and, 5) the ratio of maintenance dose of proton pump inhibitors to all PPI items. Crude ranking of general practices compared well with the results of the more robust rankings generated by the Marshall and Spiegelhalter ranking. There were significant improvements in the achievement of all five indicators by Tayside practices from 1993 to 1997, and ranking of each practice was robust over time. However, there was no relationship between an individual practice’s ranking by different indicators. CONCLUSIONS: Differences between practices are statistically robust and practices have shown significant improvements in all indicators over time. However, the absence of concurrent validity between indicators casts doubt on their use as measures of general prescribing quality.