THE TAC EVALUATION PROCESS

The TAC members and the Secretariat met in Vienna from 7 to 10 July 1998. Before arriving members had reviewed many but not all of the submissions. The remaining submissions were received at the beginning of the meeting and were reviewed over the first day. To expedite the initial review the TAC operated on the basis of six geographical sub-groups: Africa; Asia and the Pacific; Europe; Latin America and the Caribbean; North America; and Arab States.

This year, for the first time, the TAC was assisted in its task with the results of a validation process. This had been conducted by the Steering Committee members of the Best Practices and Local Leadership Programme (BLP), the Secretariat and other collaborating organisations who provided expert opinion on the compliance of the submissions with the criteria and working methods of the BLP and its partners. The members of the TAC were encouraged to make their own independent appraisals and then to consider the results of the validation process. They were reminded that the results of the validation process should in no way affect the independence of their decisions.

The TAC spent Tuesday morning in plenary during which time it reviewed the 1996 process and established its own working methods and agenda. The TAC recognised the co-chairs of 1996, representing Europe and North America, and elected two new co-chairs, representing Africa and Asia. In the afternoon, regional sub-groups established lists of best practices from their respective regions. The TAC adopted a recommendation by the Secretariat to assess submissions on their absolute merit as individual practices as well as on their relative merit as compared to other submissions. This process continued during Wednesday morning and resulted in a list of best practices, good practices and submissions that did not comply with the criteria and/or had insufficient information. The number of best practices at this stage was a little over one hundred. As planned in the earlier session the members of the TAC then regrouped in two committees each containing one member of each region. These committees proceeded independently in a "double blind" process to identify those best practices deemed to be worthy of inclusion in a final list of not more than forty.

The Arab States group of the TAC reviewed a total of 51 submissions most of which were submitted in Arabic and required translation. The group had to meet in Vienna for a number of days prior to the TAC meeting in order to translate and summarise these submissions.

Each committee adopted its own working method. One committee adopted a regional approach leaving it to regional representatives to bring forward their recommendations for submissions to be included in the short list. These were discussed within the group and from these discussions the final list was prepared. The other committee endeavoured to have as many members of the group assess as many of the 100+ best practices. They then consolidated their findings through a simple scoring system and produced a ranked list. It is the opinion of the TAC that the use of two different approaches by each committee enriches the selection process.

In the plenary session on Thursday morning, the two lists prepared independently by the two committees were reviewed. There was a substantial level of agreement between the two groups resulting in an overall first round consensus on 28 submissions (representing a 70% concurrence) which were automatically included in the shortlist. The plenary devoted itself to reviewing the remaining submissions in more detail. A final list of 40 was unanimously agreed upon with the understanding that all 40 are of equal merit. From there, the TAC divided itself into various task forces to prepare its documentation and final report.