General findings concerning the assessment of the methodological guidelines for assessing the professional activities of a judge
1 Introduction
Compared with the history of the development of a comprehensive assessment system for the evaluation of judges it must be concluded that over the years major progress have been made to enhance the appraisal system and to incorporate international best-practice approaches into the current methodological guidelines. For example, in 2010 the judicial monitoring system for the work of judges was limited to examining the ‘index of administration’ (related to the number of repealed/altered decisions, the number of violations in of procedural terms for legal investigations and the occurrence of special statements for the violation of the law during a legal investigation compared with other judges in a district or region), the evaluation of the public opinion of the work of the judge, the level of training and education received and the occurrence of disciplinary penalties and complaints against a judge.
One of the recommendations already provided in that same year was to extend the number of criteria for monitoring and to introduce a system of judicial performance evaluation. For example, in the area of legal knowledge and skills of a judge, communication skills, the (ethical) behavior of a judge, etc..
A selective number of these recommendations where included in a revised methodology for judicial performance evaluation adopted in 2016. As a part of the recommendations new practices were introduced with regards the frequency of the evaluation. Newly appointed judges should be evaluated after one year of appointment, whilst more experienced judges should be evaluated every 5 years. Judges with a working experience of more than 20 years are exempted from evaluations though. Incidental evaluations will be carried out when a judge is applying for promotion or a managerial position (e.g. chairmen of the court) in the judiciary. Besides the frequency of the performance evaluations, also a new set of criteria was being introduced, including the evaluation of the professional knowledge and skills of the judge, as well as the evaluation of their professional and ethical behavior. These criteria were added to the already existing criteria related to the number of complaints, the workload of the judges, the number of altered or repealed decisions in higher courts and the number of violations in procedural terms.
Also, in this period recommendations were provided with regards the methods of evaluation, the criteria of evaluation, the need to introduce a uniform scoring mechanism and the weighting of the criteria. Examples of the recommendations given in 2016 were the introduction of the use of an analysis of randomly selected court decisions (this to examine the quality of the content of court decisions), a methodology to observe the behavior of judges at court hearings (e.g. through courtroom observation studies) and the use of 360-degrees feedback mechanisms (to evaluate the work of the chairmen and colleague-judges especially in the area of interpersonal skills, communication skills, efficiency and managerial skills).
In 2019 the new methodological guidelines for assessing the professional activities of a judge have been approved by a decision of the Commission on the quality of justice under the Supreme Court of Kazakhstan (latest version was published on 18 November 2019). When reviewing the new guidelines, we can conclude that a large majority of the recommendations given in 2010 and 2016 have been included in the methodology. As the result of this the new methodology is in line with international standards and best-practices and includes even innovative approaches in the evaluation of the work of judges such as the use of audio and video recordings of hearings to analyze the behavior and communication skills of judges and the application of 360 degrees feedback surveys to appraise the interpersonal qualities, leadership skills and managerial qualities of judges functioning in managerial positions or as a chairmen of the court.
2 The practice of the methodological guidelines
Since the methodology is relatively new, the evaluation system for assessing the work of judges must ‘prove’ itself in practice. Already more than 300 judges have been evaluated according to the new methodology, which gives insight in the practicality and workability of the system and the need to modify on certain elements the evaluation system and methodology. Based on information received, we can conclude that the methodology is complete and endorsed by the judges. All the main criteria for evaluation of the work of judges that can be found in international best-practice examples have been incorporated into the new methodological guidelines. Furthermore, a detailed scoring and weighting mechanism is available, as well as a variety of methods for the collection of information related to the quality of the work of judges, varying from case studies, the examination of a number of court decisions, judicial performance statistics, the use of audio and video recordings, data related to complaints, administrative data about income and assets, etc.
In this sense, the methodological guidelines can be seen as up to the international standards on paper. However, it is also important to verify how the methodological guidelines are working in practice. Therefore, a thorough evaluation of the practice of the guidelines is recommended.
Based on the information received during a mission to Nur-Sultan in January 2020 and the number of available Commission members to carry out the evaluations (4 Commission members who are active judges and 2 retired judges) we can conclude that the implementation of the new evaluation system for judges requires a high volume of resources in terms of evaluators and time that is required to evaluate the work of the individual judges. It seems to be that the current capacity of Commission members is not sufficient to conduct a large number of evaluations in a relative short time period and that the members of the Commission have to make long working hours to analyze the necessary information for rating a judge. Furthermore, there is a shortage of supporting staff that can help the members of the Commission in the evaluation process. Therefore, it is recommended to consider in the future to increase the number of evaluators/Commission members and the number of staff that can support the work of Commission members.
In this context it is important to discuss the option of having full-time evaluators/Commission members and part-time Commission members (who have still an active role as a judge in a court). Given the relative high workload of the current Commission members it is advisable to opt for full-time evaluators and not for part-time evaluators. There is another reason why a full-time evaluator position is recommended, and this is related to the protection of the independent position of the evaluator and the avoidance of potential conflict of interests. A part-time position as an evaluator may give problems if the evaluator who is working as a judge in a specific court, might be confronted with the evaluation of his or her colleague judge working in the same court. Of course, anonymous reports and judicial files may prevent this situation, but it will be hard to avoid a conflict of interest if the judge evaluator is working at the Supreme Court and he or she has to evaluate another judge of the Supreme Court. To maintain a high level of independence and to minimize conflicts of interests it is advisable to appoint only full-time Commission members.
3 The positioning of the Commission on the Quality of Justice: a choice between the Supreme Court and the High Judicial Council
In 2016 the High Council of Justice was established in Kazakhstan. This Council is formally responsible for the selection of new judges, promotion, transfer and dismissal of judges. Also, the Council plays a significant role in disciplinary proceedings against judges due to the existence of a Judicial Jury. Based on the examination of violations of the codes of ethics of judges or even the conduct of criminal acts the Jury can decide to impose disciplinary measures and sanctions or to suggest dismissing a judge (in case of grave violations). The judicial performance evaluation though is currently not in the hands of the High Council of Justice but is a responsibility of the Commission on the Quality of Justice of the Supreme Court. The key questions that are currently being under discussion are the following: should the evaluation of judges be removed from the Supreme Court and transferred to the High Council of Justice? What are the advantages and disadvantages of such a transfer? What is the international practice?
A good starting point for answering these questions is to examine the current practice in countries where there exist High Judicial or Supreme Judicial Councils. In most of the countries where there are High Judicial Councils, the key tasks of these councils are concentrated on the appointment of new judges, promotion, transfer and disciplining judges (see for example the Councils for the judiciary in France, Italy, Moldova, Spain and a number of Councils in the Middle East region). To a more limited extend there are Councils that are also responsible for the management of the courts (e.g. in the Netherlands, Finland, Ireland and Sweden). In the High Councils for the judiciary with tasks related to the appointment, promotion, transfer and disciplinary proceedings it is a common practice that in these Councils there are (independent) evaluation boards/commissions and disciplinary boards. In other words, it is a ‘natural task’ of a High Judicial Council.
However, there is one important precondition that must not be overlooked and that concerns the level of independence of the High Council of Justice. To avoid undue influence from the executive power and the legislative power in for example the appointment and removal of judges, it is a common practice and European standard (see the Opinion of the Consultative Council of European judges of the Council of Europe on Councils for the judiciary) that Councils for the judiciary are independent from the executive and legislative power, form a part of the judiciary and that the majority of the members of the board of the Council are members of the judiciary. Therefore, it is more and more the trend to remove Ministers of Justice from the board of a Council for the judiciary, as well as other representatives of the executive power (e.g. high-level civil servants of the Ministry of Justice). In addition to this it is quite common in High Councils of Justice that the chairman of the Supreme Court is also the Chairman of the High Council of Justice.
Besides, that High Councils of Justice can play a role in the evaluation of judges (if the European standards on Councils for the judiciary are being met) there are other modalities possible as well. For example, in Ukraine it is the High Qualification Commission that is responsible for the evaluation of candidate-judges and the regular evaluation of judges. Recommendations of the Commission for appointing a judge and about the results of the evaluation are being transferred to the High Council of Justice of Ukraine for further decision making. Other modalities that can be found in the world are a Supreme Court or a Ministry of Justice (in the past in Turkey the Ministry of Justice was responsible for the evaluation of judges) which have incorporated in their formal duties the task of evaluating the work of judges. Supreme Courts come often into play in the evaluation of judges, if there is no High Council of Justice and if the Supreme Court has a central role in the management of Courts. The positioning of evaluation of judges in a Ministry of Justice is lesser often applied and is also not recommendable given the protection of the independent position of judges and the application of the principle of separation of powers between the legislative, the executive and the judicial branches of power.
As has been indicated at the beginning of this paragraph in the current situation the Supreme Court is responsible for the evaluation of judges, besides the existence of a High Judicial Council. The key question is the following: should this situation be continued, or should the evaluation tasks be being transferred to the High Council of Justice? To answer this question, it is important to look at the level of independence of the High Council of Justice and if the Council is part of the judiciary. At the moment the High Council of Justice is seen as an administrative authority. This is also reflected in the composition of the board of the Council, whereas members of the executive power are being represented, in addition to members of the judiciary. According to the international standards to secure a high level of independence of a Council for the judiciary the majority of the members of the board should be members of the judiciary and most ideally no representatives of the legislative and executive power should participate in the board of a Council. Furthermore, High Councils of Justice should be exclusively part of the judiciary, completely separated from the executive power and the legislative power. When comparing these conditions with the current statute of the High Council of Justice in Kazakhstan, we can conclude that the Council is not an independent body, but an administrative organ belonging to the executive branch of power.
Since the conditions of independence of the judiciary in the High Council of Justice are not met yet, it is therefore advisable to maintain the current status quo, where the Commission of the Quality of Justice is being kept under the authority of the Supreme Court. This can change over time though when the level of independence of the High Council of Justice will be strengthened.
4 Transparency and accountability of the work of the Commission on Quality
One important aspect in the work of the Commission on Quality is not only related to the execution of the evaluations of the work of the judges, but also to the role of the Commission in contributing to the strengthening of the public trust and confidence in the judiciary. To realize this, it is necessary that the Commission collect and present statistics about the number of judges that have been evaluated, the results of the evaluations (in terms of number of judges that are meeting the requirements/standards, the number of judges that are underperforming and the number of excellent judges), the number of judges that have been promoted as the results of the evaluations, etc. (on an annual basis). Furthermore, the Commission can contribute to other human resource management statistics, such as the average age of the judges, the distribution of male and female judges, etc. as well.
This information can help to realize a better understanding among the general public how the judiciary is performing, not only by looking at the performance at the level of the courts (district level, regional level and Supreme Court level), but by publishing information about the quality and performance of the individual judges as well. With regards the last point it is relevant to mention that for the protection of the independent position of the judges it is not advisable to publish the names of the judges that are underperforming. This can hinder their daily work as a judge and can have a further negative influence on their performance and quality of their work. Therefore, it is advisable to present only general statistics about the work of the individual judges, with the (optional) exception regarding the outstanding performing judges. Of course, this category of judges can be publicly rewarded for their good work by naming them in the publications of the judiciary.
5 A more detailed analysis of thee methodological guidelines for assessing the professional activities of a judge
In general it must be welcome that in the provisions of the guidelines it is clearly underlined that the main purpose of the assessment of the professional performance of judges concerns: the improvement of the quality of the judiciary, the stimulation of the growth of the professional knowledge and skills of judges, to strengthen the rule of law and protecting the rights of citizens. In other words, the evaluation is seen as a positive tool for self-improvement of judges, and not as a sanctioning instrument (clearly separated from disciplinary proceedings).
The evaluation criteria
Furthermore, all relevant criteria for the evaluation of judges are being included in the methodology that can be found in other methods of judicial performance evaluation too. It contains the assessment of the professional knowledge and skills of judges, the judicial performance, the managerial knowledge and skills and the compliance with professional and ethical standards for working as a judge. This must be seen as a positive aspect of the methodology.
The frequency of the evaluation
In the methodology a distinction is made in the frequency of the evaluation depending on the level of seniority of a judge and the purpose of the evaluation (e.g. for promotion). Recently appointed judges are evaluated after one year of appointment, whilst (senior) judges are being evaluated every five years. Judges with 20 years or more working experience are being exempted from the judicial performance evaluations. If a judge is applying for a higher court position or the position of a chairman of the court, he or she must be evaluated too. In general, the frequency for evaluations of newly appointed judges and judges with working experience is in line with common international practice. However, one can decide to increase the frequency for (senior) judges from 5 to 3 years, depending on the available capacity of Commission members/evaluators. This can improve the quality of justice, since judges are more frequently monitored on their knowledge skills and performance and this can give a good insight in the potential training needs of those judges.
In the current methodology the senior judges with 20 years or more of working experience are excluded from the judicial performance evaluation. The question is if this is advisable. Since it is mentioned as an important rule on the law on the status of judges it is difficult to change this practice. However, since also for senior judges the principle of ‘lifelong learning must be applied’ it is necessary to introduce a more informal method of evaluation of their current capacities, knowledge and skills. A solution can be to request these judges every year which kind of training needs they have (and if they should mandatory participate in a number of specific training courses). In addition to this, the workload and the performance of older judges must be monitored too. Certain categories of older judges might not be able to keep up with their current workload and there might be a need to reduce their workload and to replace them with other e.g. out of court duties, such as mentoring and training younger judges. Informal methods such as the use of ‘personal development plans’ can be of use in this respect. The aim of these plans is to document a process of self-analysis, personal reflection and an honest appraisal of the strength and weaknesses of the judge.
The creation of a personal development plan contains three stages:
Stage 1: personal analysis. The first stage is designed to analyze your strengths and weaknesses. This analysis must be supplemented with the perceived opportunities you can identify based on your experiences and any potential threats that can weaken your performance results.
Stage 2: setting goals. This stage involves of defining (performance) goals, about what you want to achieve in the future. Most ideally, these goals should be set in a SMART manner.
Stage 3: personal objectives. This stage involves the setting of your personal objectives. These can also be set in the context of your employment.
The personal development plans are a good instrument for having regular communication between the judge and their supervisors (chairmen of the courts), in order to discuss the personal targets that the judge wants to set, the needs of the organization and the career ambitions. Moreover, the instrument is effective to stimulate the life-long learning principle, through a continuous focus on training and education.
The methods of evaluation
The new methodological guidelines are using a mixture of various methods to collect the necessary information for examining and rating the quality and performance of the judges. As has indicated already in earlier paragraphs one of the key methods for assessing the quality and knowledge of judges is to evaluate a random number of court decisions in order to check the correct application of the law, logical reasoning, the quality of the motivation, grammar, etc. In the current methodology for each judge three decisions will be selected for this purpose. One can question if this number is sufficient to determine that level of quality of the court decisions. Depending on the available capacity of the evaluators it is recommendable to increase the number of court decisions to be examined per judge from 3 cases to 5 cases. Another remark that must be made in this context is related to the number of evaluators that is evaluating the cases for an individual judge. At the moment only one Commission member is evaluating and scoring the quality of the cases /court decisions per judge. To reduce the level of subjectivity in the evaluation and scoring it is advisable that at least 2 Commission members/evaluators are looking at the court decisions of a judge and that the average score per evaluator is being used for rating a judge with regards the quality of the court ruling.
A similar remark can be made concerning the use of the audio and video recordings to determine the quality of the judicial processes of the judges. In the current method only one Commission member is scoring and evaluating the quality of judicial processes by reviewing the audio and video recordings. Though to improve the level of objectivity of the scoring it is advisable that two evaluators are reviewing the same audio and video recordings and that the average scores of the two evaluators are being used to rate the quality of the judicial processes.
For the assessment of the business qualities of a judge or a chairman of a court, the methodology includes the use of a 360-degrees feedback form. As has been indicated earlier in the report, the 360-degrees method is innovative and must be welcomed as a part of the methodology. Currently the survey is being submitted through web-based questionnaires. According to previous experiences not always the replies submitted by the recipients were reliable, due to the fact that no unique identification number was being used in the forms. This resulted in certain courts in a situation that there were more replies submitted of the questionnaire than the total number of available judges and court staff… This problem though, has been already solved based on the earlier experiences.
When looking at the 360-degrees feedback questionnaire itself, there seems to be a possibility to improve the quality of the questionnaire. For example, in Australia (the judiciary of Victoria) is also using a 360-degree feedback questionnaire. In addition to the questions that have been included in annex number 5 of the methodological guidelines the following topics are questioned as well: communication skills (“ensures that what they say is understood), dealing with people (“treats others with respect and courtesy”) and organization and planning (“appears to be well prepared”). The behavior aspects that are included in the Australian survey are related to the following categories: communication, environment, judgement, learning, non-verbal communication, organization, people focus, relationship and self. Also a number of snapshot questions are included in the survey about the strengths of judge (e.g. efficiency, self-awareness, enthusiasm, respect and courtesy) and their perceived challenges (e.g. in the area of: problem solving, decision making, efficiency, written communication, empathy, influencing skills, social skills, etc.). After that in Australia (district of Victoria) all the questionnaires have been filled in and analyzed a confidential report is send to an organizational psychologist who is facilitating the program. He or she arranges a 90 minutes meeting with the evaluated judge, to explain the content of the report and how it can be used as feedback information to improve their skills and changing their behavior. At the end of this process, a group of participating judges will attend a session with the organizational psychologist to discuss the general findings and to present aggregate data to the group.
Weighting, scoring and motivation of the recommendations for the results of the evaluations
It is positive to note that the methodological guidelines are based on a sophisticated, advanced and well-developed scoring and weighting mechanism to weight the professional knowledge and skills, the judicial performance, the managerial qualities and skills and the moral qualities for compliance of the Code of Judicial Ethics. Also, it is good to see in the methodology that for the scoring of the workload/performance of judges a distinction is made on the basis of the level of the complexity of the cases in the civil, criminal and administrative law areas. This will guarantee that the performance of each individual judge will be fairly judged and compared with the national norms and standards.
With regards the weighting between the four criteria though, we can conclude that relative high weights are given to the quality of judicial activities and relative lower weights are given to the criterion of judicial workload/performance and the moral qualities of a judge. In international practice it is common that the quality of court decisions is equally weighted with the judicial court performance. This to have a proper balance between ‘judicial quality’ and ‘efficiency of justice’. Apparently, in Kazakhstan there seems to be lesser problems in the sphere of the individual workload and performance of judges. Most of the judges are meeting the minimum standards for judicial performance. Therefore, it is understandable that this criterion is given a lower weight compared with the other criteria for judicial performance evaluation. To have a better understanding for how the weighting is done in the current methodological guidelines it is recommended to include this way of reasoning in the guidelines.
Another suggestion for improving the weighting of the evaluation criteria is to give more weight to the moral qualities of a judge. This to underline that these qualities are also an important aspect to determine the general quality and performance of individual judges.
Besides the weighting, it is also important to pay attention in this report on the way of total scores are ‘calculated’ by making use of specific scoring forms/evaluation sheets, such as for scoring the quality of court decisions and the quality of judicial processes. The evaluation sheets for scoring the quality of decisions contains all the relevant elements to assess the main parts of a court decision, such as the introductory part, the descriptive part, the declaration part, the operative part, and the presence of spelling and stylistic mistakes. For each of these elements the evaluator can give a “+” score or a “- “score, as well has the possibility to make remarks. What is problematic with the evaluation sheet is that it is unclear how the total number of “+” or “- “marks will relate to the determination of the overall score ranging from a score of 1 to 5. In other words, how is an evaluator providing a total score, based on the number of “+” and “- “? Since this is not clearly defined, there is room for the evaluator/Commission member to determine arbitrarily the total score/overall assessment. The lack of a clear relationship between the markings in the evaluation sheet and the provision of a total score is another reason for the use of two Commission members instead of one for examining and rating the quality of a court decision.
The same problem can be found in the evaluation sheet for rating and scoring of the compliance of the administrative, criminal and civil proceedings during court hearings. Also, for this sheet it will be necessary to improve the relation between the marking of “+” and “- “scores and the overall rating (from 1 to 5).
In addition to these observations, we can conclude that the methodological guidelines are strongly focused on a quantitative approach for rating and scoring the quality and performance of individual judges. This has the advantage that the level of subjectivity in the evaluation process is being reduced and that the results are summarized in quantitative scores. However, since there is not always a strong relationship between the marking of individual items, such can be found in the evaluation sheets for the quality of the court decisions and the quality of the judicial processes and the overall assessment rated from 1 to 5, it gives the evaluation a ‘quasi’ objective impression. To improve the level of objectivity of the methodological guidelines it is recommended that, besides the provision of scores, the recommendations of the Commission on quality will contain a narrative part as well. In this part a clear motivation must be provided about how the Commission (in a plenary session) has scored an individual judge and what were the underlying reasons for the end score (to determine if a judge is qualified, not qualified, etc.). A good motivating part of the recommended of the Commission to the High Council of Justice is therefore advisable.
