Executive Evaluation Report of the Refugee Determination System of the Immigration and Refugee Board of Canada

​Policy, Planning and Corporate Affairs Branch

March 2016

On this page​

  1. 1. Introduction
  2. 2. Background
    1. 2.1 Refugee determination system
    2. 2.2 Rationale for reform
    3. 2.3 Key features of reform
    4. 2.4 Overview of the post-reform refugee determination system
    5. 2.5 Structure and processes of IRB post-reform
    6. 2.6 Resources of IRB post-reform
  3. 3. Evaluation methodology
    1. 3.1 Document and data review
    2. 3.2 Key informant interviews
    3. 3.3 Survey of authorized representatives
  4. 4. Evaluation Findings
    1. 4.1 Delivery
    2. 4.2 Effectiveness
    3. 4.3 Best practices lessons learned and improvements

1. Introduction

The Immigration and Refugee Board of Canada (IRB) commissioned an independent evaluation as part of its ongoing efforts to monitor and improve its delivery of the refugee determination system, which was substantially reformed by the Protecting Canada's Immigration System Act (PCISA) and the Balanced Refugee Reform Act (BRRA). The evaluation covers the period from December 15, 2012 to March 31, 2015 and focuses on issues related to resourcing assumptions of the new system, its efficiency and effectiveness. The evaluation did not examine matters falling within the adjudicative independence of decision-makers or the legislative mandate of other portfolio organizations engaged in the new system.

2. Background

2.1 Refugee determination system

The refugee determination system supports Canada's international obligations as a signatory of the United Nations' Convention Relating to the Status of Refugees (1951), the Protocol Relating to the Status of Refugees (1967), the International Covenant on Civil and Political Rights (1966), and the Convention Against Torture and Other Cruel, Inhumane or Degrading Treatment or Punishment (1984).The fundamental rights of refugees are also protected in the Canadian Charter of Rights and Freedoms. These international obligations and Canadian law require Canada to provide protection to Convention refugees and persons in need of protection. This includes people who have a well-founded fear of persecution based on their race, religion, nationality, political opinion, or membership in a particular social group, as well as people who would be subjected to a danger of torture, a risk to their life, or a risk of cruel and unusual treatment or punishment if they were to be returned to their home country.

Foreign nationals seeking protection from inside Canada are processed through the In‑Canada Asylum System (ICAS). The program involves a number of federal organizations, including Citizenship and Immigration Canada (CIC)Note 1, the Canada Border Services Agency (CBSA), and the IRB.

2.2 Rationale for reform

Canada's refugee programs experienced a number of substantial challenges over the past decade due to a large number of new refugee claims being made each year and a shortfall of IRB decision‑makers, which led to a large backlog of unresolved refugee claims — reaching 62,000 claims by 2009 — coupled with lengthy delays in finalizing claims (averaging 18 months in 2012). Adding to these strains, approximately 18% of refugee claims were withdrawn or abandoned, which was seen by the government as an inefficient and unproductive use of the refugee system.

In response to these challenges, Parliament passed both the BRRA, which amended the Immigration and Refugee Protection Act (IRPA), and the PCISA, which further amended the IRPA and the BRRA. On December 15, 2012, changes to the refugee determination system and related regulations took effect. All claims referred to the IRB after that date were to be processed under the reformed refugee determination system, while claims referred to the IRB prior to that date — called legacy claims — were to be processed under the new system with certain exceptions, most importantly, they were not subject to the legislative time limits for scheduling the Refugee Protection Division (RPD) hearing.

2.3 Key features of reform

The reforms affected all participating organizations of the ICAS. The following provides an overview of the key features of the reform as it affects the IRB's role in the refugee determination system.

Summary of the key features of reform
Selection of RPD
members		Under the previous system, RPD proceedings were conducted by appointees of the Governor in Council (GIC). Under the new system, RPD proceedings are conducted by public service employees appointed under the Public Service Employment Act.

Designated Country of Origin status

The reforms introduced different categories of claims based on the status of the claimant's country of origin. Under the reforms, the Minister of CIC may designate countries of origin (DCO) on the basis of the criteria set out in the IRPA. These criteria are low refugee claim success rates, high claim withdrawal or abandonment rates, or if the Minister is of the opinion there is an independent judicial system, basic democratic rights and freedoms with mechanisms of redress, and the existence of civil society organizations in the country.

Legislated timelines for scheduling RPD hearings

The reforms introduced legislated timelines for the scheduling of RPD hearings, which vary depending on whether the claimant is from a DCO or not. Referring officers at CIC or the CBSA are required to schedule inland DCO claimants' RPD hearing no later than 30 days after referral, while port‑of‑entry (POE) DCO claimants are to have their hearing no later than 45 days after referral. Non‑DCO claimants are to have their hearings no later than 60 days after referral to the RPD.

Basis of Claim (BOC) form

The BOC form is intended to gather information to inform the RPD hearing, including information about the claimant's identity, family, documents and travel history, and reasons for claiming refugee protection in Canada.
Under the previous system, claimants were required to complete and submit a Personal Information Form (PIF) within 28 days. Under the new system, claimants who make their claim at a POE are required to return their completed BOC to the RPD no later than 15 days after their claim is referred, while claimants who make their claim at an inland office are required to submit their completed BOC during the eligibility interview.

Refugee Appeal Division (RAD) and legislated timelines

The reforms created the RAD with GIC‑appointed decision‑makers. Claimants or the Minister may appeal certain decisions made by the RPD on a question of law or fact (or both). The Minister may also intervene in an appeal made by a claimant. Appellants have 15 days from receipt of the RPD's written reasons to file an appeal and 30 days from receipt of the RPD's written reasons to perfect their appeal. The RAD is required to make its decision within 90 days of the perfecting of the appeal, except when a hearing is held. The RAD can confirm the RPD's determination, refer the matter back to the RPD for redetermination, or set aside a determination and substitute it with its own.
Under the reforms, decisions on claims made by DCO claimants could not be appealed to the RAD. In July 2015 the Federal Court rendered a decision impacting this right of appeal and the government has not challenged this decision.Note 2 Failed DCO claimants may now file an appeal to the RAD.

Expanded ministerial interventions

Under the previous system, the Minister had the right to intervene before the RPD, and the RPD had the obligation to notify the Minister if certain issues arose, most notably a possibility of exclusion. Under the reformed system, while the Minister still has the same right to intervene in cases before the RPD, its rules now require the Division to notify the Minister if the RPD believes that there are possible program integrity issues (e.g., fraudulent documents).

Changes to RPD jurisdiction

Under the previous system, the RPD had jurisdiction to reopen any previously decided claims in appropriate circumstances. Under the new system, the RPD cannot reopen a previously decided claim or application once a final determination has been made by the RAD or the Federal Court, as the case may be.

The IRB undertook numerous internal activities to prepare for its role in the reformed refugee determination system. The legislative reforms prompted the need for new divisional rules for the RPD and the RAD. Divisional rules are aimed at informing parties and their counsel about the practices and procedures of the IRB, ensuring parties and counsel are prepared for hearings, and promoting transparency and consistency in decision‑making. The legislative reforms also prompted changes to Chairperson's guidelines and instructions, codes of conduct, and polices, some of which were specific to the RPD and some that applied generally to all divisions of the IRB.

2.4 Overview of the post‑reform refugee determination system

This section provides a brief overview of the post‑reform refugee determination system with a focus on the IRB's role. The table below is a summary of the key stages of the refugee determination system and provides a graphic depiction of how cases move through the system. This brief overview is intended to help contextualize later discussions in the findings section of this report.

Summary of refugee determination system
Intake, referral, and screeningThe refugee determination process begins with a CIC or CBSA officer determining whether a refugee claim is eligible for referral to the RPD. After a claim is deemed eligible, the referring CIC or CBSA officer sets the date for the RPD hearing and for providing the relevant information and forms to the claimant, including a BOC form for POE claimants. The CBSA is responsible for coordinating front‑end security screening and for advising the RPD once the security screening process is complete. If security screening is not completed by the hearing date, the hearing is rescheduled.
RPD hearing

All claimants receive a Notice to Appear for Hearing that indicates when and where the hearing will take place. The Notice also includes special hearing dates that may be used if a claimant fails to complete and/or submit their BOC, where applicable, or if they fail to appear at their RPD hearing. The special hearing provides the claimant a chance to explain why their claim should not be declared abandoned (such as a medical emergency).
Hearings typically consist of testimony, representations from the claimant or counsel, and representations from the Minister's counsel, if participating. Hearings may occur over multiple sittings. Documentary evidence, including information from the IRB's National Documentation Packages, is also considered during the process.

RPD decision

RPD decision‑makers must provide their decision and reasons for the decision orally at the hearing, unless it is not practicable to do so. They are also required to provide written reasons for their decision in all cases, with one narrow exception.

Appeals

A key change in the system is the creation of the RAD. The RAD consists of GIC appointees who consider paper‑based appeals and who conduct oral hearings in certain limited circumstances set out in the IRPA. Both refugee claimants and the Minister may file an appeal within 15 days of receiving the RPD's written reasons for decision on the basis that the RPD made errors of law or fact, or both, in its decision. The person who is the subject of the appeal can present only evidence that arose after the rejection of their claim or that was not reasonably available or that they could not reasonably have been expected, under the circumstances, to have presented at the time of the rejection of their claim.
Appellants have 30 days after receiving the RPD's written reasons for decision to perfect their appeal by filing an appellant's record; if they fail to do so, their appeal may be dismissed. An application for an extension of time to file or perfect an appeal may be made.
The RAD must make its decision within 90 days of the appeal being perfected (except when an oral hearing is held); it may confirm the RPD's determination, refer the matter back to the RPD for redetermination, or set aside the determination and substitute its own.

Application for leave and judicial review

Prior to the creation of the RAD, failed refugee claimants only had the option to make an application for leave for judicial review at the Federal Court of Canada. Judicial review remains available for claimants who have received an unfavourable decision by the RPD and who do not have a right of appeal to the RAD. The Minister may also seek leave for the judicial review of a positive decision by the RPD if there is no right of appeal to the RAD. All RAD decisions are subject to an application for leave for judicial review. Applications for judicial review must be made within 15 days of a negative RPD or RAD decision

Post-reform refugee claim process

test
[Alternate format]

The image above describes the In-Canada asylum determination process.

The figure encompasses a flowchart consisting of boxes connected with arrows.

The two first boxes are centred at the top

  • The Left box is entitled Refugee claim made inland (BoC Form submitted at eligibility interview)
  • The Right box is entitled Refugee claim made at port of entry

On the left, an arrow links the first box to another one named Claim ineligible

On the right, an arrow links the second box to another one named Claim ineligible

The two boxes on the top are connected to boxes that are below them with down arrows

There are boxes Claim eligible on the left and Claim eligible (15 days to submit Boc Form) on the right

Under the right box is another box named BoC Form submitted with relevant documents.

An arrow oriented to the left links it to the box RPD hearing, which is in the left column.

A down arrow connects RPD hearing box’s to Claim accepted or rejected.

A down arrow oriented to the right, under which it is written in brackets no right of appeal, leads to the box Application for leave for judicial review in Federal Court

Another down arrow connects the box Claim accepted or rejected, under which it is written in brackets right of appeal to RAD appeal box.

A horizontal line with three down arrows connects RAD appeal box to three boxes below.

Boxes from right to left: 1st box: Confirm RPD decision, 2nd box: Set aside RPD decision and substitute determination, 3rd box: Refer matter back to RPD for re‑determination; a long up arrow connects this box to the RPD hearing box.

These three boxes are interconnected with an arrow leading to the box Application for leave for judicial review in Federal Court.

This box is in the center of two other boxes; right box is Leave denied and left box is Leave granted.

In the bottom of Leave granted box there is another box called Judicial review dismissed.

At the left of Leave granted box there is the box Judicial review allowed (matter back to RPD or RAD, as the case may be).

A long up arrow connect this box to an upper box called RPD hearing.

At the top of the figure there were two boxes, one on the left and another one on the right. Under the right box Refugee claim made at port of entry there is the box Claim eligible (15 days to submit Boc Form).

A lateral arrow on the right connects that box to another one entitled BoC Form not submitted, which is linked to a box under Special RPD hearing on abandonment.

A down line from the box entitled Special hearing on abandonment forks into a horizontal line which connects to the right box entitled Claim declared abandoned, and to the left box entitled Claim not declared abandoned. Under these two boxes it is written in brackets decision subject to judicial review.

An up arrow connects the box Claim not declared abandoned to the box Claim eligible (15 days to submit BoC Form).

2.5 Structure and processes of IRB post reform

The IRB's role in the refugee determination system involves two separate divisions. The RPD is responsible for deciding refugee protection claims and Minister's applications for vacation and cessation of refugee status. The RAD is responsible for deciding appeals of certain RPD decisions. Appeals can be made by the refugee claimant or the Minister. Each IRB division is headed by a Deputy Chairperson who reports to the Chairperson of the IRB.
The IRB's divisions carry out their work in three regional offices, located in Toronto (Central Region), Montréal (Eastern Region), and Vancouver (Western Region). At the regional level, the RPD and the RAD are each led by an Assistant Deputy Chairperson. The RPD Deputy Chairperson oversees the RPD members, the RPD Registry and analysts. The RAD Deputy Chairperson oversees RAD members and analysts.
A number of program support services operate within the IRB to assist with the operational needs of the RPD and the RAD. Program support services that were considered in the current evaluation are described in the table below.

Other IRB services considered in the evaluation
Branch/ServiceFunction with respect to the refugee determination system
RegistriesThe IRB created a dedicated registry for the RPD to support efforts to meet the shorter timelines introduced by the reforms. These changes included co‑locating RPD members and registry staff.
The Registry and Regional Support Services oversees the common registry that provides comprehensive registry services to the RAD and common services (e.g., interpretation services, records and mail) to both the RAD and RPD.
Policy, Planning, and Research Branch (PPRB)PPRB is responsible for management, oversight, and promoting efficiencies among the IRB programs and divisions. This includes strategic planning and analysis, performance monitoring and evaluation, and development of policies and processes.
Performance measurement and data analysis is conducted by the branch's Standards, Analysis and Monitoring (SAM) Unit.
The Research Directorate provides research services for the IRB, conducting Country of Origin research through publically accessible information sources. This information feeds into Responses to Information Requests (RIRs) and National Documentation Packages (NDPs). Research is also conducted with respect to specific claims.
Legal ServicesCorporate Legal Services at headquarters provides legal advice to all divisions on corporate matters, including human resources, procurement, governance, and implementation of reforms.
The Office of the Senior General Counsel at headquarters includes Senior Counsel who provide advice to the Deputy Chairpersons' offices of the RPD and the RAD on broad adjudicative matters and cross‑divisional issues.
Legal Services in the regional offices provides legal advice to the RPD and RAD Assistant Deputy Chairpersons, members, registry, and other branches in relation to specific cases before the RPD and RAD. In addition, Legal Services conducts formal reasons review and provides legal advice to members on specific files.
Legal Services also provides training to the RPD members, the RAD members, and the RAD analysts.

2.6 Resources of IRB post-reform

Financial information from the IRB's Departmental Performance Reports (DPR) peaked in 2011–12 at $140.8 million, as a result of temporary funding provided to implement the federal government's refugee reforms. Spending has since decreased below 2010–2011 levels, due to the decrease in temporary funding.

IRB expenditures (2009‑10 to 2013‑14) ($ millions)
Overall2009‑102010‑112011‑122012‑132013‑14
114.1 125.6 140.8 134.3 121.9
RPD58.263.472.469.855.8
RAD--1.83.87.5

A similar pattern is apparent in the IRB's human resources, with FTEs peaking in 2011‑12. The RPD has the largest number of FTEs compared to the IRB's other divisions, but has declined in size slightly since 2011‑12.

IRB human resources (2009‑10 to 2013‑14) for FTEs
Overall2009‑102010‑112011‑122012‑132013‑14
973 1 023 1 092 1 045 997
RPD534573602573512
RAD--142262

3. Evaluation methodology

The evaluation covered the period from the coming into force (CIF) of the reforms on December 15, 2012 to March 31, 2015. It was undertaken in three phases: the design phase, the data collection phase, and the reporting phase. During the first phase of the evaluation, key evaluation documents — including a logic model and a combined performance measurement strategy and evaluation matrix — were developed. To assist in their development and ensure broad input into the design of the evaluation, the IRB held a half‑day workshop with 15 individuals representing PPRB; Legal Services; Corporate Services; Executive Offices; Reform Office; the RPD; the RAD; and the Common Services Registry. At the workshop, a draft logic model and combined performance measurement strategy and evaluation matrix were presented. Participants provided feedback and revised documents during the session.

The logic model developed for the evaluation identified the following outcomes for the IRB refugee determination system.

Immediate outcomes:

  • Parties are aware of and have access to up‑to‑date and relevant information concerning the process and their rights and obligations.
  • Parties have access to information about their case.
  • The processing of refugee claims and appeals is in accordance with regulated time limits.
  • The RPD and the RAD have access to and use up-to-date and relevant guidance and policy to make quality decisions.

Intermediate outcomes:

  • Parties are presenting their cases efficiently and in accordance with requirements.
  • The RPD and the RAD resolve cases efficiently and in accordance with the law.

Ultimate outcome:

  • The IRB makes refugee protection decisions that fulfill Canada's humanitarian traditions and contribute to safeguarding Canadians' security.

Based on the logic model and evaluation matrix, data collection instruments were developed. The evaluation made use of multiple lines of evidence in order to support robust findings. The methodology included three main lines of evidence: a document and data review, key informant interviews, and a survey of authorized representatives. Each of the evaluation methods is described more fully below. This section also includes a brief discussion of methodological challenges.

3.1 Document and data review

The document and data review was conducted both to inform the development of data collection instruments and to address the majority of the evaluation questions. Internal IRB documents included statistical performance reports, operational manuals, business mapping exercises, and knowledge management tools. Publicly available documents include the DPR, Reports on Plans and Priorities (RPP), relevant information on the IRB, CIC, and the CBSA websites, and results from the Public Service Employee Surveys (PSES).Note 3 In addition to documents, the evaluation reviewed NOVA data for fiscal years 2012‑13 to 2014‑15.Note 4

3.2 Key informant interviews

Interviews conducted with key informants addressed the majority of evaluation questions, and were a key line of evidence in gathering information on the effectiveness and efficiency of the IRB's delivery of the new system. A list of potential key informants was prepared, and interview guides tailored to each key informant group were developed. Interviews were conducted with a total of 63 key informants.

3.3 Survey of authorized representativesNote 5

To gather the input of authorized representatives who have appeared before the RPD and RAD under the new system, the evaluation included an anonymous and confidential web‑based survey. The survey was online for three weeks, from June 15 to July 6, 2015. During this period, two reminders were sent to potential participants to increase the response rate. Invitations were sent to the 708 authorized representatives for whom email addresses were available. Six of these email addresses were no longer in service or the potential participants were not available during the time the survey was online. In total, out of the 702 authorized representatives, 120 respondents completed the survey for a response rate of 17%. Once the survey was finished, open‑ended questions were coded and the survey data was analyzed using SPSS, a statistical software package.

4. Evaluation Findings

4.1 Delivery

  1. 4.1.1 Assumptions

Underlying the IRB's design of the new refugee determination system are core assumptions about the volume and nature of the claims, time to conduct key activities, and availability of members. The evaluation found that the assumptions underestimated the resources required to deliver the new system according to its legislated timelines and operational targets.

The volume of claims in the first two fiscal years has been substantially lower than the projection of 22,500 claims annually, but it is on the rise (up just over 25% for 2014‑15 to 13,216 claims). Key informants (external in particular) noted that the lower intake was to be expected; whenever there is refugee system reform, potential claimants will refrain from filing claims until they see how the system is working. The first two fiscal years indicate a system already experiencing some strain, as pending claims after fiscal year 2013‑14 were 2,217.  Rather than reduce the pending claim inventory, 2014‑15 added 472 claims, to result in 2,689 claims pending at the end of two fiscal years.

Another key assumption is related to the complexity of the claims. The assumption was that 10% of claims would be “complex” and 90% of claims would be “regular.” This assumption affected the amount of time allocated to each type of claim, with the difference being almost two hours. To give an example of the potential effect of this assumption, if 30% of 2014‑15 claims decided on merit were complex, rather than the projected 10%, about 4,435 more hours of member time would be required to finalize these claims. This is the equivalent of about 3.5 FTEs (approximately 5% of the total FTE complement in 2014‑15).

Another assumption related to the complexity of the claims was that the percentage of total claims decided on merit would be 30% DCO claims and 70% non‑DCO claims. In reality, less than 10% of claims have been DCO claims. DCO claims have expedited timelines. Underlying the shorter timelines was also the belief that these claims could be ready for a hearing in those timelines and would require shorter hearings. Instead, approximately 90% of claims are non‑DCO claims, which are expected to have longer processing times.

Claims are more complicated than predicted as a high percentage of claimants have counsel, there are more ministerial interventions, and the claims have more documentation. These factors mean that RPD members have needed more time to prepare for the hearing and to deliver reasons. Both of these consequences have resource implications for the new refugee determination system.

Recommendation #1: The ability of the IRB to determine its human resource needs and set reasonable finalization targets for the RPD and RAD is complicated given the apparent differences between the working assumptions when the system was being developed and the reported experience two years later. The IRB should consider a limited exercise where members would track their time usage. That information could be used to update resource needs and inform training activities or other decisions related to how to distribute human resources to increase efficiency.

The evaluation found strains in the delivery of the new system. While the volume of cases is lower than anticipated, the RPD and the RAD are not achieving their finalization targets and, for the RPD in particular, an inventory of new claims is growing. The evaluation found that this is due to the challenges in keeping up with the pace and the fluctuations in intake, as well as accommodating the secondary intake of claims returned by the RAD or Federal Court, and applications for vacation or cessation.

Stress and burnout experienced by members was a recurring theme in the evaluation interviews given the pace and volume of the work. For RPD members, the main suggestion to improve efficiency and their work environment was to receive more administrative and adjudicative support. For RAD members, the main suggestion to improve their efficiency was the provision of RPD hearing transcripts.

Recommendation #2: The previous system had more extra‑hearing support for the decision‑making process by assisting members with triaging and preparing files for hearings. The IRB should consider a pilot project to determine if the provision of similar supports would be a cost-effective method of increasing the efficiency of the system in finalizing claims.

Recommendation #3: The IRB should consider a pilot project on the provision of RPD transcripts to the RAD to assess the efficiency gains the availability of transcripts would provide.

Key informants questioned the RPD's current adjudicative approach of deciding cases on one determinative issue in order to streamline and increase the efficiency of decision-making. From the perspective of RPD key informants, this adjudicative approach — while intended to save time and be more efficient — has made their decisions more vulnerable to being set aside or referred back to the RPD by the RAD. Most RAD interviewees also questioned whether the adjudicative approach supports the whole‑of‑board approach. The RAD review cannot focus solely on the one issue but must consider all issues raised on appeal. From the perspective of RAD key informants, the result of the RPD adjudicative approach is less efficient decision‑making, as the RAD has to spend more time on its decision and often has to refer the matter back to the RPD for it to consider other issues.

Recommendation #4: The RPD should review this adjudicative approach and whether it supports a more efficient refugee determination system by considering the approach from the perspective of the entire system and not just the RPD.

  1. 4.1.2 Impacts of other involved departments/agencies

The evaluation evidence identified several factors that are affecting the efficiency of the new system.

  • Scheduling hearings - While having the Citizenship and Immigration Canada (CIC) and the Canada Border Services Agency (CBSA) schedule the initial hearing was intended to help the system meet the new legislated timelines, the IRB has found the system challenging. The hearing booking tool has less functionality than hoped, as it does not interface with NOVA, the IRB's case management system, which would enable more strategic booking of hearings. The result is that the RPD cannot easily take advantage of the efficiencies available through geographic specialization nor can it manage effectively and flexibly members' workload.
  • Front‑end security screening (FESS) - The delays due to pending FESS are down substantially in the second full year of the new system's operations. Both the number of claims delayed and the length of the delay have declined. However, in 2014‑15 over 700 claims were delayed due to a pending FESS and still experienced about a four‑month delay on average.
  • Information from portfolio organizations - The ability of RPD members to efficiently and accurately assess claims is supported by having all the available information. Difficulties in getting the following from CIC and CBSA were identified as having an impact on efficient decision‑making: identity documents, temporary resident visa applications, and responses to information requests.
  • Fluctuating Federal Court jurisprudence - Key informants reported that Federal Court decisions on the appropriate standard of review for the RAD have had an operational impact by slowing down RAD decision-making.

Recommendation #5: The IRB, CIC, and the CBSA should engage in discussions on how to address issues related to scheduling hearings, FESS delays, and information‑sharing. In particular, the discussions should consider the development of a more robust/flexible scheduling tool, improvements to scheduling practices that gives the RPD more control, and more timely and responsive information-sharing. Any such discussions must be conducted in a manner that respects the institutional independence of the IRB.

  1. 4.1.3 Program support services

Registry Services

Prior to the reforms, the IRB had a single registry that served all divisions. In the reforms, the IRB established a separate registry for the RPD. The rationale for the separate RPD Registry was to support the RPD's ability to deliver on its legislated commitments. The belief was that with the new, aggressive timelines, the RPD needed to have more streamlined interactions and reporting with its registry services.

The Registry and Regional Support Services (RRSS) branch includes Common Services, which handles services such as mail, interpretation, transcriptions or recordings, and reception for all of the divisions, including the RPD. It also provides registry services for the RAD, Immigration Division and Immigration Appeal Division. This reporting structure has a clear separation of management of RAD members and registry staff.

Key informant opinion was mixed on whether the separate RPD Registry's benefits outweigh its challenges. The division largely rested on whether the key informants worked within the RPD or not.
RPD management expressed strong support for the separate RPD Registry and believes that it has generally achieved the objectives underlying the rationale for its creation. They commented on the sense of teamwork with the registry that exists now. They believe that the new structure has increased their ability to quickly and effectively address issues that arise concerning how registry processes might affect the work of the RPD.

Key informants, primarily from areas outside the RPD, pointed to challenges created by the separate RPD Registry. Their comments primarily concerned the effect of the separate registries on the potential for a “whole‑of‑board approach”. Under the single registry, staff could be reassigned across teams to respond to work demands, while with separate registries staff mobility is considered more limited. Furthermore, separate registries within the IRB create duplication of work at the organizational level (e.g., human resources, financial management) and each requires its own management structure, which adds to the cost of the IRB registry function.Note 6

Recommendation #6: For the RPD Registry, the main area for improvement is clarifying the roles for staff and more consistent application of those roles. This point is related to the members' need for more support. As the members seek assistance, they tend to expand the roles of registry staff. The IRB should consider as part of Recommendation #2, whether there should be a greater redesign of Registry roles or whether some other method is more suited to providing members with more administrative supports.

Legal Services

Key informants generally provided positive comments on the quality of Legal Services advice and reasons review. A concern expressed by key informants was whether Legal Services has sufficiently adjusted to the focus on efficiency and the need for members to meet tight timelines for making decisions. RPD members are now expected to make more oral decisions and more decisions that are based on one determinative issue. According to some key informants, Legal Services has not made a corresponding shift in its approach and considered how to support members in becoming more proficient at streamlined decisions that will withstand RAD and Federal Court scrutiny.

Research Directorate

Many key informants praised the Research Directorate's country of origin information (COI) work producing and updating the National Documentations Packages (NDPs), which were noted as having an international reputation and being used by other countries. A few key informants suggested improvements to the NDPs. In particular, the length of some country reports were considered “unwieldy,” particularly for members new to handling a refugee claim from the country, who must digest potentially hundreds of pages to prepare for a hearing.

In the fast‑paced environment of the new system, members need timely information when making decisions. The evaluation findings indicate that the Research Directorate generally provides high quality information in its various research products. However, the Directorate is experiencing capacity issues as it adjusts to the timelines of the new system and the changing jurisprudential requirements placed on members.

Recommendation #7: The ability of the Research Directorate to respect the RPD timeline pressure requires the Directorate having the capacity to handle the requests in the tight timelines. In combination with recommendation #2, the IRB should consider undertaking an examination of what needs to be done for the Research Directorate to deliver more timely responses with more focused content to the RPD.

Performance monitoring

The IRB has captured substantial data on the work of the RPD and RAD. The Standards, Analysis, and Monitoring unit (SAM) provides a national picture of key metrics. The SAM reports were considered to provide valuable information, although there was some suggestion that they could include more analysis and more regional reporting. The RPD and RAD regional analysts carry out many tasks that vary by region and division reflecting uncertainty among key informants over how the analyst group can best be deployed to support the divisions.

Recommendation #8: The analyst role is reportedly becoming clearer. However, their duties appear to be quite broad and sometimes tailored more to suit the individual skill set of the analyst than to be based on a strategic decision about what work analysts should do. The IRB should review how the IRB's analytical function can be restructured to enhance its effectiveness.

The evaluation found much frustration among key informants (primarily among RPD members, but also among a few RAD members) related to monitoring member performance, as there is the perception that the performance of the system is based solely on individual member performance. According to key informants, the current performance metrics for members that focus on targets such as the number of hearings and decisions per week provide an incomplete picture of performance. The metrics do not take into account factors such as the complexity of cases, the need to adjourn or postpone a hearing due to factors outside the member's control, or the quality of the member's job performance.

Key informants questioned the individual performance targets. The original targets were not realistic, and while they were subsequently revised, they were still difficult to justify to members given the limited information available on how they were set.

Recommendation #9: The individual performance targets should be set in consultation with members and based on evidence of what is reasonable in the current system.

4.2 Effectiveness

The IRB has created a number of products to inform parties about the new system as well as their rights and obligations. The evaluation findings indicate that parties are generally aware of these products, have used them, and have found them helpful, in particular: research products, the Chairperson's Guidelines, IRB guides, IRB forms, the Claimant's Important Instructions, and practice notices. The website postings of persuasive decisions or other decisions of public interest are used less than the other products, largely because few decisions are posted and the site does not appear updated. The Ready Tours and Information Sessions have been positively received, but there was little information on the level of uptake.

Recommendation #10: The RPD and the RAD should consider whether they should be more active in identifying and posting persuasive decisions or other decisions of public interest on their website. In addition, the RPD and the RAD should evaluate the Ready Tours and Information sessions to obtain a better idea of the uptake and satisfaction with these activities.

The evaluation found that, generally speaking, authorized representatives found the Basis of Claim (BOC) form to be easy to complete and to complement other forms that claimants complete during their eligibility interview. However, when asked whether the form allowed claimants to express their personal information in a clear and logical fashion, approximately one‑fifth of respondents strongly disagreed.  Key informant comments reflected this sentiment, noting that the BOC form did not allow for a clear narrative and that claimants often simply attached a separate chronological narrative to the form.

Recommendation #11: The RPD should review the BOC form to determine if improvements are needed to allow for a clear narrative of the information relevant to the claim.

Initial RPD hearings are typically on time though there was a spike in delays in quarter 4 of 2014‑15. Since the reforms, over 80% of claims have had their initial hearings scheduled within the time limits. However, the results also show the tensions in the system: The timeliness of the scheduling of DCO hearings has improved while that of non-DCO hearings has declined. When a hearing cannot proceed on its scheduled date, a change to date and time (CDT) may be granted. CDTs due to FESS cause considerably long delays, but the volume and duration of FESS-related delays have decreased from 16% of claims in 2013‑14 to 8% of claims during 2014‑15. FESS‑related delays have declined particularly for non‑DCO claimants. With that said, from an overall system perspective, CDTs due to pending FESS pose a larger impact on the overall average processing time (18 days in 2013‑14 and seven days in 2014‑15) compared to CDTs for fairness and natural justice or operational limitations (two days and under‑one day respectively). Approximately half the delays associated with pending FESS occur prior to the IRB receiving the FESS, while the other half of delays occur after the FESS is received by the IRB, which reflects the difficulty of getting delayed cases back on the schedule.

Recommendation #12:  Surges in intake are likely to continue to magnify the operational challenges under the new system and test RPD resilience. Delays in scheduling initial hearings resulting from such surges should be tracked and examined to understand the precise cause of delayed initial hearings during these periods. Equally important, the RPD should consider the downstream effects of such surges on member productivity, in an effort to more fully understand the division-wide impact of these surges.

While the RPD has approached its RPP timeliness targets, it still remains below them. In the current performance measure (80% of claims finalized within 90 days of being expected to proceed), the RPD came within three percentage points of meeting its target as 77% of claims were finalized on time. In the other RPP performance measure of median time to finalize claims, the RPD exceeded its target of 4 months (3.2 months in 2014–15).

Recommendation #13:  In its performance measurement framework that became effective in fiscal year 2015‑16, the IRB conceptualizes FESS as external and outside its control. Yet, data shows that delays occur after FESS confirmation has been received. This part of the delay is due to the operational limitations of the RPD and the IRB should categorize it as such in a revised performance measure. Using this approach, the RPD will be able to provide a more complete assessment of the impact of reforms and the capacity issues it is facing.

While the above measures of timeliness show that the RPD is improving, they do not show the effect of claims not being finalized on time. The importance placed on scheduling the initial hearing means that secondary intake can remain unresolved for a period of time. The growing age of the new inventory reflects key informants' perceptions that while the system is making strides in meeting established targets, the current pace may not be sustainable with currently available resources. As a result, the system may revert to a pre‑reform‑like backlog of claims.

Recommendation #14:  Monitoring and reporting on the backlog should be factored into the RPD's performance measures. The RPD should track and report on the volume and age of the inventory, and analyze and report on the nature of the cases in the inventory and the factors that contributed to these cases ending up as backlog. By including the backlog in its performance measures, the RPD will be able to provide a more complete assessment of the impact of reforms and the capacity issues it is facing.

The RAD is seeing a growing volume of finalizations over the regulatory/performance targets. Based on key informant interviews, the increase in finalizations beyond 90 days is largely due to two main issues: the fluctuating jurisprudence related to the scope of the RAD appeal that caused the RAD to take a hiatus in 2014 to allow appellants to make submissions on this issue; and the workload implications of the increasing volume,Note 7 coupled with relying on recordings rather than transcripts. Given the jurisprudential environment, the situation is too much in flux; no recommendations are offered related to the achievement of performance targets for the RAD. After the evaluation's period of review, the RAD continued to experience shortfalls in its member complement in comparison to growing intake due to pending government appointments and internal reallocation of GIC resources.
In 2013–14, only 3% (n=26) of the RAD's paper‑based appeals were finalized in excess of the regulatory target of 90 days. This changed drastically in 2014‑15 when 36% (n=677) of finalized paper‑based appeals exceeded the 90-day limit, dropping the RAD below its 80% performance target. On average, these in‑excess finalizations were finalized within 134 days — 44 days over the target.

Paper‑based appeals made up 99% of all appeals finalized during the evaluation period. Between 2013‑14 and 2014‑15, the RAD experienced a 45% increase in intake and came close to doubling the number of paper‑based appeals finalized per fiscal year (from 1,095 to 2,032). Reflecting key informant opinion that the RAD was working at capacity, the result of the higher intake was that the RAD took substantially longer in 2014‑15 to process appeals (from filing to finalization – 114 days) compared to 2013‑14 (74 days). Subsequent to the evaluation's period of review, the RAD has continued to face growing intake without a corresponding increase in capacity through either new government appointments or reassignments of members from the RPD legacy initiative.

Knowledge management tools are one way an organization can improve efficiency. For both RPD and RAD members, initial training was important: for RPD members because 80% of them were new; and for RAD members because the RAD was a new decision‑making body for refugee appeal determinations. Members who commented were very positive about the initial training provided to new members and found it to be thorough and effective training for their role. For the RPD, the average number of days of training per member in 2013‑14 was 15.8 days and 8.6 days in 2014‑15. This decline may reflect the reduced need to have training after most members had been serving for about one year, but it may also reflect the other demands placed on members. For the RAD, the average number of days of training per member in 2013‑14 was 14 days and 15.5 days in 2014‑15. 

Interview findings showed that RPD and RAD members generally considered the training sessions to be useful, but they also found it difficult to accommodate training in their schedules without impacting their productivity. The training sessions covered a variety of topics and each region was able to determine what subjects they wanted to cover. Key informants commented that the RPD and RAD are proactive with training, and they stressed the importance of choosing training topics in consultation with members to ensure that they address identified needs. Suggestions for improvement included developing a database of professional development sessions so that they are more easily accessible after the session, and offering more sessions on writing concise decisions and holding focused hearings.

The RPD and RAD provide several types of tools to assist members with their work. In particular, members commented positively on available checklists and country discussion groups. Overall, most personnel believe they have the materials and equipment to do their work, although the percentage is lower than for the IRB as a whole.

  • Training topics chosen in consultation with members;
  • More stable VPN connection to support telework;
  • A database of hearing recordings on the shared network to facilitate telework;
  • Better search capability in the National Reasons Database;
  • Access to a database of RAD decisions for RPD members; and
  • Macros standardized nationally and available in French.

Recommendation #15: The RPD and the RAD should review the suggestions for the improvement of knowledge management tools raised in this evaluation and take action as warranted.

4.3 Best practices, lessons learned, and improvements

Best practices

Key informants identified a number of potential best practices, including the following:

  • Flexibility with staff: The IRB made efficient use of its resources by maintaining GIC capacity to process outstanding legacy cases. Working together with members to take advantage of scheduling efficiencies, while also considering member preferences, was also cited as a best practice.
  • Regional analysis: The IRB undertook mapping exercises in the IRB regions to better understand regional efficiencies and best practices.
  • Co‑location: Co‑locating members, coordinating members, and registry staff in the Central Region were praised as creating an effective approach to improving communication within the division.

Lessons learned

Key informants identified a number of lessons learned about getting new members up to expected levels of performance. While concerns were raised over how the learning curve of new members can contribute to the inconsistent application of divisional rules, others saw how the training of new members poses a key challenge for the IRB, given that it takes a considerable amount of time for members to develop expertise and get “up to speed.” Concerns were raised over how the stressful nature and pace of the work could diminish job satisfaction, lead to burnout, and drive members to seek opportunities elsewhere.

Operational changes

Key informants also mentioned a number of operational changes that could be made to help the IRB achieve results more efficiently or effectively. Examples include the following:

  • Expectations for finalizations: Numerous concerns were raised about the IRB's expectation for finalizations. Concerns were raised over how the unpredictable and complex nature of refugee hearings could be distilled into assumptions about the number of finalizations. There was a general sense that finalization targets were too high for the RPD and that too much attention was being paid to the number of cases completed. One suggestion was to focus on the number of finalizations by team rather than by individual members.
  • More members and more support/flexibility for members: In addition to a need for more members, key informants recommended providing additional administrative support for RPD members, which would allow decision‑makers to focus on the hearing and rendering a decision. Other suggestions included developing a “floater” team of members and administrative staff to fill gaps due to absences, and continuing to build upon flexible work arrangements for members, such as more flexible work hours or working from home if a member does not have a hearing.
  • Modernized technology: Numerous key informants suggested modernizing the IRB's technology, particularly to enable electronic filing and greater use of electronic documents. These changes were expected to lead to greater efficiencies, including reduced transit time for documents and improved flow of information between the IRB and parties. Key informants also believed modernized IT could contribute to improved analytical capability, reporting, and case management.
  • More control over scheduling: There is a strong desire to improve the scheduling of claims. Suggestions range from moving the role of scheduling from the CBSA and CIC to the IRB to implementing more strategic and efficient scheduling practices. Examples of more strategic and efficient scheduling include the use of an assignment court (used by the IAD) to assess and assign cases upfront, or grouping similar cases to be processed by specific members, while other, more complex, cases are handled by other members. This was practiced under the previous system and is believed to promote efficiencies in scheduling and hearings. Improved coordination with Ministers' counsel during the scheduling process was also suggested.

    Notes

    Note 1

    In November 2015, CIC became known as Immigration, Refugees and Citizenship Canada (IRCC). For purposes of this evaluation report, CIC will be used instead of IRCC.

    Return to note 1 referrer

    Note 2

    Y.Z. and the Canadian Association of Refugee Lawyers v. M.C.I. and M.P.S.E.P. (F.C. nos. IMM-3700-13 and IMM-5940-14). Boswell, July 23, 2015; 2015 FC 892. A notice of appeal has been filed in the Federal Court of Appeal (F.C.A. no. A-338-15).

    Return to note 2 referrer

    Note 3

    The PSES is conducted every three years by Statistics Canada on behalf of the Office of the Chief Human Resources Officer.

    Return to note 3 referrer

    Note 4

    NOVA is the IRB's electronic case tracking and reporting system.

    Return to note 4 referrer

    Note 5

    Under the IRPA, only the following persons may represent or advise a person for consideration before the IRB: a member in good standing of the bar of a province (including lawyers or paralegals), the Chambre des notaires du Québec, or the College of Immigration and Citizenship Consultants​.

    Return to note 5 referrer

    Note 6

    While this statement reflects comments from some key informants, the overall cost of the separate registry structure was not included within the scope of this evaluation.

    Return to note 6 referrer

    Note 7

    Y.Z. v. M.C.I and M.P.S.E.P., (F.C., no. IMM-3700-13, IMM-5940-14), Boswell, July 23, 2015; 2015 FC 892).

    Return to note 7 referrer