Selection and Assessment: Are we moving with the times?

So, here we are nearly ten years on from the first publication of the IATA Guidance Material and Best Practices for Pilot Aptitude Testing, putting selection in the spotlight, generating discussion and in some quarters, irritation. Nikki Heath looks at how best to select a future workforce.

It is generally accepted that new recruits should be evaluated for skills and processing ability, for assurance and ethical reasons. The Guidelines suggested that all pilots should be assessed for a range of employability markers: the ability to cope with the demands of the job, be good communicators and provide evidence that they were self-disciplined, resilient and professional.

However, there was some sucking of teeth when it was suggested that a similar approach should be applied to Captains or more experienced First Officers; in this case a relevant Type Rating, an interview with a pilot and a simulator session was generally considered sufficient.

Now, it is accepted that all candidates, whatever their level will expect to complete psychometric testing as well as an interview with a non-pilot to assess non-technical skills. But it is an interesting thought that at the time of the Guidelines publication, aviation was still battling a recession, there were more pilots than jobs and it was much easier to be selective. The resistance to implementing new process then was fundamentally based around cost and resources; equally, on-line testing and video interviews were not established, so testing was more labour intensive. For some, selection was considered a tick box exercise, others saw value in ensuring that they got the right people into their employment, who would be safe, reliable operators.

Industry accepted a more inclusive selection process, but the world continued turning and economic pressures eased; airlines ordered more aircraft and the industry started to flourish. Aviation’s concerns and challenges shifted to finding sufficient young people who wanted to be pilots: who will fly this expanding fleet given their career expectations and the competition for other career options? The new generation of aircraft also brought a change in the skills of the job and with it the cause of incident/accidents, i.e. Upset Recovery, Monitoring issues and Startle Effect.

Next, aviation took a more sinister turn, when accidents were caused by deliberate pilot acts; no need to dwell on the details but the outcome has been a potentially seismic shift in attitudes to pilot well-being. The impact of 9/11 and the economic forces driving the job, have resulted in more routine, more isolation and more uncertainty on a day-to-day basis. The responsibility and performance expectations, combined with challenges in lifestyle and job security terms are not glamorous, pay benefits and opportunity are not the same. Uncertainty and not feeling in control are two of the most common forms of anxiety and stress for humans, when combined with financial and family pressures it means resilience and effective coping strategies are now a priority.

At the cadet end of the market, testing needs to include trainability in terms of skills, knowledge and attitude. Aviation has a responsibility to young people and their families who are taking on debt that not only will they pass training, but also that they are employable at the end of it. This now includes Abnormal Psychologies, to gauge whether candidates might pose a risk to themselves or others as they progress through training and into the demands of operations; also now true for experienced pilots. This is a tricky topic. Many well-being tests are designed for self-assessment rather than for selection purposes; they require honesty about issues such as alcohol, drugs, mental state and relationships. Is anyone needing employment likely to be totally honest in this situation? It is counter-intuitive. It is hard enough when you are in employment to self-report personal issues to colleagues or seniors, but to ask someone to self-report when applying for a job is not reasonable or reliable. There are some good developmental questionnaires available that are non-intrusive and supportive; when they are augmented/validated by other means they can be good predictors of resilience and well-being and the management of the data collected is not controversial.

Selection is a form of risk identification. Legally, a risk is based on whether a situation could have been predicted by the average, reasonably well-informed person. Historically, some airlines chose the option of not testing certain things so that they would not have knowledge of a potential risk; in today’s world this is no longer an acceptable argument or legal position. The emphasis of responsibility has shifted: an employer of those engaged in safety-critical roles has a Duty of Care to their employees and customers; to ensure that to the best of all reasonable knowledge, their staff are fit for purpose both mentally and physically. There is a growing expectation that airlines “are to” be aware of their employee’s state of mind, to provide support and act on behaviour that is raising concerns. If airlines and training organisations are to be compliant with this expectation, selection must look at skills and competencies, behavioural fit for the role and organisation, and now mental well-being and stability. How can that be managed ethically and reliably without getting into a legal tangle if the candidate, who is not an employee has recorded some abnormal results? Add General Data Protection Regulation (GDPR) into the mix and there is a significant responsibility laid at the doors of an industry that has attraction problems and at the moment, is being squeezed in many different ways on cost.

Current Selection Process

A loose methodology has evolved consisting of on-line tests, an interview often based on the Situation Task Action Results (STAR) method, a Group/Team session and then either a simulator session or some form of aptitude testing depending on the level of applicant. It would appear that this method has a satisfactory level of success for identifying people who will complete training and who are motivated and well informed about the role, but may not be so good at picking up who could be high maintenance employees or those that could pose a risk.

The STAR process is trainable, and many candidates are taught how to pass it, so often it does not elicit the behaviours that may be relevant to modern airlines and can be masked by the formulaic interview approach.

The ability to monitor and respond well to unexpected situations – the startle effect – is a strong predictor of ability to manage Upset Recovery; the ability to know about attitude regardless of orientation. Research has shown a link between extrovert/introvert profiles versus field dependency; someone who is field dependent is more likely to get disoriented and this has been shown to be more prevalent in extroverts than introverts. Introverts also are usually able to focus attention for longer periods and need less stimulus and external distraction to keep them “in the loop”.

Recruits should be attracted to the role if they like well-defined frameworks and want clear boundaries, to be well prepared, to have practiced and rehearsed skills. This profile is well suited to the EFIS cockpit and competencies of a modern-day pilot, they can stay focussed for long periods of time and are rule-based. However, they may not be the ones that react well when presented with sudden, unexpected, unrehearsed situations. They have well-formed mental models of what a situation should look like, therefore when this expectation is violated their ability to regroup and understand what’s happening takes longer than someone with less well-formed ideas. However, people who are more reactive and respond well to uncertainty are often less successful in formal aviation assessments. They can be seen to be less conformist, who may make unforced errors through lack of structure, (the slips and lapse error model which focusses on a person-centred cause of error), but often can be more resilient.

So, the job has changed, people have changed, aircraft and economics have changed, the emphasis on the type of accident prevention has changed; therefore selection and assessment should change to meet these demands. How should industry respond given that the need to attract the right people, retain them, remain operational and, if lucky, faintly profitable to boot? So first things first – get the right people into the job of the future.

Marketing

Pique the interest of applicants who are enthused by the demands of sitting inside a large, fast moving airborne computer, not adjacent to it on terra firma (not yet!). Find the people who enjoy the analysis and decisions required, but also flourish when this is conducted in a well-bounded framework where feedback, rules and process are well-defined, with a well-constructed career path, then retention is likely to be longer, resilience becomes more likely as they are happier and fulfilled. Market the job honestly, phrase the role in modern terms of job benefits and skill sets that mean something to 18 to 25-year olds of 2020; the romantic “Leonardo di Caprio” model is no longer reality!

Profiling

Creating your Company Profile to act as the blueprint for your selection process is an activity that can offer real value - well worth the investment. Just like a Training Needs Analysis will identify what to train, how to train and how frequently, so a Profile will act as a route map for the attributes that are important for your airline and will help identify and prioritise the risks associated with employing specific candidates. However, the Profile must be seen as a living document that is reactive to change. Selection methods should be closely mapped to the Profile, which identifies who you need, why they will work for you and what attributes they must/must not have. Keep it simple.

Profile Considerations. Credit: Author.

Profiling Considerations

Assessments need not be more expensive or require more resources but may benefit from a slightly different approach to get relevant results:

Prioritise self-awareness, reflectiveness, self-esteem and communication openness, which are recognised as being the strongest indicators for robust, mature attitudes and stable behaviours.

Future-Proofing

Summarising, we have seen several, significant changes over the last decade that should have influenced selection and assessment processes and outcomes. The next decade will bring further challenges, issues and change that aviation should respond to. If selection and assessment is to be seen as a value-added process, it must be fluid and reactive; a living and dynamic procedure that adapts to changing environments: technology, socio-economic shifts etc., if it is to be relevant to industry’s needs. The changes should not just be added to current requirement, and thereby the resource bill.

Base your selection requirements on your needs as identified in your Profile and as priorities change remove elements that are no longer sufficiently important to your organisation. Establish sound processes today, but accept that they must be reviewed regularly, amended where appropriate and improved, to reflect the changes that will occur in time.

Selection is here to stay and, done right, it can be informed, valuable and future proof!

Published in CAT issue 2/2019