A Silicon Valley love triangle: Hiring algorithms, pseudo-science, and the quest for auditability

Area of Science:

• Socio-technical systems analysis
• Algorithmic auditing
• Hiring technology

Background:

• Algorithmic decision-making systems (ADSs) are increasingly used in hiring.
• Existing auditing practices for ADSs often overlook underlying assumptions.
• Regulatory landscapes for AI in hiring are rapidly evolving.

Purpose of the Study:

• To develop a matrix for auditing hiring ADSs.
• To provide a socio-technical assessment framework for these systems.
• To critically evaluate the assumptions and knowledge claims embedded in hiring ADSs.

Main Methods:

• Development of a socio-technical assessment matrix.
• Analysis of hiring ADSs within current and proposed regulatory contexts.
• Contextualization within emerging algorithmic hiring practices.

Main Results:

• The matrix surfaces underlying assumptions justifying ADS use.
• It reveals the forms of knowledge or insight ADSs purport to produce.
• It highlights the importance of assessing the well-foundedness of design intentions.

Conclusions:

• The matrix can expose pseudo-scientific assumptions about human capability in hiring ADSs.
• It facilitates critical investigation of auditing standards that ignore these assumptions.
• A socio-technical approach is vital for robust auditing of hiring algorithms.