HomeMy WebLinkAboutExhibit 4JOB ANALYSIS OF
POLICE OFFICER
CITY OF MIANlT POLICE DEPARTMENT
i
Prepared by: Testing and Validation Division
Department of Human Resources �
City of Miami
September 2000
TABLE OF CONTENTS
Page
1. Job Analysis
Overview........................................ ......................3
2. Job Analysis
Methodology— ............................................................. 7
3. Job Analysis
Results....................................................................10
Appendix A: Police Officer Job Analysis Questionnaire-
Appendix
uestionnaireAppendix B: Police Officer Job Analysis Questionnaire Results
Appen and kesults
I
O VERV1EW
JOB ANALYSIS OF POLICE OFFICER
CITY OF MIAMI POLICE DEPARTMENT
This report describes the rationale, procedures, and results that pertain to the
job analysis of the Police Officer position in the City -of .Miami Police
Department. The job analysis was undertaken as a prerequisite to the
development of tests that are to be- used for selection into the position. This
report is presented in three sections: (1) Job Analysis Overview; (2) Job
Analysis Methodology; and (3) Job Analysis Results. The following section
will provide an overview as well as the rationale for the job analysis process.
1. JOB ANALYSIS OVERVIEW
The job analysis identifies the critical tasks and requisite knowledge, skills,
and abilities (KSAs) required to perform those tasks. Additionally, the job
analysis establishesthe lifkage e ween tile La=ni
basis for linking the tests to the tasks and KSAs.
The relationship between job analysis and test validation is established in
several documents. These documents are as follows:
1) Society for Industrial and Organizational Psychology, Inc. (1987).
Principles for the Validation and Use of Personnel Selection
Procedures. (Third edition) College Park, MD: Author.
This document specifies principles of good practice in the choice,
development, evaluation, and use of personnel selection procedures.
2) American Educational Research Association, American Psychological
Association, & National Council -on Measurement in Education.
(1999). Standards for Educational and Psychological Testing. Wash-
ington, DC: Author.
OVERNUIV
This document provides criteria for the evaluation of tests, testing
practices, and the effects of test use. It is a technical guidethat can be
used as the basis for evaluating testing practices.
3) .Equal Employment Opportunity .Commission. (1978). Uniform
Guidelines on Erilployee Selection Procedures. Washington, DC:
Author.
This document incorporates a single set of principles that are designed
to assist employers, labor organizations, employment agencies, and
licensing and certification boards to comply with requirements of
Federal law prohibiting employment practices that discriminate .on
grounds of race, color, religion, sex, and national origin. The
document provides a framework for determining the proper . use. of
tests and other selection procedures.
ep icuars i n
analysis methodology, procedures, and data and their relationship to content -
oriented validation are presented below:
1) Society for Industrial and Organizational Psychology, Inc. (1987).
Principles for the Validation and Use of Personnel Selection
Procedures. (Third edition) College Park, NO: Author.
Pp. 4-5: "Before a selection procedure is considered, or a validation
effort is planned, the researcher should be able to make a clear
statement of the objective of the procedure. The statement of purpose
must be based on an understanding of the work performed -on the job
and of the needs and rights of the organization, and its present and
prospective employees."
P. 5: "Job analysis is essential to the development of a content -
oriented procedure or to the justification of a construct considered
important to job behavior:" _ .. .........
4-
OVERVIEW
P. 6: "Inferences about the worker characteristics are made by the
researcher or job experts by combining knowledge of the work
performed and what workers have to do to perform the work."
2) American Educational Research Association, American Psychological
Association, & National Council on Measurement in Education.
(1999). Standards for Educational and Psychological Testing_ Wash-
ington, DC: Author.
P. 11: "Evidence based on test content can include logical or
empirical analysis of the adequacy with which the. test content
represents the content domain and of the relevance of the content
domain to the proposed interpretation of test scores. Evidernce...based
on content can also come from expert judgments of the relationship
between parts of the test and the construct."
Standard 1.6: When the validation rests in part on the appropriateness
of test content, the procedures followed in specifying and generating
test content should be described and justified in reference to the
construct the test is intended to measure or the domain it is intended to
Tepresent. If the definition of the content sampled incorporates
criteria such as importance, or criticality, these criteria should also be
clearly explained and justified.
Standard 14.8: Evidence of validity based on test content requires a
thorough and explicit definition of the content domain of interest. For.
selection, classification, and promotion, the characterization of the
domain should be based on job analysis.
Standard 14.9: When evidence of validity based on test content is a
primary source of validity evidence in support of the use of a test in
selection or promotion, a close link between test content and job
- -content shoizld-be demonstrated.--
Standard 14.10: When evidence of validity based on test content is
present, the rationale for defining and describing a specific j ob content
5
OVERVIEW
domain in a particular way (e.g., in terms of tasks to be performed or
knowledge; skills, abilities or other personal characteristics) should be
stated clearly.
Standard 14.11: If evidence based on test content is a primary source
of validity evidence supporting the use of a test for selection into a
particular job, a similar inference should be made about the test in a
new situation only if the critical job content factors are substantially
the same (as is determined by a job analysis), the -reading level of the
test material does not exceed that appropriate for the new job, and
there are no discernible features of the new situation that. would
substantially change the original meaning of the test material.
3) Equal Employment .Opportunity Commission. . (1978). Uniform
Guidelines on Employee Selection Procedures, Washington,, DC:
Author.
Section 0n--rev7
information about the job. Any validity study should be based upon a
review of information -about the job for which the selection procedure
is to be used. The review should include a job analysis_..
Section 14 (C2): Job.analysis.for content validity. There should be a
job analysis which includes an analysis of the important work
behavior(s) required for successful performance and their relative
importance and, if the behavior results in work product(s), an analysis
of the work product(s). Any job analysis should focus on the work
behavior(s) and the tasks associated with them. if work behavior(s)
are not observable, the job analysis should identify and analyze those
aspects of the behavior(s) that can be observed and the observed work
products. The work behavior(s) selected for measurement should be
critical work behavior(s) - and/or important work behavior(s)
constituting most of the job.
I
JOB ANALYSIS METHODOLOGY
2. JOB ANALYSIS METHODOLOGY
Job analysis information may 'be developed in a number of ways, for
example, by direct observation, by performing a job directly, by
interviews with subject matter experts (SMEs), by examination of written
documents, and by questionnaires. Several of these methods, with the
exception of direct j ob performance, were used to conduct the current job
analysis of the police officer position. The job analysis procedure was a
sequential one in which -data collected in one stage were reviewed and
processed for use in subsequent stages.
In brief, the objectives of the job analysis were: (1) to identify the tasks
that are important and frequently performed on the job; (2) to identify the
essential knowledge, skills, and abilities (KSAs) required to perform
these tasks effectively and that are needed on the first day on the Jab, and.
(3) to link the essential. KSAs to task groups and tasks that require them.
ass that survive ese proce ures are erne cri ica . n e
aggregate, this pool of critical tasks and essential KSAs provides
guidance to test' experts regarding the most appropriate assessment
procedures and test content for the particular job under study.
Development of Job Analysis Data Base: Overview
This section presents a general overview of the methodology used in the
job analysis. Specifics, in terms of actual tasks, KSAs, and results will
be provided in the following section of this report.
Step 1: Panel Discussions
The first step in the process was to hold panel discussions with
incumbents for the purpose of editing and modifying job analysis
questionnaires that were developed based on experiences in other police
departments.
JOB ANALYSIS METHODOLOGY
Step 2: Job Analysis Questionnaire Development.
Based upon information obtained in Step 1, a Job Analysis Questionnaire
(JAQ) was constructed for the Police Officer position in the City of
Miami Police Department. The JAQ gathered information in areas, as
follows:
1) General Background: Questions were asked regarding current
assignment, years in rank, -years with the City of Miami Police
Department, status in the department, ethnic group, and gender.
2) Job Tasks: A list of tasks was presented and the respondent was
asked to indicate (1) the frequency of task performance, (2) how
important the task was to effective performance, (3) the time spent on
each task, and (4) the physical effort needed to perform the tasks_.
3) Task Group Percentages: This section identified and defined task
groupings groups of similar tas-Zs-y7-Tiae-candidate'responded to the
task groups in terms of percentages of importance and time spent
categories, with each category summing to 100%.
4) Knowledge: This section listed knowledge that may be required to
perform the tasks in the position. The respondent needed to indicate
(1) the importance of the knowledge for effective performance, (2)
whether the knowledge is needed at the time of appointment, prior to
entering training, and (3) the level of recall required of that knowledge
at the time of appointment.
5) Abilities: This section listed abilities that may be required to
perform the tasks in the position.. The respondent needed to indicate
(1) the importance of the ability for effective performance, and (2)
whether the ability is needed at the time of appointment.
6) Supplemental Questions: This section listed various physical
activities (e:g.; restraiziiiig 'are'stirig -subject that may be required to "
be performed in certain circumstances , or under certain conditions.
The respondent needed to indicate the percentage of time or frequency
that the activities were performed.
N.
JOB ANALYSIS METHODOLOGY
7) Additions: Finally, the respondent was asked to indicate the degree
to which the tasks, knowledge, and abilities presented in the JAQ
reflected the job by circling a percentage value that ranged from 0% to
100%. Also, if tasks, knowledge or abilities .were missing, the
respondent could add them in the space provided.
Analysis of the information provided in the JAQ is intended to identify
the -most critical tasks and KSAs in the job.
Step 3: Development of Task Group Linkage Questionnaire
Criteria are applied to the information gathered with the JAQ (Step 2) in
order to generate the information needed for the ne>,Tt step, development
and administration of the Task Group Linkage Questionnaire.. This
questionnaire is designed to gather information on whether the critical
KSAs .(as identified from the above Step 2 analysis) are essential, useful,
or not ne tor peg ormance of tasks in the task groups. -This -
questionnaire thus identifies the most essential KSAs for the position
and establishes a link between KSAs and task groups.
0
JOB ANALYSIS RESULTS
3. JOB ANALYSIS RESULTS
At the time of the study, there were 866 permanent Police Officers in the
City of Miami Police Department, of which 313 (36%) participated in the
job analysis process. The job analysis plan called for collecting data from a
representative sample of the permanent incumbents. The sample was to be
representative in terms of job assignment, and ethnic and gender
composition of the Police .Officer's position. Across the phases, the SME
composition included 37 Caucasian (12%), 51 African-American (16%), and
158 Hispanic (51%); 216 male (69%) and 30 female (10%) Police Officers.
Sixty-seven participants' (21%) did not provide information regarding
ethnicity and gender. The participant representation can be compared to the
ethnic composition of all Police Officers in the City of Miami Police
Department, which is 16% Caucasian, 30% African-American, and 54%
Hispanic; 81 % Male and 19% Female.
In February, 1999, panel discussions were held with a total of 5 Police
Officers (members of this group were representative of both genders and
three ethnicities) for the purpose of revising and reviewing the job analysis
questionnaires that had been developed based on experiences in other police
departments. Participants reviewed the existing - -JAQ statements and
indicated which statements needed to be edited to reflect City of Miami
Police Department terminology or practices. Participants also generated
additional task, knowledge, and ability statements to reflect City of Miami
Police Department practices. Finally, participants examined task groupings
to determine if they adequately represented the major components of their
i obs.
As a result of the panel discussions, the Miami Police Department Police
Officer JAQ contained 276 tasks, 13 task groups, 119 knowledge areas, 142
abilities and 31 supplemental questions. The 13 task groups are: (1)
Preparing for Work, (2) Patrolling the District, (3) Responding to Calls for
Service, (4) Conducting Investigations, (5) Com.muniiy Policing, (6) Making
Arrests/Detaining Suspects, (7) Pursuing Suspects, (8) Enforcing
Traffic/Parking Violations, (9) Report Writing/Record Keeping,
10
JOB ANALYSIS RESULTS
(10) Interacting with Superiors, Peers, and Other Department Personnel, (11)
Preparing/Participating in Planned Events, (12) Preparing for/Attending
Court, and (13) Participating in Self -Development. A copy of the JAQ
appears in Appendix A.
Step 2: JAQ Analysis
The JAQ, the second phase of the job analysis process, was administered to
212 incumbent Police Officers. This sample included a representative
sampling with regard to gender and ethnicity. Accordingly, 21 Caucasian
(10%), 31 African-American (15%), and 96 Hispanic (45%); 132 male
(62%) and 16 female (8%) Police Officers employed with the City of Miami -
participated in this phase of the process. Sixty-four participants (30%) did
not provide ,information regarding ethnicity and gender. All participants
were categorized as "permanent" and the average time as a Police ;Officer
exceeded 5 years.
Tasks and KSAs
The responses to the JAQ sections on tasks and KSAs were analyzed
according to the following criteria. For a task to be considered critical it
had to meet one of the following criteria:
1. it had to have a standard deviation of 1.50 or less, and an
importance mean of 3.0 or greater;
or
2. it had to have an importance rating of 2.0 through 2.99 and a
mean frequency rating of 6.0 or greater, and 'a standard
deviation of 1.50 or less.
JOB ANALYSIS RESULTS
For a knowledge or ability to be considered critical, it had to meet the
following criteria:
1. 60% of the respondents had to indicate that the knowledge or
ability was "needed on Day I";
2. it had to have an importance mean rating of 3.00 or greater and
a standard deviation of 1.50 or less.
The results for each individual task and KSA statement are shown in the
printouts of the analyses of the JAQ and are presented in Appendix B. The
essential data in these printouts are the means and standard deviations, for
task statement importance and frequency; the means and -standard deviations
for knowledge and ability importance; and the "valid percent" for
knowledge/ability "needed day I" and knowledge "recall .level" responses.
When reviewing the results in Appendix B it is important to first familiarize
oneself -Mth the abbreviations used to label the variables in the printout. As
such, TF, TI, TT, and TP are labels for task frequency, importance, time
spent, - and physical effort, respectively. KI, KD, and KR represent
knowledge importance, whether it is needed day one, and recall level. needed
day one, respectively., Al and AD represent ability importance and whether
it is needed day one, respectively.
Note that the responses from Police Officers who indicated that they did not
perform a specific task were excluded from the analyses for the frequency of
the task as well as from analyses of the other scales used to assess tasks.
Likewise, if a respondent indicated that he/she viewed a knowledge or
ability as `'Not Important," his/her responses were not included in other
analyses of scales for knowledge and ability statements. Finally, a number
of respondents either did not complete a section or incorrectly completed
sections of the JAQ. Thus, sample sizes vary within a section .as well as
across sections.
12
JOB ANALYSIS RESULTS
Application of the criteria mentioned above resulted in the retention of 263
task statements (all numbered _statements on the JAQ [1-276] with the
exception of: 10, 15, 36, 37, 40, 68, 73, 74, 78, 207, 224, 225, and 263).
Note that tasks survived for all 13 Task Groups.
For the 263 above referenced tasks,. 27 (10%) were reported to require a
moderate level of physical effort. For these tasks (numbered 16, 48; 49, 75,
76, 77, 78, 81, 83, 84, 85, 132, 133, 157, 173, 175, 176, 177, 178, 180, 181,
191, 194, 195, 197, 199, and 224), the task groups Preparing for Work,
Responding to Calls for Service, Conducting Investigations, Making
Arrests/Detali ng Suspects, Pursuing Suspects, and Enforcing
Traffic/Parking Violations accounted for 1, 10, 2, 8, 5, and 1 tasks,
respectively.
For the knovaledge areas, 7 met the "criticality" retention criteria: of
"importance" and "needed day 1" (statements numbered 5, 13, 15, 75, 79,
90 and 110)These results are surprising in that it is expected that Police
Officers will be trained on the knowledge. needed where they enter the
position and are not expected to bring that knowledge with them to the
position. The one explanation for these results is that the respondents may
have misunderstood the JAQ directions; respondents were to respond with
respect.to what knowledge is needed prior to entering training. Based on an
understanding of the Police Officer position by the Human Resources
Department as well as Subject Matter Experts, and based on experience
conducting job analyses the knowledge areas cited should not be
emphasized in test development.
In order to confirm this interpretation of the fundings an additional section
was added to the final phase of the job analysis. This additional section
consisted of the list of seven remaining knowledge areas and requested the
SME to respond in a "Yes/No" fashion as to whether each knowledge was
needed prior to police training. This brief "Yes/No" questionnaire was
included with the Task Group Linkage Questionnaire (see Appendix C). -
Responses to this questionnaire indicated that for each knowledge statement,
virtually .all (i.e., 83' to 91) ofthe 96 participating incumbents (87%o to 95%0)
considered each knowledge not to be needed prior to training.
II
JOB ANALYSIS RESULTS
For the ability areas, 71 met .the "criticality" retention level mentioned above
(statements numbered: 2, 4, 5, 6, 7, 9, 14, 15, 16, 21, 22, 23, 24, 25, 29, 30,
34, 35, 36, 38, 39, 41, 42, 45, 46, 47, 48, 49, 50, 51, 52, 57, 59, 60, 61, 62,
83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 98, 101, 107, 108, 109,
110, 112, 113, 1.15, 116, 118,. 121, 123, 124, 125, 126, 127, 128, 129, 130,
and _,131). These results appear consistent with other police officer -job
analyses.'
Task Group'Weigghts
The average task group weights for the 13 Task Groups is shown in Table 1.
The most important task groups were Patrolling the District, Responding to
Calls for Service, Report Writing/Record Keeping; Making
Arrests/Detaining Suspects, Preparing for Work,. and Conducting
Investigations. These task groups accounted for more than 60% of the
relative importance and relative time spent ratings.
14
JOE ANALYSIS RESULTS
TABLE 1
TASK GROUP WEIGHTS
Relative
Importance
Relative
Time Spent
Mean
Standard
Deviation
Mean
Standard
Deviation
8.7
5.43
Preparing for Work
8.9
5.73
12.3
9.17
Patrolling the District
13.1
9.25
11,9
10.14
Responding to Calls For Service
11.7
9.08
8.6
9.40
Conducting Investigations
7.6
7.68
7.5
6.88
I Community Policing
7.7
6.77
9.4
6.75
Making Arrests/Detaining Suspects
8.5
5.37
J
5.1 I
4.03
Pursuing Suspects
5.1
3.87
6.1
4.61
Enforcing Traffic/Parking Violations
6.5
5.67
957
8.02
Report Writing/Record Keeping
10.7
8.33
5.9
4.89
Interacting with Superiors, Peers, and
other Department Personnel
6.3
4:05
3.5
4.18
Preparing/Participating in Planned Events
3.5
3.91
5.2
3.93
Preparing for/Attending Court
5.5
4.20
6.3
7.42
Participating in Self Development
4.9
4.81
100.0
TOTAL
100.0
1C)
JOB ANALYSIS RESULTS
Supplemental Questions
The job analysis questionnaire included a set of supplemental questions that
dealt specifically with physical aspects of the job: Accordingly, these
questions concerned actions such as pulling, pushing, carrying, and
physically manipulating individuals and objects during the course of one's
duties. Unfortunately few respondents provided meaningful data for these
questions and it is likely that additional information will be required in order
to assess the physical requirements of the City of Miami Police Officer
position.
Additions
Respondents indicated that an overwhelming majority of the position (83%)
was covered by the JAQ, though only 110 respondents (52%) provided
meaningful data, It appears that. many respondents did not respond to this
section or misunderstood what information -was requested. 'For example, if a
respondent indicates that less than 100% of the job is represented by the
JAQ, that respondent should indicate what needed to be added; many did not
do so. Review of the "additions" provided by the respondents indicated
trivial tasks or KSAs or rewording of existing statements.
Step 3: Task Group Linkage Questionnaire
Based on the analysis for the JAQ (Step 2, above) the critical tasks,
knowledge,. and abilities were included in the Task Group Linkage
Questionnaire for the Police Officer position. Appendix C contains the Task
Group Linkage Questionnaire; it also contains the listing of tasks, by. task
group. This latter listing shows the critical tasks that survived the
previously mentioned a priori -criteria for retention of tasks; the Task
Group Linkage Questionnaire itself can be used to see the knowledge
and abilities that survived the analysis of the JAQ.
--Ninety-six -(96) ' Police ""Officer incumbents- completed the Task Group
Linkage Questionnaire. The gender and ethic composition of this group was
consistent with that of the City of Miami Police Department. Accordingly,
the SME composition included 15 Caucasian (16%), 18 African-American
16
JOB ANALYSIS RE' S11TS
(19111c), and 60 Hispanic (62%); 82 male (85%) and (11).female (12%) Police
Officers. Three participants (3%) did not provide information regarding
ethnicity and gender. To analyze the data for the questionnaire, an a priori
rule was established to identify the "essential" knowledge and abilities. The
criterion applied to the Task Group Linkage Questionnaire is that a KSA is
retained for further analysis if 57 of the respondents (60%) indicate that
the KSA is "essential" for a task group. Appendix C also shows the
matrix of knowledge and abilities and their linkages to task groups. (The
"Y' in any cell of the matrix indicates that at least 60% of the respondents
rated the KSA as "essential" to that task group). Results indicate that 71 of
the 142 abilities that survived the analysis of the JAQ were considered
"essential" for at least one of the task groups. Only the "essential" ability
statements will be considered for test development purposes.
The contribution- of the Task Group Linkage Questionnaire analysis is that
the results show which specific knowledge and abilities are linked to which
specific task groups These data contribute to the development of tests and
the establishment of the content validity of the subsequently developed tests.
17