Apply Now

Senior Psychometrician

Chicago, Chicago, Illinois, United States of America Req #60
Wednesday, June 5, 2019
The mission of the American Osteopathic Association is to advance the distinctive philosophy and practice of Osteopathic Medicine.


As a member of the Certifying Board Services (CBS) department, the Senior Psychometrician will work closely with the 18 certifying boards and 9 conjoint examination committees in all aspects of examination development. In addition, the Senior Psychometrician coordinates with the Director of Psychometrics and Assessment on activities associated with the certifying boards’ exams and other assessment or psychometric initiatives.


A. Participates in the design, development and maintenance of assessments
B. Conducts item and test analyses, test equating, standard setting, score reporting, and develops technical reports
C. Assists with reliability and validity studies and other psychometric activities to support the certifying boards
D. Assists with any and all AOA psychometrics initiatives including but not limited to basic and applied research in psychometrics and/or other medical certification related research for submission to educational conferences, scholarly journals, or other venues
E. May coordinate with psychometric contractors on specialized projects
F. Facilitates meetings with subject matter experts/examination committees
G. Communicates psychometric results and findings to members of the certifying boards
H. Collaborates with subject matter experts to improve current assessments
I. Conducts scholarly research
J. Assembles and submits proposals for presentations at testing and measurement conferences


Education:  Doctoral degree required (Ph.D., ED.D or Psy.D) in educational measurement, psychological measurement, evaluation, assessment or a related field with an emphasis on psychometrics.

Experience:  A minimum of five (5) years of experience in psychometrics, measurement, analytics, or high-stakes testing required.  Experience in the development and scoring of examinations for certification in the healthcare field is preferred.
Licensure or Certification:  N/A.

Systems:  Highly proficient with computer programs and applications including MS Office and database management and data manipulation is required.  Knowledge and demonstrated use of and statistical software (e.g., SPSS, SAS, R, Winsteps or other IRT software) is required.

Communication: Ability to prioritize conflicting goals while negotiating in a persuasive manner during group decision making. This would generally occur during item review calls and the evaluation of the suitability of new or equated pass points to guide subject matter expert decisions that are congruent with psychometric and assessment best practices.  Excellent verbal and written communication skills are required with a demonstrated ability to simplify complex information and craft messages for a wide range of audiences.  

Special Skills/Equipment and Attributes: Expert knowledge and track record of success with qualitative and/or quantitative data handling, knowledge of assessment standards, testing methodologies, data validation and analysis and reporting is required. This position requires the ability to interact tactfully and in a sensitive manner with professionals on all levels within and outside the AOA.  Key competencies also include:

• Solid understanding of complex data structures and the ability to manage, manipulate and summarize small and large quantities of data
• Working knowledge of statistical theory, IRT with an emphasis on Rasch models, research methodologies, and applied measurement
• Self-starter, organized, versatile, capable of performing work independently with minimal direction
• Excellent problem solving skills and ability to prioritize simultaneous, competing client activities
• Takes personal responsibility for service and quality of work delivered
• Excellent communication skills in both written and spoken English
• Strong interpersonal skills with the ability to in team leader and team member roles
• Detail oriented, with strong time and project management skills


This position may require remote work while traveling.  This work may include weekday and weekend travel.  Extended workday hours may be necessary around major meetings and other high-volume work periods.

Opportunities for remote work will be considered

Physical requirements include the ability to remain in a stationary position, the ability to move supplies and equipment up to 25 pounds and the ability to operate office equipment.


This position description reflects the assignment of essential functions.  It does not restrict additional tasks that may be assigned.  This job description is subject to change based on the needs of the organization.

AOA is an Equal Opportunity Employer who fully and actively supports equal access for all people regardless of Race, Color, Religion, Gender, Age, National Origin, Veteran Status, Disability, Genetic Information or Testing, Family and Medical Leave, Sexual Orientation and Gender Identity, expression or any other characteristic protected by applicable federal, state, or local laws.  We prohibit retaliation against individuals who bring forth any complaint, orally or in writing, to the employer or the government, or against any individuals who assist or practice in the investigation of any complaint, or otherwise oppose discrimination. We strongly encourage women, minorities, individuals with disabilities and veterans to apply to all of our job openings.

Other details

  • Pay Type Salary
  • Travel Required Yes
  • Required Education Doctorate
  • Chicago, Chicago, Illinois, United States of America