My Experience with Academic Job Search (Search Committee's Perspective)

Let's look at the Academic Search from Search Committee's Perspective:
Before you start, you already know that the ad has gone out and candidates have applied. The search committee has pretty good idea what kind of role is expected from the chosen candidate:

• When search committee is formed, they will first come up with the timeline for the search. They will then decide the criteria for evaluations (what to evaluate and how to score) .... This could include: 
• degree/qualification (what, where, when), 
• teaching experience (courses taught/TA-ed), 
• research (quality, quantity), 
• area (of research/teaching), 
• and may also include professional development, recommendations, quality of teaching/research statement, potential growth, and much more depending on specific position and the expected role. 

• Each committee member will then evaluate all applicants based on the criterion and then they will meet to pick the top 10-12 candidates after discussion (the number may change based on committees time and other factors).... committee chair will then set up the phone / skype interviews.
• Phone Interviews are mostly about confirming the details that you have read on the application pages and getting to know the candidates a little more to decide whether they are worth bringing in for the visit. Committee will have a set of questions prepared for each candidate (usually the same questions are used for all applicants for the sake of consistency). At the end of this process, committee members will meet to rank the top 6 or so candidates based on their strengths and fits for the job. Although you usually bring in 3 or 4 top candidates but sometimes you may have to bring in extra if someone drops out or if schedule doesn't work!
• On-campus Interviews will be setup by the committee chair. The visit should be arranged in a way that the candidate can present teaching/other demos, can talk formally/informally with faculty/ administrators/students, can eat/relax, etc. It's not just about evaluating the candidate, but also about giving the candidate opportunity to get to know the campus/faculty. Remember at the end of the day, they have to accept the offer and if they are comfortable with the school, their role, moving to new area, etc. - that is the desired outcome.
• After the interviews, the committee members share/discuss their evaluations and observations. The chair will write a detailed summary with strengths and weaknesses of each candidate and their recommendations will then be sent to higher-ups for final decision. They will make the final decision and make the offer.

The search process is not an exact science. The committee members all have different opinions, ideas and different perspectives. Each member will evaluate the same candidate in his/her own way which may not be the same. At the end of the day candidate's profile on paper may be very different than the ranking. It is committee's job to ensure that best candidates are interviewed - now the definition of 'best' may evolve during the search process! The hope is that candidates offered the position will accept it and they will not only do a good job but also stay at the job for a long time!