How we ranked India's Best B-Schools 2013
The BT-MDRA Best B-schools ranking is based on three separate surveys - objective, perceptual, and experiential. Add to it an unprecedented physical audit of data and the first-ever ranking of executive programmes. We have a unique and varied offering this year.
Since it started in 1998, the Business Today Best B-schools survey has seen four distinct avatars. Last year, we continued two powerful surveys to come up with something that was extremely robust and balanced. An objective survey that ranks B-schools based on data they submit, combined with a perception survey, which ranks B-schools based on opinions of key stakeholders such as teachers, students, recruiters, and others. This was an evolution from our earlier approaches of using one of the two methods - objective or perception.
This year, we have added a new facet - called the experiential survey - in collaboration with our knowledge partner, Delhi-based market research firm, Marketing & Development Research Associates (MDRA). The experiential survey brings in the views and experiences of alumni. The idea was to capture the merit of a business school based on how its alumni have fared in the job market. This is a unique facet that has not been captured before in B-school surveys in India, and adds a whole new dimension to the ranking process.
The current methodology, therefore, is the most complete that has been devised so far. The objective survey tells us what kind of facilities and track record each B-school has. The perception survey tells us what key stakeholders feel about various B-schools. And the experiential survey gives us the real deal - have the facilities and the reputation contributed to the students being satisfied in their jobs?
What's New
We've made some other significant changes, too, in the survey this year.
Executive ranking: Till last year, our survey would combine two-year and one-year programmes into one, unified ranking. However, given that one-year or executive programmes are markedly different in nature and in scope, we decided to delink them from the ranking of the basic, two-year MBA programme. Accordingly, a new ranking has been introduced this year, that of one-year executive programmes. This was reached on the basis of a perception survey only, but one in which the colleges were not required to participate. Only the opinions of recruiters and deans/directors of leading B-schools were sought.
Scoring system: In the objective survey last year, schools scored zero on sub-parameters where they did not provide data. It led to some good schools scoring low on some parameters. This year, MDRA devised a unique statistical formula to normalise the data for parameters where schools did not give data. But the maximum limit allowed for such missing data was five per cent of all data. Fortunately, we did not have to use the formula at all, because no school that participated gave less than 95 per cent data. The comprehensive nature of the data we gathered is also due to the extensive follow-ups and audits done by MDRA. Read column by MDRA's Deputy Director Abhishek Agrawal,
businesstoday.in/bschools2013-audit.)
Parameters: Several parameters of the objective survey have been changed, and much new information sought. For example, in "learning experience", two new sub-parameters - "student connect" and "industry connect" - have been introduced. Student connect captures data such as attendance of students, scholarships given by colleges and the industry, etc. Industry connect captured elements such as value of research projects that colleges undertook jointly with industry and government, revenue generated from consulting assignments, etc. New information such as international accreditations, teaching of foreign languages, whether students were trained to do business in a global environment, were also captured. See businesstoday.in/bschools2013-parameters to view the full list of parameters of the objective survey.
Weights: Since the methodology was being overhauled, we felt the need for expert advice on the weights to be assigned to the different parameters. And the experts needed to be from both sides of the academia-industry fence. Accordingly, MDRA's representatives ran a separate survey on a panel of experts - HR heads of reputed companies and deans/directors of reputed colleges - to determine weights of the objective survey's parameters. Since the experiential survey had the same parameters, similar weights were used there too. See Weighty Affair for the list of experts. The weights they individually assigned to various parameters were averaged to get the final weights.
Awareness deflator: Last year, in the perceptual survey, we introduced an "awareness deflator" to level the playing field for schools that had high perceptual scores, but from very few respondents, leading to distorted rankings. We have retained it, though its working has been integrated into the process and is not shown at the final stage.
The process
First, a draft list of parameters and sub-parameters for ranking was prepared and shared with the experts' panel. Based on their inputs, the final parameters, sub-parameters and their relative weights were determined.
Next, a comprehensive list was prepared of schools that offered two-year full-time management courses (PGDM/MBA), and had been established at least five years ago with a minimum of three batches having passed out. A separate list of schools offering one-year full time Executive MBA/PGP programmes was also prepared. An objective questionnaire, its design based on the finalised parameters, was put up on the websites of BT and MDRA. More than 1,500 institutes that qualified were requested, through emails, calls and advertisements in the magazine, to participate in the objective survey. In all, 216 colleges across the country submitted their objective data on time, of which 213 were ranked. Three institutes have not been considered because three batches had not graduated.
More than 130 institutes declined to participate, including prominent ones such as IIM Bangalore and IIM Lucknow.
Simultaneously, the questionnaire for the perception survey was designed and face-to-face interviews conducted with 1,091 key stakeholders including deans/directors, recruiters, alumni and final year students of PGDM/MBA. The sample design used MDRA's "3S" criteria - sample size, spread and selection criteria of the respondents. The perception survey was carried out across 16 cities - Delhi-NCR, Chandigarh, Mumbai, Chennai, Kolkata, Kochi, Bangalore, Pune, Indore, Hyderabad, Jaipur, Ahme-dabad, Lucknow, Bhubaneshwar, Guwahati and Ranchi.
The respondents comprised 369 final year MBA/PGDM students, 110 deans/ directors, 312 alumni and 300 recruiters with rich experience. Recruiters and deans/directors were also asked to rate one-year programmes. Only colleges offering 11to15 months executive MBA/PGP programmes were considered for ranking.
Then, data was captured for the experiential survey from alumni of only those colleges which had submitted their objective data. A separate questionnaire was sent to the alumni asking them to rate the colleges on various sub-parameters in view of their contribution to career growth. In cases where we did not receive alumni response about their alma mater, colleges were given marks in proportion to their objective and perceptual score.
The challenge we faced here is that the expectations of someone from a top IIM would be very different from that of one going to an average, regional B-school in a remote part of the country. So, instinctively, a student from an average school would give high ratings to his college even if he got a low-paying job. Comparatively, an IIM graduate would be dissatisfied with anything less than top-dollar. Attitudes and expectations, therefore, could throw the rankings completely out of whack. To avoid such fluctuations, the experiential scores were adjusted against the colleges' combined objective and perceptual score. This approach helped balance expectations with reality.
The second phase involved physical verification and audit of objective data provided by B-schools. Seventy-five B-schools were visited by MDRA's researchers, who cross-checked the data with the original documents and infrastructure. Irregularities were removed/corrected in the final objective data. For other schools, MDRA's researchers carefully examined every aspect of the details provided by B-schools through available information and validation (past data, telephonic check, website visits, etc.).
The marks of the five parameters of the objective phase were then summed to get the total objective score by each institute. To arrive at the final composite score, objective, perception and experiential scores were added in weights of 50 per cent, 40 per cent and 10 per cent, respectively.
The MDRA core team was led by Deputy Director Abhishek Agrawal, and comprised Research Manager Abnish Jha and Research Executive Barkha Soni.
This year, we have added a new facet - called the experiential survey - in collaboration with our knowledge partner, Delhi-based market research firm, Marketing & Development Research Associates (MDRA). The experiential survey brings in the views and experiences of alumni. The idea was to capture the merit of a business school based on how its alumni have fared in the job market. This is a unique facet that has not been captured before in B-school surveys in India, and adds a whole new dimension to the ranking process.
The current methodology, therefore, is the most complete that has been devised so far. The objective survey tells us what kind of facilities and track record each B-school has. The perception survey tells us what key stakeholders feel about various B-schools. And the experiential survey gives us the real deal - have the facilities and the reputation contributed to the students being satisfied in their jobs?
What's New
We've made some other significant changes, too, in the survey this year.
Executive ranking: Till last year, our survey would combine two-year and one-year programmes into one, unified ranking. However, given that one-year or executive programmes are markedly different in nature and in scope, we decided to delink them from the ranking of the basic, two-year MBA programme. Accordingly, a new ranking has been introduced this year, that of one-year executive programmes. This was reached on the basis of a perception survey only, but one in which the colleges were not required to participate. Only the opinions of recruiters and deans/directors of leading B-schools were sought.
Scoring system: In the objective survey last year, schools scored zero on sub-parameters where they did not provide data. It led to some good schools scoring low on some parameters. This year, MDRA devised a unique statistical formula to normalise the data for parameters where schools did not give data. But the maximum limit allowed for such missing data was five per cent of all data. Fortunately, we did not have to use the formula at all, because no school that participated gave less than 95 per cent data. The comprehensive nature of the data we gathered is also due to the extensive follow-ups and audits done by MDRA. Read column by MDRA's Deputy Director Abhishek Agrawal,
businesstoday.in/bschools2013-audit.)
Parameters: Several parameters of the objective survey have been changed, and much new information sought. For example, in "learning experience", two new sub-parameters - "student connect" and "industry connect" - have been introduced. Student connect captures data such as attendance of students, scholarships given by colleges and the industry, etc. Industry connect captured elements such as value of research projects that colleges undertook jointly with industry and government, revenue generated from consulting assignments, etc. New information such as international accreditations, teaching of foreign languages, whether students were trained to do business in a global environment, were also captured. See businesstoday.in/bschools2013-parameters to view the full list of parameters of the objective survey.
Weights: Since the methodology was being overhauled, we felt the need for expert advice on the weights to be assigned to the different parameters. And the experts needed to be from both sides of the academia-industry fence. Accordingly, MDRA's representatives ran a separate survey on a panel of experts - HR heads of reputed companies and deans/directors of reputed colleges - to determine weights of the objective survey's parameters. Since the experiential survey had the same parameters, similar weights were used there too. See Weighty Affair for the list of experts. The weights they individually assigned to various parameters were averaged to get the final weights.
Awareness deflator: Last year, in the perceptual survey, we introduced an "awareness deflator" to level the playing field for schools that had high perceptual scores, but from very few respondents, leading to distorted rankings. We have retained it, though its working has been integrated into the process and is not shown at the final stage.
The process
First, a draft list of parameters and sub-parameters for ranking was prepared and shared with the experts' panel. Based on their inputs, the final parameters, sub-parameters and their relative weights were determined.
Next, a comprehensive list was prepared of schools that offered two-year full-time management courses (PGDM/MBA), and had been established at least five years ago with a minimum of three batches having passed out. A separate list of schools offering one-year full time Executive MBA/PGP programmes was also prepared. An objective questionnaire, its design based on the finalised parameters, was put up on the websites of BT and MDRA. More than 1,500 institutes that qualified were requested, through emails, calls and advertisements in the magazine, to participate in the objective survey. In all, 216 colleges across the country submitted their objective data on time, of which 213 were ranked. Three institutes have not been considered because three batches had not graduated.
More than 130 institutes declined to participate, including prominent ones such as IIM Bangalore and IIM Lucknow.
More than 130 institutes declined to participate, including IIM Bangalore and IIM Lucknow
Then, data was captured for the experiential survey from alumni of only those colleges which had submitted their objective data. A separate questionnaire was sent to the alumni asking them to rate the colleges on various sub-parameters in view of their contribution to career growth. In cases where we did not receive alumni response about their alma mater, colleges were given marks in proportion to their objective and perceptual score.
The challenge we faced here is that the expectations of someone from a top IIM would be very different from that of one going to an average, regional B-school in a remote part of the country. So, instinctively, a student from an average school would give high ratings to his college even if he got a low-paying job. Comparatively, an IIM graduate would be dissatisfied with anything less than top-dollar. Attitudes and expectations, therefore, could throw the rankings completely out of whack. To avoid such fluctuations, the experiential scores were adjusted against the colleges' combined objective and perceptual score. This approach helped balance expectations with reality.
MDRA Deputy Director Abhishek Agrawal (front left), Deputy Research Manager Abnish Jha (front right), Research Executive Barkha Soni and Senior Research Executive Avanish Kumar (PHOTO: Sounava Ray Sarkar)
The second phase involved physical verification and audit of objective data provided by B-schools. Seventy-five B-schools were visited by MDRA's researchers, who cross-checked the data with the original documents and infrastructure. Irregularities were removed/corrected in the final objective data. For other schools, MDRA's researchers carefully examined every aspect of the details provided by B-schools through available information and validation (past data, telephonic check, website visits, etc.).
The marks of the five parameters of the objective phase were then summed to get the total objective score by each institute. To arrive at the final composite score, objective, perception and experiential scores were added in weights of 50 per cent, 40 per cent and 10 per cent, respectively.
The MDRA core team was led by Deputy Director Abhishek Agrawal, and comprised Research Manager Abnish Jha and Research Executive Barkha Soni.
