Automatic Publications Ranking for Research Institutions

Communications of the ACM, 50(9), September 2007, p. 14

This is a revised version of the original letter because of refinements in the measurement.[1]

The original version (May 2007) can be found here.

In their paper entitled “Automatic and Versatile Publications Ranking for Research Institutions and Scholars,” in Commun. ACM (June 2007, pp. 81 ff.), Jie Ren and Richard N. Taylor show that automatic publications ranking can lead to results similar to those from manual processes [2]. Although the authors warn for the sensitivity of the measurement for parameter choices, they suggest that the reproduction of the rankings validates the use of the instrument for quality assessment.

In addition to numbers of publications, citations can be used for the ranking. Fourteen of the seventeen journals used by these authors (based on [3]) are also included in the Science Citation Index-Expanded Version:

Table 1: Fourteen journals in the SCI-Expanded Version included in the analysis

ACM Transactions on Database Systems

ACM Transactions on Programming Languages And Systems

ACM Transactions on Graphics

ACM Transactions on Information Systems

ACM Transactions on Computer Systems

ACM Transactions on Software Engineering And Methodology

ACM Transactions on Modeling And Computer Simulation

IEEE Transactions on Computers

IEEE Transactions on Knowledge And Data Engineering

IEEE Transactions on Parallel And Distributed Systems

IEEE Transactions on Pattern Analysis And Machine Intelligence

IEEE Transactions on Software Engineering

IEEE Transactions on Very Large Scale Integration VLSI Systems

JOURNAL of the ACM

A publication and citation count of these journals for precisely the same period (1995-2003) leads to different rankings. Table 2 provides the extension of the table with publication and citation rates for the 50 programs under discussion; the correlation coefficients between the original scores and these new ones are provided in Table 3.

Table 2: Top 50 US computing graduate programs as provided in Ren & Taylor (2007, at p. 84)

extended with citation and publication rates using the ISI journal set.

[1]	[4]	Score		ISI publications	Citations	c/p ratio	First Author	Citations	c/p ratio
1	2	87	Massachusetts Institute of Technology	169	4801	28.4	113	3617	32.0
2	1	79	University of Maryland, College Park	149	2134	14.3	89	1223	13.7
3	6	78	Carnegie Mellon University	147	3027	20.6	96	2385	24.8
4	19	73	Georgia Institute of Technology	104	1093	10.5	68	740	10.9
5	7	70	Stanford University	123	2110	17.2	78	1246	16.0
6	3	64	University of Illinois, Urbana-Champaign	142	1511	10.6	86	793	9.2
7	4	63	University of Michigan, Ann Arbor	125	842	6.7	75	445	5.9
8	5	61	University of Texas, Austin	88	788	9.0	57	583	10.2
9	10	59	Purdue University	97	980	10.1	61	632	10.4
10	11	47	University of California, Berkeley	98	2552	26.0	62	1429	23.0
11	26	46	University of California, San Diego	147	3042	20.7	71	1191	16.8
12	12	45	University of Massachusetts, Amherst	75	950	12.7	50	728	14.6
13	30	44	Rutgers University, New Brunswick	71	1102	15.5	50	689	13.8
14	9	43	University of Southern California	110	1441	13.1	62	690	11.1
14	15	43	University of Washington, Seattle	90	1015	11.3	56	716	12.8
16	21	40	Cornell University	69	1799	26.1	43	1192	27.7
16	13	40	University of California, Santa Barbara	68	1029	15.1	48	812	16.9
18	32	39	Michigan State University	83	2765	33.3	59	2372	40.2
19	20	38	University of California, Irvine	95	1095	11.5	54	593	11.0
20	16	36	University of Minnesota, Minneapolis	79	549	6.9	43	402	9.3
21	8	35	University of Wisconsin, Madison	66	874	13.2	44	550	12.5
22	22	31	Princeton University	65	994	15.3	43	792	18.4
22	31	31	Columbia University	40	548	13.7	28	393	14.0
24	14	29	Ohio State University	64	694	10.8	34	414	12.2
24	34	29	University of Florida, Gainesville	45	736	16.4	34	219	6.4
26	37	28	Pennsylvania State University	65	704	10.8	38	466	12.3
26	27	28	Texas A&M University	59	259	4.4	35	147	4.2
26	18	28	University of Pennsylvania	39	458	11.7	27	298	11.0
29	50	27	University of Texas, Dallas	45	292	6.5	24	197	8.2
30	29	26	State University of New York, Stony Brook	42	463	11.0	35	369	10.5
31	93	25	Oregon State University	43	488	11.3	26	381	14.7
31	17	25	University of California, Los Angeles	56	369	6.6	32	183	5.7
31	51	25	University of Virginia	42	279	6.6	28	220	7.9
34	65	24	California Institute of Technology	53	1114	21.0	30	902	30.1
34	25	24	University of Arizona	44	595	13.5	29	395	13.6
34	24	24	University of Illinois, Chicago	50	457	9.1	31	316	10.2
37	43	23	State University of New York, Buffalo	36	421	11.7	26	205	7.9
38	38	21	Rice University	33	340	10.3	24	237	9.9
38	40	21	Louisiana State University	38	293	7.7	24	137	5.7
38	68	21	Washington University in St. Louis	28	197	7.0