Sunday, November 27, 2011

Top Web technology conferences based on citation data

Last week HCI researcher Antti Oulasvirta has an interesting post on ranking HCI conferences using the average citations per paper based on data from Microsoft Academic Search (MAS). Some of the results surprised him, including that the venerable CHI was not the top conference in this group.

WWW, ISWC and WebDB are the top Web conferences based on Microsoft Academic Search citation data
His ranking metric for conference significance is essentially the impact factor (IF) used for journals, a measure of the average number of citations a paper in a given journal receives in a time period. The IF metric has become widely used in the scholarly journal publication industry since it was defined by Eugene Garfield and first implemented by the company he founded, the Institute for Scientific Information.

Oulasvirta used data from Microsoft Academic Search (MAS), which provides citation and publication numbers for conferences in sixteen different subjects domains and a number of sub-domains for each. For computer science, there are 24 sub-domains including one for "World Wide Web" conferences.

Following Oulasvirta, we ranked Web technology conferences using the average number of citations received in the last ten years. Starting with 68 Web technology conferences (not a complete list, btw), we narrowed the set to those that had at least 100 papers in the past ten years and some papers in the past five. This resulted in 26 conferences, eliminating many series that only ran a few times or have stopped. Here are the results.







The results should only be taken as a rough estimate of conference impact. One reason is that IF is only a measure and does not take into account all aspects of scientific importance. For example, as computed here, all citations count equally, including those from high- and low-ranking sources. Another is that while Thompson-Reuters (nee ISI) journal citation data is carefully collected and curated, the Microsoft Academic Search data is the result of a largely automated process that starts with data from Bing. When I tried using the citation information from the past five years, for example, I noted that it reported 23 papers in the past five years for Adaptive Hypermedia and Adaptive Web-Based Systems. This is because the conference merged with User Modeling in 2009 to become User Modeling, Adaptation, and Personalization. Yet another shortcoming is that the MAS list of Web conferences in not complete, for example, omitting the popular ESWC, which has been running since 2004.

The original excel spreadsheet (with full conference names hidden in column B) and a PDF version are available from the UMBC Ebiquity site.

Friday, November 18, 2011

Elsevier press release on 2011 Semantic Web Challenge winners

Elsevier published a press release on the awards for the winners of the 2011 Semantic Web Challenge.
"Elsevier, a world-leading provider of scientific, technical and medical information products and services, today announced the winners of the Semantic Web Challenge. Participants in the Challenge applied semantic web techniques in building online end-user applications that integrate, combine and deduce information needed to assist users in performing tasks. The Challenge took place at the International Semantic Web Conference (ISWC 2011) in Bonn, Germany, from October 23-27, 2011."