How do Programmers Ask and Answer Questions on the Web?

Stackoverflow.com is a popular Q&A website featuring questions and answers on a wide range of programming topics. In the 2 years since its foundation in 2008, more than 1 million questions have been asked on Stack Overflow, and more than 2.5 million answers have been provided.

Stack Overflow — like many other Q&A websites such as Yahoo! Answers, Answers.com or the recently created Facebook Questions — is founded on the success of social media and built around an “architecture of participation” where user data is aggregated as a side-effect of using Web 2.0 applications.

To understand the role of Q&A websites in the software documentation landscape, Ohad Barzilay, Peggy and I wrote a paper in which we pose research questions and report preliminary results to identify the role of Q&A websites in software development using qualitative and quantitative research methods. The paper was just accepted at the NIER (New Ideas and Emerging Results) track of ICSE 2011 in Hawaii.

We pose the following five research questions:

  1. What kinds of questions are asked on Q&A websites for programmers?
  2. Which questions are answered and which ones remain unanswered?
  3. Who answers questions and why?
  4. How are the best answers selected?
  5. How does a Q&A website contribute to the body of software development knowledge?

For the NIER paper, we focused on the first two questions. We created a script to extract questions along with all answers, tags and owners using the Stack Overflow API. We then analyzed quantitative properties of questions, answers and tags, and we applied qualitative codes to a sample of tags and questions.

Our preliminary findings indicate that Stack Overflow is particularly effective at code reviews, for conceptual questions and for novices. The most common questions include how-to questions and questions about unexpected behaviors.

Understanding the processes that lead to the creation of knowledge on Q&A websites will enable us to make recommendations on how individuals and companies, as well as tools for programmers, can leverage the knowledge and use Q&A websites effectively. It will also shed light on the credibility of documentation on these websites.

A pre-print of the paper is available here
(© ACM, 2011. This is the author’s version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version will be published in the Proceedings of the International Conference on Software Engineering (ICSE) 2011.)

This is the abstract of the paper:

Question and Answer (Q&A) websites, such as Stack Overflow, use social media to facilitate knowledge exchange between programmers and fill archives with millions of entries that contribute to the body of knowledge in software development. Understanding the role of Q&A websites in the documentation landscape will enable us to make recommendations on how individuals and companies can leverage this knowledge effectively. In this paper, we analyze data from Stack Overflow to categorize the kinds of questions that are asked, and to explore which questions are answered well and which ones remain unanswered. Our preliminary findings indicate that Q&A websites are particularly effective at code reviews and conceptual questions. We pose research questions and suggest future work to explore the motivations of programmers that contribute to Q&A websites, and to understand the implications of turning Q&A exchanges into technical mini-blogs through the editing of questions and answers.

Advertisements

7 thoughts on “How do Programmers Ask and Answer Questions on the Web?

  1. I think that this is an interesting area that you have ventured into and I have been wondering about it..can I get a link to the paper ?

  2. How do you plan to deal with the critical mass? Even though your approach seems to point to novice programmers, a website do not get expert programmers just to answer novice questions (that is the value of StackOverflow).

  3. In this YouTube video, Stack Overflow founder Joel Spolsky talks about critical mass as one of their nine design decisions: http://www.youtube.com/watch?v=NWHfY_lvKIQ (starting at 26:30).

    There are lots of expert programmers on Stack Overflow, but we found (probably not surprisingly) that questions by novices have a higher answer ratio than questions by experts.

    Over the years, several sub-communities with lots of experts have emerged around specific topics on Stack Overflow. This happens particularly for topics that don’t have a community somewhere else (yet).

Comments are closed.