mixedmath

This is the second in a miniseries of posts on internet fora, and Math.SE and StackOverflow in particular. In the previous entry in the miniseries, I described some of the common major problems facing community cohesion. I claimed that when communities get large, they tend to fracture and the ratio of meaningful communication to noise plummets. To combat this tendency, communities use some mixture of core moderation, peer moderation, membership requirements, or creating subcommunities/splitting off in to other communities.

In this chapter I focus more on Math.SE and StackOverflow. Math.SE is now experiencing growing pains and looking for solutions. But many users of Math.SE have little involvement in the rest of the StackExchange network and are mostly unaware of the fact that StackOverflow has already encountered and passed many of the same trials and tribulations (with varying degrees of success).

Thinking more broadly, many communities have faced these same challenges. Viewed from the point of view from the last chapter, it may appear that there are only a handful of tools a community might use to try to retain group cohesion. Yet it is possible to craft clever mixtures of these tools synergistically. The major reason the StackExchange model has succeeded where other fora have stalled (or failed outright) is through its innovations on the implementation of communition cohesion strategies while still allowing essentially anyone to visit the site.

Imaginary Internet Points

Slashdot^{1 1}Known to have News for nerds. Stuff that matters, and also the first tech news site that I was aware of. popularized the idea of associating imaginary internet points to different users. It was called karma. You got karma if other users rated your comments or submissions well, and lost karma if they rated your posts as poor. But perhaps most importantly, each user can set a threshold for minimum scores of content to see. Thus if people have reasonable thresholds and you post crap, then most people won't even see it after it's scored badly.

As Joel Spolsky says, ^{2 2} in his post on gamification on StackOverflow.

What reputation and karma do is send a message that this is a community with norms, it’s not just a place to type words onto the internet. (That would be 4chan.) We don’t really exist for the purpose of letting you exercise your freedom of speech. You can get your freedom of speech somewhere else.

Astoundingly, karma even contributes to some sort of community cohesion when there are no benefits or detriments to having karma. See reddit, where karma is on the one hand almost worthless^{3 3}To be fair, reddit karma does modify how long something stays visible on the page and how close to the top a post will be in a subreddit or comment section when ranked by "best". But the amount of karma a user has only indirectly affects their reddit experience , and on the other hand highly valued. Even sought after.

Seriously, if you look you can found thousands upon thousands of people asking how to get more reddit karma (and a much smaller number asking what it's good for).^{4 4}Getting useful nuggests like Post a picture of a cute cat to /r/cats.

I credit StackOverflow with popularizing the idea that imaginary internet points can be used as a formal (rather than informal) indicator of community standing. SO even calls it "reputation". As a user gains more reputation points, they are given more peer moderation abilities. A user gains the ability to upvote^{5 5}It is an innovation in itself to not let just anyone cast votes. , downvote, edit any post, close/reopen posts, or even delete/undelete posts.

This has worked astoundingly well. But it's not perfect.

The New User Experience

Very often I hear the same sort of story. A new user comes to ask a question, but it gets downvoted and negatively commented immediately. Then a moderator comes in and closes or deletes the question. And if even the mods are against new users, then what are they to do? That isn't so welcoming, is it?

I frequently look into these cases and find a slightly different backstory. What usually happens is that several very high reputation users decided to close/delete the question with a somewhat minimal comment, such as This is a duplicate of [this other question] or What have you tried? or RTFM. The source of the confusion is that these high rep users have lots of moderator powers that new users don't have. At first I thought that this distinction was important: it's not the mods that are unwelcoming new users; it's just some high rep users.

But then I realized that to a new user, this distinction is completely meaningless. The typical new user doesn't care about their own reputation or badges or even the community itself — they just want an answer to a question. Any any obstacle in their way (like reading a How to Ask a Question page or comments saying Use MathJax) are pitfalls to be navigated through on the way to the goal. The fact is that they wanted help with something, went to get some help, only to feel like they were shut down.

This has been a major complaint about StackOverflow for years. In 2012 StackOverflow tried to reform new user culture through their Summer of Love initiative (Summer of Love, aka the Hunting of the Snark. Goal: keep SO welcoming and friendly without lowering standards).^{6 6}It is not a coincidence that the Summer of Love campaign occurred a few months after Joel Spolsky took over from Jeff Atwood as SO Chief. This change coincided with a change in focus from Great Questions to Lots Of Questions.

Did it work? Not quite. In fact, the opening post on SO's meta site^{7 7}Each StackExchange site has a corresponding "meta" site where people discuss the status of the site itself, as opposed to the topic the site is about. So meta.Math.SE concerns topics about Math.SE, whereas Math.SE is about mathematics. generated so much bickering and negative commentary that it was deleted.

Other ideas were tried, but they had at most temporary success. A few years later John Slegers wrote a highly viewed post The decline of Stack Overflow, documenting the standard negative first impression received by new users, and how even older users can be at the whims of Privileged Trolls. These Privileged Trolls are those high reputation users who user their powers extensively.

Not Without Reason

Why doesn't StackOverflow ban/suspend/quell the class of Privileged Trolls? In short, it's because they're not wrong. Most often these so-called Privileged Trolls are seeking to combat low quality questions and the existence of Help Vampires^{8 8}Commonly described: The Help Vampire problem is the idea that some users will continually ask the same tired questions in the hope that someone else will do their work for them, irrespective of whether the same question has already been asked or whether they could easily find the solution elsewhere. .

There exists a class of high reputation user who is very frequently on the site, has a corner that they care about, and is very familiar with the majority of content in that corner. We might optimistically call them a Caretaker instead of a Privileged Troll.

A typical bad scenario might go as follows. A new user comes and asks Why is this python program hanging?. A Caretaker sees the question, recognizes that the user was trying to pull stdin from an IDE, and then marks the question as a duplicate of some question about how to get around this. In the abstract, does this answer the question? Yes. But to the new user who is following some tutorial and doesn't even know what import means yet, this is probably unhelpful or confusing.

In Math.SE, this problem might be further exacerbated by the fact that there are high power mathematical results that quash all sorts of weaker statements. But having your introductory real analysis question about How do I show that this function is integrable? closed as a duplicate of some question which states that Any function which is almost everywhere continuous is Riemann integrable is most definitely unhelpful (and yet similar occurrences definitely occur).

Camps of Users

Caretakers are trying to maintain high site quality. One aspect of quality is the ratio of signal to noise, and the existence of a vast number of duplicate questions is a source of noise. Using the enumeration from this answer to the meta.SO question Why is StackOverflow so negative of late?:

Basically there are 4 camps of users on Stack Overflow:

1. The "caretakers" who want to keep the site clean and with good content.
2. The "help vampires" who flood the site with bad/duplicate questions who only want their question answered and care nothing for the site.
3. The "repwhores" who answer everything they can (or can't).
4. The ones who no longer give a shit.

For the most part: 2 and 3 love each other. They should get married. 1 hates 2 because they're flooding the site making good questions impossible to find. 1 hates 3 because they're encouraging 2 to keep going. 2 hates 1 because 1 constantly downvotes/closes/deletes/flames 2. 3 hates 1 because they keep closing/deleting the questions that 3 likes to answer. 1 and 3 have all the moderation powers, but only 1 cares to use them. 4 is sitting on the sideline shaking their heads... 1 hates 4 because 4 isn't helping the situation. With so much hate, there's going to be conflict.

There are too many moderators (both true mods and very-high-rep-users) for a single common viewpoint to dominate the others. And from my point of view, a central division is over the purpose of a StackOverflow. Is it to

1. Quickly get people great answers to their programming questions, or to
2. Serve as a repository of useful programming knowledge.

In many ways these work together. Providing great answers to new questions serve both. But repeatedly answering the same question (especially with slightly different answers) makes the site less useful as a repository of knowledge — a visiting user may need to check several variants of a question to find an answer that works for them. Why not just ask another variant instead, adding to the tidal wave of similar questions? Conversely, requiring users to interpret a canonical question and answer in their own situation is annoying, especially to novice users who don't know enough to recognize alternate phrasings of the same topic.

I believe the intent of the site was the latter, but somehow a large minority of users much prefer the former.

On Math.SE, there is perhaps a third category. One can ask whether the purpose is to

1. Teach people mathematics,
2. Answer mathematical problems from all levels, or to
3. Serve as a repository of useful mathematical knowledge.

I think the reason why Math.SE cares so much about teaching mathematics is that many of the veteran users are (or have been) educators (teachers, professors, teaching assistants, lecturers, etc.). But similarly to the StackOverflow case, I believe the founders had the last option in mind, but frequent answerers are often interested in actually teaching people mathematics.

Despite the apparent difference, most cultural problems appear to be the same. Or rather, since Math.SE is a bit younger and a bit smaller than SO (but still the second largest site on the StackExchange network), the cultural problems facing Math.SE are a mix of the current problems facing SO and the problems from a few years ago.

Parallels

Let us now dive into parallel responses between StackOverflow and Math.SE.

StackOverflow

1. Why is "Can someone help me?" not an actual question? Response summary: the site is intended to create a knowledge repository of solutions to programming problems. When you ask a question, make sure you actually ask a question.
2. Why the backlash against poor questions? Response summary: bad questions are noise while good questions and answers are the signal. If the signal is drowned out by the noise, then people interested in answering questions go away, leaving behind people asking questions.
3. Can we adopt a stop-whining-about-bad-questions policy? Response summary: No. Bad questions = noise. Constantly seeing the same question will lead to people not answering anymore and leaving.^{9 9}There are hundreds of posts commenting on the rising tide of bad questions.
4. Should trivial re-occurring questions really be answered? The response is complicated. As long as answers to these questions will be upvoted, then there are incentives to answer them (and therefore the asker, even if downvoted, will probably get the answer they were looking for). Some suggest downvoting answers to bad questions to remove the incentive structure. But that's quite a complicated thought process. It is also noted that there is a dichotomy between the Atwood Keep Question Quality Really High to Optimize for Pearls vs the Spolsky Ask Any Question As Long As It Hasn't Been Asked policy.^{10 10}Referring to the two major founders of StackOverflow, Jeff Atwood and Joel Spolsky.
5. Should one give advice on off topic questions? The upvoted response is to downvote and close off-topic questions, and to absolutely not help or advise as this incentivizes poor questions.
6. Off topic questions have to be cleared out of the way, but NOT via closure. The theme of this post is that the current reputation system incentivizes people giving answers to poor questions, which in turn inventivizes people asking poor questions. The responses have an interesting theme: most say that hoping for an ideal site where people don't answer low-quality questions is probably a waste of time (perhaps even counterproductive), even though there is definitely a real problem there. Others advise users to downvote low-quality posts.
7. Should SO be awarding As for effort? This is really about people asking questions and others saying "This doesn't show enough effort to merit a good response" and the related viewpoint that questions with lots of effort shown do deserve a good answer. The answers hit a really wide set of contradictory opinions, and reading this question and its answers gives good insight into different trains of thought on the topic.
8. How to ask and answer homework questions?

From these topics, you may get the impression that there is a central response to downvote good answers to low-quality questions, as that is frequently advertised as a central method to maintaining high-quality content. But then you read Is it okay to downvote answers to bad questions? and see that the overwhelmingly upvoted response here is No, it's not okay to downvote good answers to bad questions. But in fact the subtler issue here is that As long as users don't engage in vote fraud, they can vote however they want. There is also a rebuttal by Brad Larson that notes that targeting downvotes at people who answer low quality questions will most likely drive those frequent answerers away (definitely undesirable); further, he doesn't believe the assumption that making people stop answering bad questions will make bad questions stop coming.^{11 11}This is a point of view I agree with.

Thus both identifying low-quality content and deciding how to prevent them are almost entirely unresolved. In practice, there are people who downvote low-quality questions and answers to low-quality questions (Caretakers), and people who upvote them, and people who answer them, with the dynamics described by the various Usercamps above.

Math.SE

It should be noted that on Math.SE, the vast majority of low-quality questions (and indeed, the majority of all questions), are from students of mathematics trying to learn new material. A typical question comes either from a suggested or assigned problem from an instructor, or from a math book that someone is trying to understand or solve an exercise from. So on math there is a big conflation between "homework", "cut-and-paste" questions, and "low-quality".

With that noted, these problems (and mostly their suggested responses) also appear independently on Math.SE.

1. Why isn't more being done to avoid facilitating copy paste homework questions?
2. Can I try to tell experienced users to not answer bad questions?
3. What to do when other users answer low quality questions?
4. Dealing with zero effort questions
5. Howto deal with just-google-it questions
6. Have the questionson Math.SE changed in quality?

As with SO, many responses suggest downvoting low-quality posts more, that there really is a problem, but that the problem may not be solveable. Trying to prevent experienced users from answering bad questions may be a waste of effort (or a noble effort), and these should be ignored (or upvoted, or downvoted).

And if you think that there is a recurring suggestion to downvote or delete low-quality content, then one would be going against the (upvoted and respected) thought process behind the answers to Downvoting complete solutions.

There simply isn't consensus on these issues, or on What the purpose of Math.SE is.

What's to be done?

One major takeaway from the above discussion is that there are real problems facing Math.SE and SO, and these problems stem from underlying problems that are essentially unresolved. There isn't consensus on the purpose of the site or how to deal with low quality questions (or even if they're a real problem).

Does that mean that trying to resolve these problems is a waste of time? No! In fact StackOverflow has implemented a variety of tools not (yet) present on other sites in the network that can help some of these problems. (And these don't have anything to do with the recent StackExchange blog post suggesting to make SO a more welcoming community.

A recent suggestion that gained some traction on meta.Math.SE was to introduce another site to the network where novice mathematical questions are welcome.

In the next chapter, I will say why I think NoviceMath.SE is a bad idea (but that there are some changes that can be made now that will relieve some of the tension on the site.

Leave a comment

Comment via email