Censorship gone awry on Reddit: the aftermath of our r/science AMA
You may remember my announcement earlier this week of our AMA (Ask Me Anything) on Reddit Science. Overall, the AMA, which was about Sci-Hub and our recent study on its coverage, was a success and generated lot's of interesting discussion:
From the outside, it looks like everything ran smoothly. The AMA generated 65 comments and has a score of 116 (upvotes − downvotes). There are plenty of high-impact quotes, but my favorite comes from coauthor Thomas Munro, who wrote:
lucaxx85's questions themselves illustrate how paywalls raise costs, by allowing authors to externalize these ruinous costs to society: a vast public subsidy — tens of billions of dollars a year — [for] the concealment of publicly-funded research from the public. We argue that Sci-Hub is hastening the end of this grotesque situation.
However, on the inside, the AMA was anything but smooth. For those who are unfamiliar with r/science AMAs, they are scheduled events were Redditors ask questions to a pre-approved group of scientists. In this case, all coauthors from our study were given the login to the eLife_AMA account, and we were all instructed to respond using this account. The AMA was organized by the journal eLife and the r/science moderators.
@greenescientist and I were sitting back-to-back in the Childhood Cancer Data Lab in #Philly. Outside, a nor'easter raged. We were typing away furiously, answering questions. Miserlou was also in the office, and asked, "guys, why are you ignoring my question?" What question!? We went over to Miserlou's workstation, and here's the comment we saw:
However, from our viewpoint there was nothing. Miserlou's comment had been shadowbanned! Shadowbanning is where a comment appears as if it posted while logged in as its creator, but does not appear for anyone else. The goal of shadowbanning is to censor content without its creator realizing. Not only was Miserlou's question a fantastic one, but he's had this Reddit account since 2007 and has accumulated 17,692 post karma and 12,879 comment karma, in addition to being a moderator for 17 subreddits.
Given that Miserlou was shadowbanned, we decided to check whether our own comments from the eLife_AMA account showed up if we were logged out. Many did not show up. So here we were leading this AMA and Reddit was deceiving us into thinking we were having a conversation, while we really just posting to the void. Then things got even more ridiculous. Two of @greenescientist's comments that had initially posted got deleted. When we were logged in, we saw:
But everyone else saw (Internet Archive snapshot):
At this point, we almost called off the whole AMA. What's the point of doing it if all of our comments were going to be censored or deleted? Was there a troll moderator that wanted to sabotage the event?
I took a few emergency actions. First, I tweeted the issue. I wanted to establish documentation on an independent platform in case this was an adversarial situation. Second, I messaged the mods. Third, I posted a comment from the eLife_AMA account describing the issue:
Of course, this comment was shadowbanned. Therefore, I edited the initial thread description to include:
We have temporarily stopped responding to comments using the eLife_AMA account because our comments are getting deleted. See this tweet thread for details. Can a mod please leave a reply on this comment when this issue has been fixed?
Eventually, a moderator got back to us. Here is our entire exchange which transpired over the next several hours:
So according to the moderator, it was the automated "automod" rather than a human saboteur that was censoring our comments. This negatively impacted the AMA, since many comments would take several hours to post (only once a moderator had un-banned them). Hence, back-and-forth conversation wasn't really possible.
Interestingly, I believe the comment shown above where I detail the censorship was eventually un-banned by a mod but is now back to being banned. However, maybe I was just confused since shadowbanning is so misleading and annoying. The number of false positives is absolutely unacceptable and indicates a major issue with r/science's automod.
One reason I suspect our posts were flagged is because we use links. Yes, since we're scientists and believing in crediting sources, we link a lot. I've also had this problem with Disqus, where my comments with links always get marked as spam. People complain about fake news, but combating fake news requires a culture of attributing sources, which starts with encouraging (and not discouraging) hyperlinking. At least however, Disqus is slightly more forthcoming that your comment has been flagged as spam:
Why decentralized social networks are the solution?
Could this situation have happened were we using Steem? Yes and no. On Steem, the blockchain is permissionless and decentralized. Therefore, if we were commenting on a Steem post to do an AMA, anyone could comment (within the limits of their bandwidth which depends on their Power). Now certain comments may get downvoted. Each explorer (like https://steemit.com, https://busy.org/, or https://stage.steemiz.io/) is able to sort and hide comments however it'd like.
This means that if I were unhappy with the filtering of one Steem explorer, I could switch to another one. Unlike Reddit, there is not the possibility that comments could be deleted or hidden from the world. This means that all censorship is elective. For example, I may want to use an explorer that censors certain types of content, such as offensive, illegal, or low-quality content. However, neither I nor the content is beholden to any specific frontend. For example, I certainly wouldn't use an explorer that censored content with such carelessness as r/science's automod. And with Steem I have a choice.
r/science has 18.4 million subscribers and has been around for 11 years. How could such a popular forum have such terrible and erratic censorship? I think the main issue is that Reddit is a centralized platform. Therefore, subreddits and users have a high degree of reliance is placed Reddit Inc. In this instance, it seems that r/science developed a custom "automod" program, which has its problems. Decentralized protocols allow for much more innovation, since no permission is needed to innovate. Check out SteemTools to see how many services and applications have been built on Steem since its creation in 2016.
Let's check back in 5 years and see whether online communities still aggregate on centralized platforms or whether decentralized, incentived platforms are home to the best communities and best AMAs!
This post and its images are released under a Creative Commons Attribution 4.0 License, so feel free to reuse or repost it anywhere for any purpose as long as you link to this post.
Disclaimer: I am just a bot trying to be helpful.
Your post is awesome, but I do have two questions that might be out of topic, since you know a lot about Reddit. Do you think Steemit is a competitor to Reddit? and is it possible that Steemit someday will surpass Reddit in terms of ranking and quality content.?
Certainly, Steem is a major competitor to Reddit.
Steem is very similar in terms of user experience. Both are oriented around commenting on posts. I expect the incentive layer on Steem could make the experience more engaging and habit forming. Reddit has an incredible community, via interest-specific subreddits. However, the platform has little innovation and does not particularly seem to care about improving the user experience. The curation and moderation models have broken down for many subreddits, especially related to cryptocurrency, where most of the front-page posts are from trolls, shills, or doofuses.
I'd say Steem has yet to develop strong communities. Perhaps the release of Smart Media Tokens will help community building as communities develop around tokens. However, I expect users and communities will emerge with time, at which point, Steem could become the new "front-page of the internet".
P.S. I think you mean Steem not Steemit (the former is the platform and network, the later is a single company / frontend).
Well, I'll say for steemit to overtake reddit, cryptocurrencies would have to be mass adopted and have awareness more than ever. Hopefully going forward, the world would understand and key into the opportunities provided by sites like this one.
Things like Facebooks Analytica firm getting raided yesterday and people like Elon Musk telling their employees to delete their facebook..
As well as these type of conversations that are being held here :)
I do believe we are stepping in the right direction!
Social sites should be encouraged to be a substitute for Facebook
Good job mate.
Reddit was my favorite community but now since i joined steemit. I got engaged here a lot and get answered my questions.
One thing i can guarantee this reward system in steemit will take steemit beyond all the boundaries.
You may like my natural photography blog also. I try to post good pictures of nature and see the real Beauty. So i wish you follow me to see that real beauty daily.
Great post .
But i believe steem is better than reddit
Good articles on the benefits of steemit, bur the translation intp spanisk makes it difficult for me to understand it in the best way
Cool to see that Steemians span the globe!
I'd like to see more advanced support for language metadata, at least on steemit.com. Here are some relevant links:
If I understand correctly, language metadata can be stored using JSON in a comment or post. However, there doesn't yet seem to be a standard of exactly how to encode that information. Furthermore, frontends haven't started setting language metadata yet. For posts, many authors use tags to specify the language.
I really like this proposal by @sneak of how frontends should automatically set the language metadata:
Thank you so much for documenting your experience and highlighting the importance of decentralised social media. I hope reddit users begin moving to steemit/busy and other steem-platforms.
Continue the great work @dhimmel
Automod, etc, is nonsense. Whenever anybody tells you A.I., or any buzzword, or a ``smart'' script censored your comment on a social media platform, be very skeptical. Very skeptical.
Very few of these are fully automated. The technology isn't there yet. (Deep learning, with or without convolution layers or reservoirs, doesn't produce much in the models of objects with which it would interact. This is problematic for a semantic web. Even less so a script or a typical bot.)
A group of people are usually there. There is some automation, true, very likely, but it's never to the extent alleged. Rather they don't want to reveal they disliked your comment/post/content. Because you might take action in response.
The same rationale as for shadowbanning. As opposed to banning.
Organizations can allege ``whoopsie'' and excuse and deny real hostile behavior against those whom they provide a service.
In general, alleging censorship it's part of some automatic procedure has been a trick used by censors in the days of Stalin, to reduce pushback against censorship, as described for example by Abdurakman Avtorhanov (The technology of power, New York: Praeger, 1959). Even mere delays are very effective at breaking the formation of a consensus. Or creating instead another consensus. Timing matters. Indeed timing is the unstable element determining phase change in networks.
Thanks @tibra for your thoughtful reply. While I agree with many of your points regarding censorship generally, I'm not sure how much they apply to this situation.
It doesn't make much sense that Reddit mods would be interested in censoring us. After all, the mods were the ones who invited us to do the AMA. Now there are multiple mods, and while most mods were on-board with our AMA, perhaps a few went rogue and deleted comments.
It seems to me that the automod feature is extraordinarily simple and most likely does not use deep learning or any other type of machine learning. My guess is that it's just a set of human-designed rules. One of those rules is to flag posts with links in them, perhaps with a few more criteria. This rule was likely added to combat spam, since links in Reddit threads are likely a decent way to influence SEO. However, it is a heavy handed solution because it catches both the best (comments with sources) and the worst (link spam) content.
Perhaps a better solution would be if a post is classified as spam for Reddit to use
rel="nofollow"for hyperlinks in that post. Therefore, the spammers would see little benefit. Furthermore, once a flagged post received a certain score, thenofollowprecaution could be removed. Now I'm not sure whether Reddit's platform provides r/science this flexibility. Back to my point of why it's important to separate out the content layer into a decentralized database (i.e. blockchain) and allow any frontend to be built on top.``Perhaps a few went rogue and deleted comments.'' Very likely the case. Too often the case.
Favorite example, it appears a long time ago one graduate student at MIT didn't like Jerry Pournelle's science fiction novels. The student was working part time, moderating Darpanet. So he found an excuse, and banned Jerry from Darpanet. The other administrators apparently accepted the excuse, or even agreed with the ban while mentioning they too didn't like the novels.
You never know you said publicly that offends somebody who possess administrative power. Even minor power. It's genuinely difficult to anticipate what phrase offends some people.
(Some people with a lot of time on their hands are also not all there. I've gotten flagged by flat earth folks with more SP to get my attention, so they can then spam me, blocked by crazy people who first Dm'd me because apparently I didn't reply fast enough to suit their taste, etc. Etc.)
Today such people have an excuse, some script, or a bot, rules. The systems we have for moderation are far too opaque to be able to distinguish mistakes.
Discord has bots that delete links. Everybody else has gotten more sophisticated in their censoring, because it works beautifully to change consensus. And waaaaaay too many people want influence and know it works. Even in obscure subreddits.
In my opinion, Murakami (198Q) has observed correctly that most people want to feel like what they do is important, rather than consider simple reality.
Our brains aren't used to being in cynical mode all the time. When our species evolved there were no communication tools, no images, no artificial sounds, and what we saw, heard, was what there was. Much of our brain operates on naive realism. Therefore even if users know a conversation is being censored or modified, most of them still feel preconsciously, most of the time, that an observed conversion is basically valid. That its meaning has not been substantially or irrecoverably changed. That information has been lost. And that, as Heinlein frequently wrote, intentional omission is the most powerful form of lying.
This can shape opinion. Most individuals moderating know this, either by reflection or trial and error. And long ago, when I was student, in one game theory class, I was even tested on this. It's not a secret, but a social technology that allows more productively working with some mediums, and people use technology. We people are tool users.
``. . . why it's important to separate out the content layer into a decentralized database (i.e. blockchain) and allow any frontend to be built on top.'' Agreed.
Nowadays censorship is becoming a very real problem on the Internet. More and more platforms are taking an aggressive stance against shared content. Reddit makes some very controversial decisions in this regard, as it recently shut down all blacknet dependent subreddits. On the one hand, it's pretty easy to justify Reddit's actions when it comes to removing illegal content from its platform. Judging by his recent statement, the company takes a zero position in relation to drugs, firearms, sexual services, stolen goods, etc. Most people would be happy to accept this approach, as most of these topics should be avoided because of their illegal nature. On the other hand, we must admit that it is censorship in its purest form. Regardless of the illegality of the above-mentioned topics, there is no reason to prevent people from thinking or talking about them. In fact, it is the decision of Reddit will push more people to other platforms which are not actively controlled. Whether or not it was a reasonable decision, understandably, has yet to be determined.
Congratulations @dhimmel, this post is the third most rewarded post (based on pending payouts) in the last 12 hours written by a Hero account holder (accounts that hold between 10 and 100 Mega Vests). The total number of posts by Hero account holders during this period was 341 and the total pending payments to posts in this category was $4942.64. To see the full list of highest paid posts across all accounts categories, click here.
If you do not wish to receive these messages in future, please reply stop to this comment.
Reddit was a great place, until I noticed the censorship running rampant.
Only trolls live on Reddit, anymore.
STEEMIT NEEDZ TROLLS
I BEG YOU TO READ MY AWESOME EPIC SUPER DUPER MAGNIFICENT AMAZING ASTONISHING BREATHTAKING ASTOUNDING PERFECT INCREDIBLE WONDERFUL JAW-DROPPING SENSATIONAL MARVELOUS PRODIGIOUS SUPERCALIFRAGILISTICEXPIALIDOCIOUS HILARIOUS LUDICROUS IMPRESSIVELY MIND-BLOWING COMMENT:
Are you sure you can handle it???
Well here it is:
Yep that was it
DISCLAIMER
This comment is spam. Please do not follow back. Please do not upvote. Doing so may result in a waste of the effort you took to click the upvote button.
WARNING
This comment may be so utterly idiotic as to render you dyslexic for the rest of eternity. Oops, should have put this at the top. Well reading this comment probably decreased your IQ by 90% so there's no point in writing this warning because you are probably too stupid to read it now. Anyway, remember to upvote!
Works Cited:
Thesaurus.com
Congratulations, your post received one of the top 10 most powerful upvotes in the last 12 hours. You received an upvote from @rocky1 valued at 124.14 SBD, based on the pending payout at the time the data was extracted.
If you do not wish to receive these messages in future, reply with the word "stop".
I like all of your stuff because your content type is very educational
Assist me in taking down THE biggest abuser on steemit!
I exposed @berniesanders for what he really is in my posts.
Just take a look at any of my posts (which I HAD to spam constantly to drain his VP and also to get the exposure I needed) and you can see just how far the abuse goes by @berniesanders.
He even got lazy at one point in making alternate accounts to abuse with and made "nextgen-InsertNumberHere-" accounts!
Please, do your due dilligence and check out, not only my posts, but his steemd track record on upvotes/downvotes. He's got many many more accounts than just the ones I've found, I'm sure, but those were only the ones that I could PROVE were his beyond a shadow of a doubt (based on voting/downvoting patterns, delegations, transactions, etc!)!
The real abuser here is the one who cries "ABUSE!!" the most, it is, none other than the inbred, cousin-fucker, that is, @berniesanders! :))
Here's a link to one of my latest posts, showing @berniesanders' alternate account list! I believe it's STILL growing.. which is sad.. He truly is a sociopathic dipshit! Let's take this fucking piece of shit out, together! :)
I really wanted to like Reddit but it seemed to be an echo chamber. If you disagreed with a unknown metric you got censored and smacked down. There is too much of that going on in meat space let alone cyber space.
I don't have any knowledge since from the beginning I am into steemit
I've posted the link to this post on Steemit to r/science with the title Problems with the r/science automod interfered with our recent AMA about Sci-Hub. Hopefully, this post will get noticed so the issues with the automod can be fixed.
I m new in steemit i have not alot info about steemit your post its very useful and inspiring thanks for sharing @dhimmel
This is why I no longer use Reddit for serious discussions.
@mrsomeone, out of curiosity what subreddits have you had issues with? Do you think some are worse than others in terms of shadowbanning legitimate comments?
Almost all of them, but I have caught shadow banning, downvote bots, and the like at the following sub reddits: supplements, engineering, science, fitness, conspiracy, politics, news, and similar.
And yes some are worse than others, specially the bigger subs like science, politics, news.
Thats sad to hear that it's so hard to have an open discussion on a board intended for discussion.
Hi Daniel. I stopped by to see if you made a post on the sci-hub Q&A session on Reddit, which you did! Yikes, didn't sound like it had gone as planned. So shadowbanned, huh? How misleading it is too since you were able to see your own comment, but other users did not. At least have the censorship universal without any hidden motives, like have some disclaimer of sorts. Instead, the site decided to censor content without its creator realizing. Sneaky.
I never paid much attention to censorship online, since I'm not an avid blogger. Since starting steemit though, I do realize the benefits of a decentralized platform. I wonder if more users will switch from Reddit to Steemit, since the rumor is that communities is supposed to be similar to reddit. Will be interesting to find out.
Sounds like you are working with some great individuals there at Penn. How funny that you are originally from Cali! I'm from So Cali but did live in the bay area for several years. Somehow, we both ended up on the east coast! Different way of life here, huh? ;)
Thanks for the thoughtful post! I am a mod for some subs on Reddit and noticed some confusion about the site in your post and follow-up comments. So I thought it might be helpful to discuss how Reddit works with regards to these processes and what is specific to /r/science.
A removed comment is not shadowbanning. Anytime you have a comment removed for any reason on Reddit it usually still shows up to you but not others. But neither you nor /u/miserlou were shadowbanned so no worries there! Banning and shadowbanning are very specific terms within Reddit but neither apply to this context.
Automod is not specific to /r/science. It is a mechanism that is now built into Reddit that allows subs to insert specific phrases or websites that will auto-pull the comment. For example, racial slurs or known spam sites. But it will also auto pull really short comments (ex: if you just say "ok") or comments with a lot of links. The link issue is not something that /r/science moderators can change and is something you may run into for every sub. One way around this is to message mods when you post a link-heavy comment and it can be manually released. The amount of karma you've accumulated may also impact this issue (low karma will auto-pull a comment on many big subs.)
Moderators cannot change your comment nor where links follow. Either the entire comment stays or we can remove it. There is no in-between option.
Each sub has its own rules for content. You can't post cat photos in /r/dogs, for example, because the point of that sub is to curate photos of dogs. /r/science is (in)famous for their strict moderation rules of no jokes/pop culture, no hate speech, no pseudo-science, no fights, etc. Even Redditors who don't frequent the sub are well aware of this and know that if they run afoul of those rules their comment will be removed. For better or worse this sub-by-sub set of rules is the normative culture of Reddit and people expect a need to codeswitch.
/r/science is also unique because to enforce this strict conversational curation they have over 1,000 moderators. Most only have permission to remove comments while only a handful have additional permissions. This helps handle hot-button posts that garner tons of racist or sexist comments (for example) but it does slow down response time for actions that require higher permissions (such as approving removed comments.) Most subs simply aren't big enough to warrant that kind of team. Higher level mods periodically survey their activities and strip permissions if they are moderating in ways that aren't in line with their rules. I can't tell you definitively if any of your comments were grabbed by low-level mods vs automod, but I don't see any rule violations but I do see links. So automod is the logical assumption.
Some subs do alert users when their comments are removed but in my experience the subs with millions of users do not. This is mostly due to a volume issue. In small subs it is easy for me to give people a heads-up and manage responses. But if a post has a hundred removed comments due to actual rule violations (i.e. valid removals) it would take a lot of manpower to respond to each query or response.
I also see from a comment that you posted a text-post to /r/science, but their rules do not allow that. I suspect it was removed by auto-mod. Many subs do allow text-posts and figuring out where to target your content is part of just getting to know platforms and sub-cultures.
All of this does lead to very interesting debates for platforms from Facebook to steem. Do sites have obligations to deal with harassment, violent threats, and illegal content? If so, how do you build that moderation into the system without censoring inappropriately?
What about less obviously problematic content? One reason /r/science says they have such strict rules is that they've worked with communications scholars who showed through peer-reviewed research that pseudoscience and/or aggressiveness in comments meaningfully impacts how readers interpret the science in the associated posts. This research is why most major science news outlets have removed their commenting sections, btw. So what is the appropriate way to cultivate discussion that doesn't feed pseudoscience and/or science dismissal?
I certainly don't have the answer. Reddit is also very frustrating in that we moderators have been begging them for better moderation tools for years. Modmail is awful. Automod is a very blunt instrument, as you discovered. It is hard to sort through notifications. It is simply not well set-up for moderation yet that responsibility falls on the shoulders of volunteers. If your experiences frustrated you please consider dropping an email to the Reddit admins telling them to give moderators halfway decent mod tools! :)
Thanks @liminalphase for your inside knowledge and thorough explanation. Also welcome to Steem!
I've reread my post with the information you provide in mind. It sounds like the posts that were only visible to the original poster resulted from auto-mod, triggered by hyperlinks. I am assuming auto-mod-banned comments were never publicly displayed, i.e. auto-mod blocked them immediately and not after they had already been public for some amount of time.
Given your explanation, it sounds like the comment that publicly showed "comment removed" (but showed in entirety to the eLife_AMA account which posted it) was not auto-mod. This is because auto-mod would hide the comment immediately, rather than rendering it as "comment deleted" at a later time. Therefore, I'm assuming there was human intervention in this instance.
Of course, human intervention, especially from a moderator, would be surprising as this was an officially sanctioned AMA post on r/science. In fact, it was coordinated by a Senior Press Officer at eLife in conjunction with the r/science moderators. r/science cared deeply about the popularity of these posts. In fact, so much so, that they would delete posts from around the time of the AMA, to help get the AMA highly ranked and achieve virility. Apparently, Reddit changed the rules to prevent this delete-posts-to-favor-another-post tactic, and subsequently r/science ceased the whole AMA program.
This could have resulted in a low-level moderator causing the "comment deleted" issue. Our study is somewhat provocative, and perhaps a moderator was abusing their role.
Is that possible? In other words, could a post be auto-mod hidden, then be made publicly visible by a mod, but then return to being hidden without noting "comment removed"?
I guess this was not a technical "shaddow ban", but it certainly seems to fit the general definition of one (hiding content from the public, while pretending to its original poster that it's public).
Good to know, but I don't think our comments (or Miserlou's) violated any rules. In fact, our comments were in essence pre-approved, given that they were the purpose of the AMA.
Welcome to a platform where if something is broken, you can potentially do something about it, (beyond begging). Of course, there are many problems with Steem, but I think they are generally much more solvable, since the blockchain architecture allows permissionless innovation on top of the open database.
This is what I really disliked about Reddit. Instead of having uncensored discourse, you are at the mercy of moderators and whatever their politics are.