CZ:Proposals/Enable external feedback

This proposal is presently driverless. Why not become its driver?
You can sign up on its proposal record, which may be found on the driverless proposals page.

Driver: None

Proposal

Only the "Proposal" and "Implementation details" sections, and their subsections, are official parts of this proposal.

Readers who notice errors, but aren't interested in creating accounts, should have an easy way to let us know without leaving the wiki.

Proposed:

• There should be a feedback submission page for each article that's accessible to non-Citizens in a way similar to Citizens clicking on "edit".
• Feedback can be submitted on all articles in the main, what's-your-article and topic-informant namespaces.
• Feedback to be attached to articles (and appear in watch lists) and viewable by any interested Citizen. If programmers find this too difficult, we may deliver feedback in another way.
• Non-Citizens should not be able to view feedback (to reduce the incentive for spam)
• In the future we might explore automation such as allowing feedback in the form of machine-readable proposed edits, but for simplicity we are deferring that decision until we get experience with a simpler scheme.

User documentation

This subsection contains a draft for the text to be presented to readers describing the feedback form. Please feel free to edit this subsection.

Use this form to:

• Point out errors, typos and omissions
• Indicate parts that are confusing or well-written
• Thank the contributors who wrote the article

Caveats:

• We offer no guarantee that your suggestions will be implemented.
• Suggestions that appear to be promoting someone or biased will be ignored.
• Information in the Citizendium must be verifiable. If an expert would doubt the veracity of your suggestion, please include references.
• If your feedback is potentially controversial, please make an account rather than using the feedback system.
• Your feedback may be accessible to all Citizens, so don't submit anything confidential here.

Summary of arguments

Support

1. Allow casual good Samaritans to help us improve.
2. Submitting feedback is a low-commitment way for potential Citizens to get their feet wet. If we act on feedback, we can send the submitter an email thanking them for the feedback and encouraging them to sign up as a Citizen.
3. Allows for addition of interesting factoids and local knowledge, not readily apparent to an outsider.

Anti-support

1. We may get swamped with too much low-quality feedback
2. Probably requires modifying MediaWiki software.
3. Possible source of computer trojans, viruses, etc, and possibly gossip/slander/libel of Citizens.

Arguments about details

Arguments in favor of allowing feedback on all articles:

1. Non-approved articles need substantial improvement, so allowing feedback on these will offer many more opportunities for readers to send feedback and get hooked on CZ. Feedback on approved articles will only happen when we mess up.
2. Readers may have local knowledge about a subject, that is hard to find, but which can be verified once it is pointed out.

Arguments in favor of allowing feedback on approved articles only:

1. Citizens may be inspired to improve an article to address feedback.
2. The quality of the approved article may be further improved with some valuable feedback.

Arguments in favor of routing comments to authors:

1. Citizens concerned may be inspired to edit an article to address feedback.
2. Authors may become more careful in future while editing.

Arguments in favor of routing comments to articles and/or workgroups:

1. Citizens may be inspired to edit an article to address feedback.
2. Current policy is that articles are not owned by authors. Changing that would be an independent proposal that shouldn't be combined with this one.
3. All authors may no longer be with CZ, therefore route comments to Workgroup AND Talk page for action.

Implementation details

wikiHow's "thank the authors" or discussion functionality might be adapted to our purposes.

Warren Schudy is looking for someone familiar with Citizendium's MediaWiki setup to help him determine what's easy to implement and what isn't. If you're interested, please contact him. See implementation discussion section for Jitse's estimate of how much programmer time different options might take.

The remaining subsections of the "implementation details" section are a draft specification of what we want. I'll use this to try to get someone to write a MediaWiki extension for us. Please feel free to edit this.

Introduction

A Citizen is a logged-in user of Citizendium (editor in Wikipedia terminology). A reader is a user of Citizendium that is not logged in and is therefore reading, not editing. Citizendium is planning to allow readers to submit feedback on articles without leaving the wiki. We need a MediaWiki extension to support this.

Reader's interface

Optional features are italicized

Readers should have access to a "feedback" tab to the left of the "view source" tab. There is no need for readers to give feedback on talk pages, so clicking on "feedback" on a talk page should send the reader to the feedback page for the corresponding article. The links for editing a section of an article should also be converted into feedback links.

The feedback submission page should include:

• A large text box for writing free-form comments
• A "submit" button
• A field where the reader can give an email address to allow us to let them know if their comment is acted upon or we have questions.

Citizen's interface

Optional features are italicized

There should be a "view feedback" tab for each article that is only viewable by Citizens. There should be some way to discuss feedback and delete spam and other useless feedback. Perhaps the simplest way to handle this is to make the feedback page be essentially a second talk page that anyone can add new sections to but is only viewable by Citizens.

There should be some way of telling if any articles one is watching have received new feedback. This can be accomplished either by including the feedback in the watchlist, or with a separate special page showing new feedback.

There should also be some way of listing all feedback recently received on any article. This should be like the recent changes page, but should be separate since recent changes is busy enough as is.

Remarks

The extension should prevent readers from introducing arbitrary HTML and wiki markup that would interfere with the proper operation of the feedback page or promote malware. One way to do this would be to make feedback be displayed plain-text only, as if inside a <pre> or <nowiki> tag. There should be some mechanism for blocking feedback from IPs that abuse it. This can probably be implemented by simply checking to see if an IP is blocked from editing before accepting feedback from it.

We aren't attached to these particular details. If there's some other way to do it we're all ears. However, we need:

• Submitted feedback is not visible to readers (reduces incentive to spam)
• Feedback should not interfere with normal operation of the wiki even if we get loads of it.
• Need some way for Citizens to mark feedback as implemented or rejected

We might eventually consider giving readers a wiki edit interface as well so they can submit a diff as part of their feedback. Citizens would then have tools for automatically merging those edits into the articles. We aren't sure if we want this, so we're not asking you to implement it, but keep it in mind when designing the extension.

Previous discussion

1. [An old thread] (January 2007).
2. A related [discussion]
3. [A reader gives feedback through the forums]
4. [Discussion of anonymous edits, which is closely related to user feedback]
5. [Mention in brainstorming thread]

Warren's Notes

This section is not an official part of the proposal, but rather a summary of Warren's reasoning.

Why machine readability is important

Scenario A:

Suppose a reader is looking at Composting. They click on "send us feedback on this article" and send the following feedback:

In the introduction, "Actually it is one of the oldest forms of recycling of waste." in the introduction is not good english. Consider removing "actually", yielding "It is one of the oldest forms of recycling of waste."

A Citizen reads this feedback and does the following:

• finds the sentence in question
• deciphers the English change instructions and evaluates the edit.
• If the edit is a good idea, actually does the edit
• Marks the feedback as "resolved" in some way so another citizen does not repeat the work.

This is an unnecessary amount of work, both for the reader, who has to express a diff in English, and for the Citizen who parses, evaluates and executes it. Shouldn't we use computers to make this easier?

Scenario B:

A reader notices this error and clicks "send us feedback on this article". The reader puts "bad english" in the feedback box. Below the feedback text box is an ordinary Wiki edit interface, which the reader uses to fix the problem. The reader then submits the feedback. (Readers are of course free to use just only the feedback text box and ignore the edit box.)

A Citizen views the feedback and is presented with a diff for the proposed edit. If the Citizen likes the edit as proposed, s/he can endorse it with one click. If not, s/he can write a comment on the feedback.

Discussion:

Scenario B is less work for the reader and the Citizen alike.

Of course, feedback of the form "I don't understand X" would not benefit from the machinery of scenario B. Often readers will know how to fix it, so why make it harder than it needs to be in those cases?

Scenario B would save time for Citizens but would presumably be a lot harder to implement in the software, so we may want to start with scenario A and defer automation for later. I therefore describe two versions of the interfaces for readers and citizens, with and without automation to handle machine readable feedback.

Reader's interface, no automation

People who are not logged in, herein referred to as readers, have access to a "feedback" tab instead of the "edit" tab. There is no need for readers to give feedback on talk pages, so clicking on "feedback" on a talk page should send the reader to the feedback page for the corresponding article. The links for editing a section of an article should also be converted into feedback links.

The feedback submission page includes:

• A large text box for writing free-form comments
• An optional field where the reader can give an email address to allow us to let them know if their comment is acted upon or we have questions
• A "submit" button

Citizen's interface, no automation

There is a global page analogous to "recent changes" that shows all recently submitted feedback. There is also a "view feedback" tab for each article that shows feedback for that article.

There needs to be a way for Citizens to email the contributing reader without disclosing the readers' email address (analogous to the "email this user" feature).

Reader's interface, with automation

People who are not logged in, herein referred to as readers, have access to a "feedback" tab instead of the "edit" tab. There is no need for readers to give feedback on talk pages, so clicking on "feedback" on a talk page should send the reader to the feedback page for the corresponding article. The links for editing a section of an article should also be converted into feedback links.

The feedback submission page is very similar to an edit page, but with a few differences to reflect its distinct purpose. It includes, in this order:

• An explanation that the reader may either write a comment in prose, make a proposed edit using the wiki machinery, or both.
• A large text box for writing free-form comments
• An optional field where the reader can give an email address to allow us to let them know if their comment is acted upon or we have questions
• A "submit" button, for use of readers who don't want to use the wiki edit box
• An ordinary wiki edit text box
• "submit", "preview" and "show changes" buttons.

Unlike a citizen edit page, a feedback page does not have:

• "minor edit", "watch this page" or "content from WP" checkboxes
• An edit summary (that's subsumed by the free-form comments box)

Citizen's interface, with automation

Submitted feedback is essentially an edit, but is dealt with separately and a bit differently. The biggest difference is that feedback does not actually modify the articles immediately.

There is a global page analogous to "recent changes" that shows all recently submitted feedback. There is a "view feedback" tab for each article, analogous to history, that shows feedback for that article.

When viewing a piece of feedback, a Citizen gets:

• A list of actions taken by other Citizens in response to that feedback, if any.
• The free-form comments
• The diff for the proposed edit, relative to the version of the article that was current when the reader submitted the feedback
• An ordinary article edit interface
• Buttons to load into the edit interface either the current version of the article, or an automatic merge of the reader's suggestion and the current version.
• Information to help with the merging process
• A text box for sending questions or thanks to the reader, if they supplied an email
• A big red button

The big red button:

• Saves the changes to the article, creating a history item in the name of the Citizen who reviewed the feedback.
• Optionally sends email to the submitting reader, informing them of the change, thanking them, and suggesting that they become a Citizen
• Marks the feedback as "dealt with"; Citizens can choose to skip those when viewing feedback lists.

I'm pretty sure it is possible to revert a change that's not the most recent one, so there must be code for merging changes. I wonder why that code isn't used to suggest resolutions to edit conflicts?

Stephen's opinion

Note: this section quotes Stephen Ewen but was written by Warren Schudy.

[| Stephen Ewen writes:]

The code to allow something close to that in MediaWiki exists, is available just for the asking, and could be adapted for CZ. You can see it at work at WikiHow--the folks there developed it. Go to any article, say, [[1]], and scroll down to the bottom. You will see a link "Thank the Authors". This goes to a page where one writes a message that then goes to each of the main contributors of the article (that's another thing that could be added--you can view that at the bottom of the page there as well). The messages then aggregate at a special page, called "Kudos", for each author. The code that names the authors and allows reader feedback are dependent.

I am not sure we should implement it--maybe we should, maybe we should not--but the ability to do it is there. One issue is that we'd really not want to receive anon IP feedback (which is what all of it would be) on anything but approved articles, so that would mean writing code or, perhaps, placing all non-approved articles on a draft subpage. One good thing--and this is a major thing that would incline me to support it now, where I formerly argued strenuously against it--is that none of the comments would need to be moderated (meaning this would give constables no new tasks), since they are only viewable to the specific authors of the article. Someone could cuss the authors out and the outcome would be no greater than receiving a spam message in one's email inbox.

Again, however, naming principle contributors to an article through the mechanism mentioned above, and this anon IP feedback system to them, are dependent. We'd have to decide on the former before the latter, or come up with something other than adapting this code from Wikihow to allow the feedback. I very much think CZ should adopt the author attribution system, for reasons I've spelled out elsewhere.

Differences from Warren's proposal:

• Feedback only viewable by authors
• Limiting feedback to approved articles

Discussion

I think its a good idea as long as its technically feasible. --Denis Cavanagh 14:27, 9 February 2008 (CST)

This could be a source of malicious code. Saveguards would be needed to parse the text. -- David E. Volk 08:24, 11 February 2008 (CST)

It would be best if outsider's comments came somehow to the notice of the main author(s) of the articles commented on. These authors could read the comments and decide whether or not to include them in the article. I don't foresee in the near future so many comments that this process must necessarily be automatized. If it becomes too much work, we can always start a new discussion on how to parse and edit automatically (including the difficult problem of automatic saveguards). --Paul Wormer 08:53, 11 February 2008 (CST)

Is there some reason why we aren't working together on a single proposal? --Larry Sanger 11:32, 11 February 2008 (CST)

A single proposal is the eventual goal of course, but currently people disagree about some things, such as:

• Whether feedback should be routed specifically to authors of an article, or to whoever is watching the article whether authors or not
• Whether to allow feedback on all articles or just approved ones

If the proposal system had a distinction between issues and proposals, this would probably be an issue, not a proposal. I think it's best to flesh out in detail several different plans so we can make a decision based on a comparison of detailed plans rather than just rhetoric. I am also trying to make this document a summary of people's opinions similar to CZ:Summaries of policy arguments, so it needs to discuss differing views. Accepting feedback is an important step in the evolution of CZ, so it deserves a summary of arguments IMHO. Warren Schudy 18:58, 11 February 2008 (CST)

After spending some time recently thinking about the details, I've changed my mind and I now propose that the initial feedback system should not have machine-readable edits. I still strongly believe that machine-readability is the way to go long-term, but I think we should get some experience first with a simple system before implementing automation. I'll write/rewrite the basic proposal section sometime in the next few days. Warren Schudy 18:58, 11 February 2008 (CST)

The first three sections now summarize what I believe is uncontroversial, what is uncontroversial, and some arguments. Please comment. Warren Schudy 19:49, 11 February 2008 (CST)

Reader feedback must go the workgroup associated with the article, because the primary author, even possibly all of the authors, may no longer be with CZ at time of the comment. David E. Volk 10:56, 13 February 2008 (CST)

Perhaps some kind of digest of feedback received in a particular workgroup each day could be included to be sent to the relevant workgroup email list? Hopefully, everyone will be subscribed to their relevant workgroup lists. Mark Jones 06:46, 25 February 2008 (CST)

Not everyone is subscribed to their workgroup lists (saying this as the Mailing List Manager--I track mailing list numbers for fun, and activity and subscriptions is much lower than those who have categorised themselves as workgroup authors). I think this is a great way of getting people to utilise their workgroup lists more and become more involved.

A digest as suggested by Mark is relatively easy to implement if we can get a couple of workgroup members to step foward as additional mailing list moderators. External mail sent by non-subscribers is automatically set aside for approval (like spam catching) and willing volunteers can filter through relevant feedback and forward on a digest to the list rather than approve every external individual feedback email as they come. Seems like a relatively straightforward process.

Louise Valmoria 07:18, 25 February 2008 (CST)

Some comments moved to implementation discussion section

Implementation discussion

This section is for discussion of programming only. Discussion that strays too much to policy may be moved to the previous section. Warren Schudy 18:59, 29 February 2008 (CST)

I think it is an excellent idea to enable external feedback, and the proposal here seems to be pretty well thought out. There is one significant problem with it: it simply can't move forward without programmer support. The next step, therefore, should be to find someone who will help write or adapt some appropriate commenting extension for us. Actually, the next step is to find the extension itself, or in lieu of that, write some requirements for the extension (though this is already done to a great extent above). I would like to see exactly how it works before we install it. --Larry Sanger 12:01, 27 February 2008 (CST)

I updated the proposal sections near the top. I'm going to try tracking down some people who know the technical details to advise me on what's easy to implement. Warren Schudy 20:06, 21 February 2008 (CST)

Firstly, let me say that I think some way to allow external feedback should be implemented.

All the mechanisms proposed here need some changes to the software, but the good news is that they look not too hard to implement. Here are some more detailed comments on the bits that are needed, plus my guess to how hard they are to implement.

1. A feedback button leading to a page with a form where non-citizens can submit feedback. This is quite easy to implement.
2. Emailing the feedback to a central mailing list. This is the easiest way to handle feedback.
3. Emailing the feedback to the authors of an article. Again, quite easy.
4. Emailing the feedback to workgroup mailing lists. This is harder because the software has to find out what workgroup an article belongs to. That information is in the metadata page written by humans, so the software has to read this page and understand it. It will not be very robust (meaning that the software will sometimes misunderstand the metadata page and thus send the feedback to the wrong mailing list). It's also not very clean because it's very specific to how we use the metadata page.
5. Putting the feedback on a central feedback page. This is a bit harder than just emailing the feedback, because a new type of page has to be created (the feedback page). I assume that we make the feedback page editable, so that we can add comments as "acted on" and remove irrelevant feedback; that's not so hard. In any case, only Citizens will be allowed to view the feedback.
6. Putting the feedback on the user feedback pages of the authors. We probably want users to get a notification when feedback is added to their user feedback page. However, it's possible that we can leverage the WikiHow code.
7. Putting the feedback on a feedback page attached to the article. The main problem here is that the feedback page probably has to be added to the watchlists, otherwise this does not seem to be useful. This means that all watchlists in the database have to be amended when the feedback extension becomes active; I don't quite know how hard that is. Furthermore, the watch/unwatch button will have to add the feedback pages to the watchlist, just as it know automatically added the talk pages to the watchlist; that should not be so hard.
8. Putting the feedback on a feedback page attached to the workgroup. The same problems as #4 apply.
9. Storing the email address of the person providing feedback (if provided) and allowing Citizens to send email to this person without disclosing the email address. This requires some way to store the email address (thus a change to the underlying database) and some button on the feedback page to allow for sending email. This is probably the hardest point.

It's nigh impossible to predict how much time it will cost to implement this. My guess would be that somebody with experience in writing MediaWiki extensions will need one hour (each) for #1, #2, #3, and #5; two hours for #4, #6, and #8; four hours for #7, and six hours for #9 (this includes some testing and excludes writing the specification). Double or triple these estimates for somebody with only limited experience, like me.

My personal recommendation is to go for #7 (and #1). A minimal implementation would be #1 and #2, but that does not add that much to the current mechanism for providing feedback. Item #9 is nice to have, but might not be worth the trouble. -- Jitse Niesen 11:17, 29 February 2008 (CST)

There is a discrete failure to implement forms here in the wiki. I've tried to encourage it many times, often with either lukewarm response, no response, or confusing response. I'm quite aware of the technical challenges, but there just isn't any motivation for them. --Robert W King 11:25, 29 February 2008 (CST)

Robert: I don't understand what your comment has to do with this proposal. Warren Schudy 18:59, 29 February 2008 (CST)

I also like #1 and #7. If the email feature #9 is as hard as you estimate, I agree that it's not worth the trouble.

Ways to find a developer that come to mind:

• MediaWiki extension request page
• Submit a request using bugzilla
• Ask the developer who wrote our account creation extension.
• Post on the MediaWiki tech mailing list.

Does anyone know how we found someone to write the account creation extension? My inclination is to post to try posting to the MediaWiki extension request page first. I'll write up a specification. Warren Schudy 18:59, 29 February 2008 (CST) Jitse, thanks for your help! One way to make #7 easier to implement is to just put the feedback in a special section of the talk page (each comment gets its own subsection within the feedback section). Feedback would then appear in the watchlist automatically as talk edits. The biggest problem that I see with the on-talk-page approach is it would be difficult to restrict feedback to Citizens only. This may be fatal to that approach. Warren Schudy 21:46, 29 February 2008 (CST)

Hi Warren, please don't submit the specification until the proposal has been vetted and accepted by the Approval and Feedback Group. --Larry Sanger 22:19, 29 February 2008 (CST)

How do I submit the specification to the Approval and Feedback Group for vetting? The CZ Proposals Policy page says "E-mail the Approval and Feedback Editor (not yet selected)." I'd like to wait a few more days for comments before officially submitting to the approval/feedback group, but come Monday or so I'd like to submit it. Warren Schudy 14:12, 1 March 2008 (CST)

I have submitted the proposal to the approvals and feedback group for permission to find an implementor. Warren Schudy 10:37, 7 March 2008 (CST)

Worry

I'm concerned that this will be a gateway for spam/hostility/nonsense etc. No objection if there is a robust filter system - easy to delete rubbish fast. Real names is ideal or at least e-mail addresses; if comment was e-mailed I guess I'd be happy.Gareth Leng 06:31, 29 February 2008 (CST)

The current plan is to not require any contact info, but to make it fast and easy to delete rubbish. Warren Schudy 14:12, 1 March 2008 (CST)

My experience is that editors/authors are usually very sensitive about their work. Email lessens the exposure and real names lessons the offensiveness. Is there a reason why we wouldn't want to require names and email addresses? Maybe much the way we confirm emails initially; if their email address is verifiable their email gets through.. otherwise nobody sees it. Just a thought. --D. Matt Innis 14:48, 10 March 2008 (CDT)

People are often reluctant to give their email address in fear of getting spam. Requiring a confirmed email address would also add 30 seconds to the time required to comment, which would discourage a lot of people from doing so. Warren Schudy 09:18, 11 March 2008 (CDT)

I think there's going to be too much abuse of this system to make it feasible. If people want to provide feedback and improve article content, then they should just become citizens themselves. --Robert W King 09:44, 11 March 2008 (CDT)

It's quite a process to become a citizen, and I'm pretty sure that people who only want to point out a mistake in one of our articles, because that's the decent thing to do, will not be willing to go through the whole process. The question is whether we're willing to live with the rubbish in order to receive feedback. I don't think the rubbish will be much of a problem. There is not much incentive to post spam etc because as far as the poster is concerned, it just disappears in a black hole; only Citizens will be able to see it. The current feedback mechanism for non-Citizens is to post on the forums, and they have not become a gateway for spam/hostility/nonsense. Even on Wikipedia talk pages, where the incentive to spam or troll is far greater, well-meant feedback is more common than abuse in my experience. Finally, if it does turn out that we get only abuse via the feedback system, then we can just shut it down. -- Jitse Niesen 10:32, 11 March 2008 (CDT)

Approved articles or all articles?

Please sign your support in one section or the other depending on whether you think we should accept feedback on all articles or only on approved ones.

In favor of accepting feedback on all articles

1. Warren Schudy
2. David E. Volk
3. Larry Sanger
4. Martin Baldwin-Edwards
5. Supten Sarbadhikari
6. Mirzhan Irkegulov
7. Daniel Mietchen