Improving issue reports: recognizing users

Various discussions related to Adblock Plus development
Wladimir Palant

Improving issue reports: recognizing users

Post by Wladimir Palant »

While Andrey is adding the finishing touches on this proposal let's start discussion on another improvement. The goals are:
  • Encourage long-term contributions via issue reports.
  • Improve prioritization of issue reports, make it easier to recognize the useful ones.
Our current reports system treats all users equally. While this is a very democratic approach, we get lots of useless reports that waste the time of subscription authors. I think that we need to start recognizing returning users and acknowledge their contribution. IMHO the simplest approach is:
  • Users are recognized based on their email address.
  • The Issue Reporter user interface is changed to encourage users to enter their email address - it is required as a communication channel, so submitting an issue report anonymously should require an additional action (explicitly switch from "If you need further information you can contact me under this address:" to "Submit report anonymously" along with a warning that anonymous reports usually have lower priority).
  • On the server, create an "account" for each email address seen (don't actually store the email address, merely its MD5 hash). Unused accounts can be removed after 3 months.
  • The account stores the total number of submitted reports along with a "helpfulness" rating.
  • To be able to calculate the "helpfulness" rating for a user the report statuses have to be changed. There will be predefined statuses like: "Helpful: issue fixed", "Helpful: will be fixed later", "Helpful: fixing not possible", "Helpful: other", "Not helpful: unlikely to be related to Adblock Plus", "Not helpful: issue unclear from the supplied information", "Not helpful: other". There will still be a text field for additional information.
  • Depending on the proportion of helpful reports to the ones marked as "not helpful" reports by some users will be listed closer to the top when the reports are sorted by "relevance". Also, there will be a visible indication of the helpfulness rating in the issue report (probably stars of some kind) and a link to the user's "profile" listing his statistics.
Comments?
anonymous74100
Posts: 213
Joined: Sat Mar 19, 2011 3:45 pm
Contact:

Re: Improving issue reports: recognizing users

Post by anonymous74100 »

Wladimir Palant wrote:explicitly switch from "If you need further information you can contact me under this address:" to "Submit report anonymously" along with a warning that anonymous reports usually have lower priority
Why would anonymous reports have lower priority?
Wladimir Palant wrote: On the server, create an "account" for each email address seen (don't actually store the email address, merely its MD5 hash).
Would the users be identified by a username or the email hash? Also MD5 is too weak, Whirlpool would be better.
Personally I would like to see ABP decreasing reliance on adblockplus.org infrastructure, it creates a single point of failure. Since it's on .org TLD, the domain could easily be stolen by a certain government.
Wladimir Palant wrote:Unused accounts can be removed after 3 months.
So the accounts will be stored permanently unless someone (who?) deletes them?
Wladimir Palant wrote:"Helpful: will be fixed later"
Why would someone delay a fix?
Wladimir Palant wrote: Depending on the proportion of helpful reports to the ones marked as "not helpful" reports by some users will be listed closer to the top when the reports are sorted by "relevance". Also, there will be a visible indication of the helpfulness rating in the issue report (probably stars of some kind) and a link to the user's "profile" listing his statistics.
What if users don't want to be included in any statistics?

Even if you don't have any malicious intent, I don't like the idea of gathering data of contributing users.
Last edited by anonymous74100 on Tue Jan 31, 2012 5:04 pm, edited 1 time in total.
Latvian List maintainer
Wladimir Palant

Re: Improving issue reports: recognizing users

Post by Wladimir Palant »

anonymous74100 wrote:Why would anonymous reports have lower priority?
Because we have too many of them - and in very many cases they are missing the information required to resolve the issue.
anonymous74100 wrote:Personally I would like to see ABP decreasing reliance on adblockplus.org infrastructure, it creates a single point of failure. Since it's on .org TLD, the domain could easily be stolen by a certain government.
Erm... I seriously considered answering to that. But I won't, it's pointless.
anonymous74100 wrote:So users won't be able to delete the accounts before 3 months have passed and the accounts will be stored permanently unless the user deletes them. How nice!
No, this was a proposal to remove unused accounts automatically after 3 months - which should be a reasonable time span. On the one hand we have people who don't want their information stored too long for no good reason (and we don't want trash in our database anyway), on the other hand returning contributors who might take an extended break and wouldn't be happy to see their acquired status gone.
anonymous74100 wrote:Why would someone delay a fix?
I don't know? These are just suggestions. There seem to be cases where the subscription author cannot act on the report immediately where the report is useful nevertheless.
anonymous74100 wrote:What if users don't want to be included in any statistics?
Then they send their reports anonymously (and don't use forums). In which case they won't have any advantage over the wast majority of reports. Which usually means: if enough users report the same issue it gets fixed. If you are the only one reporting it then your report might get lost (EasyList & Co. don't have the capabilities to look at each and every report).
anonymous74100 wrote:Even if you don't have any malicious intent,
Thank you for your trust. Don't forget to add something like this to each of your posts.
anonymous74100
Posts: 213
Joined: Sat Mar 19, 2011 3:45 pm
Contact:

Re: Improving issue reports: recognizing users

Post by anonymous74100 »

Wladimir Palant wrote:No, this was a proposal to remove unused accounts automatically after 3 months - which should be a reasonable time span.
I misread unused as user and you replied before I corrected my post.
Wladimir Palant wrote:Erm... I seriously considered answering to that. But I won't, it's pointless.
I would really like to read your response. Why would making ABP not rely on adblockplus.org be bad?
Latvian List maintainer
Wladimir Palant

Re: Improving issue reports: recognizing users

Post by Wladimir Palant »

anonymous74100 wrote:Why would making ABP not rely on adblockplus.org be bad?
I'm answering just to stop the off-topic discussion here. I don't like removing off-topic posts but will have to do that if this unrelated discussion continues.

You are asking the wrong question. Fact is, your original question is wrong on several levels:
  • Talking about a single point of failure in the context of auxiliary functionality like issue reports is just ridiculous. Issue reports can stop working tomorrow and most people won't even notice.
  • The single point of failure for Adblock Plus isn't adblockplus.org but versioncheck.addons.mozilla.org (the update channel). I can publish an update and fix any issue (like moving services to a different domain) as long as the update channel is working. If it fails - that would be a disaster. Now you probably want to file a Mozilla bug and suggest making versioncheck.addons.mozilla.org more resilient.
  • However, when you ask somebody to invest significant effort into making a service more resilient you need to find good arguments. And "government X is secretly planning to take control over the Internet and start confiscating domains randomly" isn't one (and please don't come with SOPA now).
I hope that this answers your questions. Can we now focus on improving issue reports?
anonymous74100
Posts: 213
Joined: Sat Mar 19, 2011 3:45 pm
Contact:

Re: Improving issue reports: recognizing users

Post by anonymous74100 »

Wladimir Palant wrote:I hope that this answers your questions.
Your answer didn't contain anything I didn't already know. I guess It's my fault for not being specific enough.
Wladimir Palant wrote: Can we now focus on improving issue reports?
Fine.

So how about using Whirlpool instead of MD5?
Will issue reports be readable by everyone or only subscription maintainers? Will subscription maintainers be able to read all reports or only the ones that concerns them? (I'm talking about the user ranking table and list of reported reports, not the issue reports in their current form.)
Will users earn achievements, badges or something?
Latvian List maintainer
Wladimir Palant

Re: Improving issue reports: recognizing users

Post by Wladimir Palant »

anonymous74100 wrote:So how about using Whirlpool instead of MD5?
When you make suggestions, don't forget to explain why you make them. Are you afraid of rainbow tables? Email addresses (unlike passwords) typically are long enough that they aren't an issue. And if somebody compromised our server - extracting email addresses from the reports is a lot easier than reversing MD5 hashes. So MD5 hashing is more a way to avoid having plain-text emails in the database than a layer of protection (plus the resulting hashes are short).
anonymous74100 wrote:Will issue reports be readable by everyone or only subscription maintainers?
There are no plans to change the privacy policy. No, issue reports will stay private of course.
anonymous74100 wrote:Will subscription maintainers be able to read all reports or only the ones that concerns them?
As usually, subscription maintainers will only get access to the reports that concern them.
anonymous74100 wrote:(I'm talking about the user ranking table and list of reported reports, not the issue reports in their current form.)
There will be no (visible) user ranking table nor the list of reported reports - only the number of reports will be visible.
anonymous74100 wrote:Will users earn achievements, badges or something?
As I said: "indication of the helpfulness rating in the issue report (probably stars of some kind)"
anonymous74100
Posts: 213
Joined: Sat Mar 19, 2011 3:45 pm
Contact:

Re: Improving issue reports: recognizing users

Post by anonymous74100 »

Wladimir Palant wrote:Are you afraid of rainbow tables? Email addresses (unlike passwords) typically are long enough that they aren't an issue.
I realise that it's unlikely, but that doesn't mean it's not an issue. Besides why choose an inferior hash function, when there's a better and stronger available.
Wladimir Palant wrote: And if somebody compromised our server - extracting email addresses from the reports is a lot easier than reversing MD5 hashes.
:? I thought only hashes are going to be stored, not the emails.
Wladimir Palant wrote:There will be no (visible) user ranking table nor the list of reported reports - only the number of reports will be visible.
What's the point of ranking them if no one will know who has the top score.
Wladimir Palant wrote:As I said: "indication of the helpfulness rating in the issue report (probably stars of some kind)"
I'm interested in the specifics. What kind of stars, what color, how big etc.
Latvian List maintainer
Wladimir Palant

Re: Improving issue reports: recognizing users

Post by Wladimir Palant »

I already said why MD5 has been chosen - it is simple (fast) and produces short hashes.
anonymous74100 wrote: :? I thought only hashes are going to be stored, not the emails.
Heh... This is the wrong place to get educated about how issue reports work. Of course the email address is stored in the report itself - it's a way for subscription maintainers to communicate with the reporter and to ask questions. Also, status changes get communicated to this email address.
anonymous74100 wrote:What's the point of ranking them if no one will know who has the top score.
The point (as I already said in the first post) is to sort the list of reports so that the reporters with a better score are at the top - these are more likely worth looking into. I am not turning this into a competition, not with the potentially privacy-sensitive data we have there.
anonymous74100 wrote:I'm interested in the specifics.
I noticed. But I am not - I'm rather interested in discussing an idea and getting useful opinions. I don't care which icon Andrey chooses in the end if it does the job.
Wladimir Palant

Re: Improving issue reports: recognizing users

Post by Wladimir Palant »

Ok, I realized now that your subscription got only one report in the past month - that explains things. Other subscription authors get thousands, they should understand what I am talking about.
anonymous74100
Posts: 213
Joined: Sat Mar 19, 2011 3:45 pm
Contact:

Re: Improving issue reports: recognizing users

Post by anonymous74100 »

Wladimir Palant wrote:it is simple (fast) and produces short hashes
Fast is subjective, have you tried implementing it to test the difference? Shortness is meaningless, you're not going to memorise it anyway.
Wladimir Palant wrote:Heh... This is the wrong place to get educated about how issue reports work. Of course the email address is stored in the report itself - it's a way for subscription maintainers to communicate with the reporter and to ask questions. Also, status changes get communicated to this email address.
Wladimir Palant wrote:On the server, create an "account" for each email address seen (don't actually store the email address, merely its MD5 hash).
Wladimir Palant wrote:Ok, I realized now that your subscription got only one report in the past month - that explains things. Other subscription authors get thousands, they should understand what I am talking about.
That was mean. :( I have only had 5 issue reports at all, and only one was targeted at Latvian List.

:idea: How about asking users of multiple subscription which subscription is their issue report targeting?
Latvian List maintainer
User avatar
Hubird
Posts: 2850
Joined: Thu Oct 26, 2006 2:59 pm
Location: Australia
Contact:

Re: Improving issue reports: recognizing users

Post by Hubird »

How about asking users of multiple subscription which subscription is their issue report targeting?
I have thought of this but most users would not know which subscription their issue is related to, especially if they have multiple overlapping subs.
anonymous74100
Posts: 213
Joined: Sat Mar 19, 2011 3:45 pm
Contact:

Re: Improving issue reports: recognizing users

Post by anonymous74100 »

It could be an optional step, for those that actually know which subscription(s) the report should target. Such reports should also get an increased "helpfulness" rating compared to non targeted ones.

:idea: Another idea:
Full issue report data in email. Currently there's not much information in the email about a issue report, it would probably be useful to have all data in it (blockable elements, subscriptions, screenshot etc.).
Latvian List maintainer
User avatar
Hubird
Posts: 2850
Joined: Thu Oct 26, 2006 2:59 pm
Location: Australia
Contact:

Re: Improving issue reports: recognizing users

Post by Hubird »

anonymous74100 wrote:It could be an optional step, for those that actually know which subscription(s) the report should target. Such reports should also get an increased "helpfulness" rating compared to non targeted ones.
I like this idea. People who make it easier for the subscription author to fix the issue the should be "rewarded".
Wladimir Palant

Re: Improving issue reports: recognizing users

Post by Wladimir Palant »

anonymous74100 wrote:It could be an optional step, for those that actually know which subscription(s) the report should target. Such reports should also get an increased "helpfulness" rating compared to non targeted ones.
Somebody submitting an issue report has absolutely no way of determining which subscription it is meant for. If you get any info this way it will simply be bogus.
anonymous74100 wrote:Full issue report data in email.
How is that supposed to work? We aren't talking about one or two issue reports. I got a mail with 100+ issue reports today, 20 kB of text as it is now. That's reasonably readable with one line per report. Now what if each report would take a dozen lines? That would be a horrible mess and nobody would be able to get an overview there.

Can we stop off-topic discussions? Or is the conclusion that nobody has an opinion on the proposal here?
Post Reply