Tracker issues continued
EAP'ers,
Most of you know about the various problems we have with the tracker and
the way we use it for collaborating with our community. The biggest problem
with bug and feature requests is that they get posted to the tracker without
being discussed beforehand which often leads to poorly formulated, impossible
to plan, hard to follow, etc issues. Given the size of the community and
the amount of features in the product, this becomes a very hard task to maintain
in real time (which is the real value of the tracker actually).
We have the following idea: we restrict the write access to the tracker to
a limited amount of EAP'ers. This team of people is elected by the EAP community.
These people will be able to submit/modify issues in the tracker and help
us do the work for maintaining the tracker in the state of order. We will
create two discussion forums dedicated to bugs and features where all those
should be posted, discussed, refined and made up as useful requests that
are possible to use for planning and implementing features and fixing bugs.
And of course, this does not concern exceptions, they will get submitted
by anyone as before.
Therefore, if you are willing to help and would like to be the one with the
write access, please send your name to Max Shafirov at max (at) jetbrains.com.
We will make up a list of all submitted names and create a poll for all
of you to vote on those 10-15 people who will make up this team.
We hope you understand, let us know if you have any other questions.
Best regards,
Eugene Belyaev
President, CTO
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"
请先登录再写评论。
Eugene,
Wouldn't you reach your goal if
- we don't change the forums,
but
- the "15 chosen ones" are allowed to tag issues as
"More info needed"
"Do not fix "
"Duplicate"
, and
- you give them a way to vote for an issue ( "Yes", "No", "no opinion"),
I feel it would work as well, at a lower cost for the helping team.
Alain
Alain,
Unfortunately I don't think it will help. We do this for pre-processing
of information, otherwise it will turn out into something like we have now.
By the way, I did not say that only helpers will have to do all that, of
course the main task is ours.
Best regards,
Eugene Belyaev
President, CTO
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"
Eugene,
I'm not sure if I got this right, so here's my stupid question: What
actually do you refer to with the term "tracker"? The old JIVE-forums based
one or the new JIRA one? In case it's the original one, what role is JIRA
going to have?
I also don't really get the picture about what the elected team is supposed
to do. Is this for transferring issues from the forums to JIRA or just some
kind of "moderation", such as identifying duplicates and filtering out junk
inside those forums or possibly both?
Who/what will ensure that valid and properly formulated requests will get the
attention/acknowledgment/feedback that's quite often missing in the current
system?
Sorry, that it turned out into more than one stupid question :)
Sascha
I can't help but wander what this will do to the community, especially
to people new to IntelliJ which are willing to contribute.
Some things come to mind:
1. Initial campaign to get to the "board of directors".
2. Is this role forever? What will be the criteria to staying in the role?
3. Is there a time period after which new "elections" should take place?
I am just a little bit too afraid that with all good will involved, this
will turn out to be something similar to an HOA (Home Owners
Association) board of directors, where the majority spirit sometimes
seems to fall between the few chairs of the board.
I would be willing to accept that you are looking for a way to restore
the momentum behind EAP of 2 and 3 years ago, but I am not sure that
this won't have the opposite effect.
Amnon
I suspect that this will go badly, based on my previous experience with such a "gatekeeper driven" issue tracking system.
Getting a tracker issue filed now has fairly low overhead, so I am willing to post issues and vote on issues others have posted. I suspect that having to sell someone else on posting my issue would involve enough additional overhead for us both that it would not happen.
That said, if you want a quiet forum, the very best way to do it is to prevent people from posting. It will quiet things down tremendously.
Scott
With all due respect this idea doesn't seems to be very helpful nor workable to me.
Why not to stick with two old and proven concepts:
1. Divide and conquer
2. Code owner institute
Here is how I would do it:
1. The whole "bug space" is divided into manageable pieces by application/functionality
and/or by other means which suit our needs best (Divide and conquer)
2. One (or more if necessary) knowledgeable company's engineer[s] is/are
assigned to each component as a designated Code Owner.
3. Code owners would form their support groups composed mostly
from chosen EAP members and/or from other company staff on
a permanent and/or temporary basis as needed.
(Frankly I don't see how election by EAP community could work at all.
These support groups must work closely with appointed Code Owner
and you don't want President vs. Congress situation here. Besides
EAP members of support groups must be willing to take new responsibilities,
nobody can force them to do this)
4. Code owners would have the final authority and RESPONSIBILITY to
sign off new requests/issues and TO MAKE SURE they got company's
attention, i.e. planned for release, allocated resources etc.
5. Code owners may delegate some or all of their authority (in case of vacation
for example) to the members of their support groups on permanent and/or
temporary basis providing that there is at least one knowledgeable company
contact person is available to group members at all times.
6. Different groups may form forums to
discuss cross issues and inform other groups/owners of proposed changes.
7. Current Code Structure is posted and explained on the company's web-site
so other EAP members can address and discuss their needs to/with specific
group. After the request got it's approval by the group members they would
generally recommend the Code Owner to incorporate it into company's
release schedule.
The advantages of the above approach are obvious. We (EAP members) will have
a group (i.e. more than one and with real names, not faceless tracker) of experts to
talk to with more or less guaranteed response time and
company's engineers will not get overloaded with stupid repeated questions, they
would generally communicate with their group members who do all the filtering
and refining. Since groups are formed on a voluntary basis and by the Code Owner
himself there is no room for arguments an/or misunderstanding between Code Owner
and the group.
--Oleg
"Eugene Belyaev" <beg@jetbrains.com> wrote in message news:unit-06599984632410727277971947@news.intellij.net...
>
>
>
>
>
>
Eugene,
I agree with the EAPers that this would have a big danger of backfiring.
If I understand you correctly, you would like to replace 'anybody can
submit a bug report' with 'anybody can discuss features/bugs on the
newsgroups, and only certain people can actually file issues'.
Assuming I understood you correctly, here's what I think would happen:
People who submit cosmetic and/or hard-to-reproduce bugs are in danger
of being ignored because the other EAPers don't 'see' the problem; The
person to raise the issue would not get email notification about the
bug's progress because they didn't actually put it in tracker; Feedback
would be slower and less consistent, reducing the desire to continue
raising issues; EAPers with write access would have too much
time-consuming responsibility to create issues that they are not
directly involved with, making them feel like data-entry clerks.
Why not have a two-step system? Issues get discussed by the community,
and then one of the few EAPers would 'turn it on' or 'escalate it' if it
is worth fixing, which would bring it to the attention of the IDEA team.
A simple flip of a switch, instead of a manual re-entry of a long
discussion.
Oleg Danilov's idea has some promise, although it seems a little
complicated. Perhaps it could be simplified.
--
Rob Harwood
Software Developer
JetBrains Inc.
http://www.jetbrains.com
"Develop with pleasure!"
Rob Harwood (JetBrains) wrote:
>
+1
Alain
In article <cs1hkd$g29$1@is.intellij.net>,
"Oleg Danilov" <o_danilov@yahoo.com> wrote:
Jira does that and is setup that way already.
That's setup in Jira it seems already, but it's setup per application, I
don't know if jira can assign a per module engineer so people know who
to approach if there is a problem in an area, if not I suggest a web
page be posted with the info on who to contact.
Yes I think it would be better for each code owner to choose his/her eap
members to help, whom ever they feel are most useful and can help them
in the task. I agree that voting is kinda skewed to a general
popularity contest.
I would add that they might still want to allow someone to add / update
a bug, and if they don't like it, they would override and make the final
call on opening/closing etc...
hum... this is a voluntary basis thing. I think that if code owner is
on vacation, someone else FROM JB takes over for him/her, not a team
member.
No closes discussion too much, it's a layer that's not needed.
I'm for this approach for sure, better than the other proposed idea.
R
Rob,
+*Why not have a two-step system? Issues get discussed by the community,
and then one of the few EAPers would 'turn it on' or 'escalate it' if it
is worth fixing, which would bring it to the attention of the IDEA team.
A simple flip of a switch, instead of a manual re-entry of a long
discussion.*+
I like your suggession of a two tier system, where bugs
and issues get digested before percolating down, in a more
concise form to the tracker.
This, together with some demarcation of the functionality
areas, along the lines of Oleg's suggestions would
probably help a great deal.
These will also ensure that anyone willing to participate
more actively in certain areas (and that may be on an
intermitent basis given people's other commitments) can
still do this.
Cheers,
Bonny
you should never forget that people who file bugs
are your biggest supporters.
Other people might just silently stop using IDEA.
Best regards,
Michael
I think I will say goodbye to the community then
- I don't have much interest in a popularity contest.
I think that 10-15 is way too low and gives too much power to each individual. Why not make it 50?
The two tier idea is also a good one, whereby everyone gets to contribute and issues can slowly bubble upwards to the dev team.
-10
IMHO, the additional friction this would cause on submitting and processing ALL issues (including the well-written ones) would not be worth the benefit of not having to deal with badly formulated ones.
As someone who has worked in support before, I know how frustrating it is to deal with unclear bug reports; therefore I always try to write concise tracker entries myself. So, I feel I would not need to have someone else approve my issues, and I would hate having to ask someone else for this (on the other hand I would hate even more having to approve other people's entries).
Don't you think it would be more efficient to simply introduce a new issue state (named "edited" or "reviewed" or something similar) which would be set right after submission and would function as a sort of approval for further processing.
This state (which would amount to meaning "yes, the problem/request is clearly described") could only be set by JB (or possibly by some wider audience); if the issue is not well-formulated or unintelligible, the entry state would be set to "To be discussed" or "Rejected".
This way, the bad submissions can be filtered out right away, and the good ones go through without too much friction.
Regards,
Jens
Ok, I see the problem. All we want is that issues get discussed before they
make it into the tracker. Any way of escalation would be good for us, I
am not insisting on either one. If you know any way you can simply implement
it in Jira, let me know.
The idea of community voting for certain people is only to make that number
limited. By write access I also mean a way to modify any issue. Maybe anyone
could submit a request but only a certain group of people would be allowed
to modify any issue.
Best regards,
Eugene Belyaev
President, CTO
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"
I am referring to Jira. The old one will get closed soon as promised.
The team would help in turning problems into well formulated tracker requests.
The JetBrains team has the responsibility to ensure that requests in tracker
are handled.
Best regards,
Eugene Belyaev
President, CTO
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"
Eugene Belyaev wrote:
I have no experience customizing Jira, but perhaps it is possible to add
a new Priority below Minor, or maybe a new Status before Open. You could
call it 'In Discussion', and default all new issue entries to that
priority/status. They would be filtered from the list of 'open' issues,
and generally ignored (for the purposes of planning and whatnot) until
they are escalated to Open or some Priority.
--
Rob Harwood
Software Developer
JetBrains Inc.
http://www.jetbrains.com
"Develop with pleasure!"
I know it's a lost cause, but I'll say it anyway:
I would gladly volunteer to filter/clean/moderate the current forums,
because they are efficient, and I feel comfortable with them.
The little time I've spend with JIRA web interface, as a poster/lurker
(not as a user, like you are) makes me want to avoid it, at all cost.
The best solution for us, as for you, would be (see motivation at the
end) to
- keep the current 2 forums, for everybody to post and discuss freely
bugs and request.
- use JIRA for your internal tracking and scheduling
but
- give some members authority to change issues status
("duplicate", "more info requested", "obsolete", ..)
- mirror automatically forum -> JIRA (
promoted tracker issue ( "open") enter the JIRA system, with 2 ways
links)
- mirror automatically JIRA --> forum
fixed issue, in JIRA => forum thread is update
Motivation:
The current forums are very efficient for us, with
- their newsgroup mapping, and
- their clean messages: the tracker overhead is minimal (1 url at the
top/end of the message), and useful when you want to see the full
thread in 1 screen
JIRA message are a lot noisier => harder to read => take more time
=> will be read less.
The tracker knows how to format.
It takes no time to post, read, and reply in the current tracker. I find
JIRA a lot more complex. Richer, but more complex;
Alain
As Alain, Sascha, Rob and lots of people pointed out, there is nothing fundamentally wrong with the current tracker. All that is needed is an additional state for new issues.
Something like "Reviewed" with substates (OK, Need More Info, Gibberish, Not A Bug) would work. Only JetBrains staff and other selected individuals (more on that soon) would be able to set these states.
About selecting some EAP members to help, I think you should NOT let the community select them, no matter how nice and democratic this idea sounds. JetBrains should pick them, based on whatever arbitrary criteria they want. Of course, the selected people still need to accept the role and be aware of the resposibility that comes with that.
If this is possible, I think it is a far better solution.
Norris Shelton
Sun Certified Java Programmer
Oleg Danilov wrote:
>With all due respect this idea doesn't seems to be very helpful nor workable to me.
>
>Why not to stick with two old and proven concepts:
>1. Divide and conquer
>2. Code owner institute
>
>Here is how I would do it:
>1. The whole "bug space" is divided into manageable pieces by application/functionality
>2. One (or more if necessary) knowledgeable company's engineer[s] is/are
>3. Code owners would form their support groups composed mostly
>4. Code owners would have the final authority and RESPONSIBILITY to
>5. Code owners may delegate some or all of their authority (in case of vacation
>6. Different groups may form forums to
>7. Current Code Structure is posted and explained on the company's web-site
>
>The advantages of the above approach are obvious. We (EAP members) will have
>a group (i.e. more than one and with real names, not faceless tracker) of experts to
>talk to with more or less guaranteed response time and
>company's engineers will not get overloaded with stupid repeated questions, they
>would generally communicate with their group members who do all the filtering
>and refining. Since groups are formed on a voluntary basis and by the Code Owner
>himself there is no room for arguments an/or misunderstanding between Code Owner
>and the group.
>
>--Oleg
>
>"Eugene Belyaev" <beg@jetbrains.com> wrote in message news:unit-06599984632410727277971947@news.intellij.net...
>
>>EAP'ers,
>>
>>Most of you know about the various problems we have with the tracker and the way we use it for collaborating with our community.
>>The biggest problem with bug and feature requests is that they get posted to the tracker without being discussed beforehand which
>>often leads to poorly formulated, impossible to plan, hard to follow, etc issues. Given the size of the community and the amount
>>of features in the product, this becomes a very hard task to maintain in real time (which is the real value of the tracker
>>actually).
>>
>>We have the following idea: we restrict the write access to the tracker to a limited amount of EAP'ers. This team of people is
>>elected by the EAP community. These people will be able to submit/modify issues in the tracker and help us do the work for
>>maintaining the tracker in the state of order. We will create two discussion forums dedicated to bugs and features where all
>>those should be posted, discussed, refined and made up as useful requests that are possible to use for planning and implementing
>>features and fixing bugs. And of course, this does not concern exceptions, they will get submitted by anyone as before.
>>
>>Therefore, if you are willing to help and would like to be the one with the write access, please send your name to Max Shafirov at
>>max (at) jetbrains.com. We will make up a list of all submitted names and create a poll for all of you to vote on those 10-15
>>people who will make up this team.
>>We hope you understand, let us know if you have any other questions.
>>
>>Best regards,
>>
>>Eugene Belyaev
>>President, CTO
>>JetBrains, Inc
>>http://www.jetbrains.com
>>
>>"Develop with pleasure!"
>>
>>
>>
>
>
>
Alain Ravet wrote:
Full ACK. It might just be an issue of learning and getting used to it,
but JIRA "doesn't feel comfortable" for me as well.
Agreed. This would eliminate
- the additional work that would be put upon the selected team which would
spend a lot of time doing copy&paste from the forums into JIRA.
- creating another layer that would allow things to fall through the cracks.
- the problem that the whole system may crash after the first weeks of enthusiasm.
Especially if it turns out to be a copy&paste job.
- the missing feedback for "standard" EAP users when just entering a problem in the
forum and the problem of the missing "that's my bug" feeling in case someone else
transfers a problem from a forum to the tracker. The reporter should always be the
"owner" of a bug/feature request.
For this to work even better, encourage people to write clear and concise reports.
Don't just ignore reports (in the dreaded "Submitted (Assigned)" state) for weeks,
months and even years, give immediate feedback, and possibly have a non-JB team to
give this feedback.
Sascha
...of course the people who file feature requests
are also your supporters ;)
the main problem currently seems to be: just too much information to cope with. It needs to be "spam"-filtered, but without false positives ;)
I wish you good luck in developing a model for the
EAP-Spam-Filter :)
In our bug system, we have a 2 tier mechanism, but the 1st tier is an "event' system.
So, a user detects a "problem" with the system. They file an event. At some point an individual (maybe an engineer, maybe not) determines that the event is mapped to another problem. That event then gets escalated into the problem. Problems with lots of events typically are higher priority then those with very few events.
This allows engineers to tie together seemingly different behaviors into a single "problem" that has a single "solution".
In the begining, I would probabaly try allowing everyone to modify anything. Let's see if we don't get a "natural selection order" out of it. If I do something stupid and create problems without searching the problem database and properly matching it to an existing problem, then the community (and indeed JetBrains) could come down on me.
Individuals who use IDEA but don't necessarily follow all the problems should generally be creating "events' and not "problems". Those who monitor what's going on and have a good feel for the program can create problems for "un dispositioned" events, further elaborating on what the root cause issue is.
Mike
"Assuming I understood you correctly, here's what I think would happen:
People who submit cosmetic and/or hard-to-reproduce bugs are in danger
of being ignored because the other EAPers don't 'see' the problem; "
I'm not sure it would end there. It could be as simple as the bug report
being deemed "unworthy" of JB's time. Does anyone remember the heated
debates on Aspects? The vocal few screamed loudly for them whereas others
didn't use them on a daily basis or didn't understand their value. What if
the bug report is regarding something like modules...and the gatekeepers
have setup their modules a long time ago? These old school experts
understand modules very clearly and cant understand for the life of them why
the new guy is reporting a bug on what appears to be a usability issue from
a new person's perspective. "He just doesn't get it...this is not a bug".
It is one thing for JB to reject the bug report. It is their product.
We're just here to help. It is an entirely different thing to have to
convince some power user that your needs are important. That said, I could
think of who some of these power users might be. I've been here long enough
to know that these are genuinely good people. I still dont want to run my
bug reports by them. My relationship is with JB.
The community is extremely valuable; we shouldn't discount them (each
other). But the Aspects situation leaves me very skeptical of a vocal
minority influencing a product even further by controlling what is deemed a
bug.
"Rob Harwood (JetBrains)" <rob.harwood@jetbrains.com> wrote in message
news:cs1ivf$re0$1@is.intellij.net...
>
>
>
>
>
+ Most of you know about the various problems we have with the tracker and
the way we use it for collaborating with our community. The biggest problem
with bug and feature requests is that they get posted to the tracker without
being discussed beforehand which often leads to poorly formulated, impossible
to plan, hard to follow, etc issues.
We have the following idea: we restrict the write access to the tracker to
a limited amount of EAP'ers. This team of people is elected by the EAP community.
+
I really think it is a bad idea to restrict write access to the tracker. People will feel less involved with the EAP development, and be less likely to contribute, myself included. I like the ability to file bug and feature requests, comment and vote on other requests.
If people are willing to contribute their time to help you review, aggregate, and organize bugs and feature requests, that's great. But I don't think these people should be gatekeepers.
You could give greater permissions to these volunteers so that they could help organize the submissions. For example, closing a new bug as a duplicate. Or, for a feature request, the person may be familiar with what's planned in the EAP, and they may be able to roll up a bunch of existing requests into the tracker request for the new EAP feature.
From my own experience, I know that we usually don't even look at feature requests once we have committed to the feature set for the next release (like your planned features for Irida). The feature requests just pile up, with nobody to look at them. But once we are onto planning for our next release, we wade through all the requests as best as we can and create a candidate list of new features which are prioritized and then based on resources, a line is cutoff line is drawn and features below that are dropped...
I think it would be odd if we told our customers that they can't file feature requests for our product.
One off-the-wall approach to reducing the number of submissions is to change the submission process. When I click Submit to submit a new request it should show me a page of similar requests by finding other open requests similar to my request description? Maybe you could score the matches. For the matches with a very high score, you could force the user to click a checkbox saying "Not the same".. The success of this would depend on how smart the search results are, but what it would accomplish is get people to think more before submitting a request.
You could also be even more proactive with the above. If the user clicks "Submit New Request", you could first bring them to a search form and ask them to first search for open requests which match your problem. After they have done at least one search, then you allow them to proceed to the real Submit New Request dialog. What this accomplishes is to get new users conditioned to check for duplicates first. This may get more people to more often comment and vote on existing requests rather than submitting new ones??
It might also help to force the user to select a Functional Category for the request,e.g. Editor, CodeCompletion, Debugger, Plugin DevKit, etc. In my own experience, this can be helpful to segment the requests and assign support/development owners, similar to what Oleg suggested. Our engineering team has about 40 developers and each are assigned as owners for differnet parts of the product. The owner gets email whenever a bug/request is created in their area. There is overlap, so sometimes multiple people get emails.. The owner is responsible to maintaining the requests for their areas.
I can see in your case the problem is that you let all customers directly add requests to your internal tracker. In our case, the customer requests are in a separate database, and the support team filters the requests. However, the support team only does simply filtering, mainly checking for duplicate requests. Any requests which are not duplicates will make it to our internal database.
It seems like you could accomplish the same by having a field called source with "External", "Internal". Also, you could have state like "New", "Confirmed" (meaning jetbrains or your trusted helpers confirm it is a bug and isn't a duplicate or it is a request and it isn't a duplicate and it isn't already in EAP plan). "Assigned", etc..
Jetbrains can then filter out all New requests if they want and only look at confirmed requests. But I wouldn't rely on volunteers to be the sole gatekeepers between "New" and "Confirmed." Jetbrains should have the development owners also be looking at the "New" requests in their area.
Maybe some people could post some links to other open bug databases that have some good ideas to help JetBrains with this problem.
One example is Apache's web server. They are using the BugZilla bug tracker software.
I like how when you click "Bug Report" ( http://httpd.apache.org/bug_report.html ), it first tells you to try to search to see if it has already been filed. They also have a few precanned searches, like "Frequently reported bugs (bugs with duplicates changed in the last 30 days)"
Also, each request submitted by external users initially has an UNCONFIRMED state. Users who have the "canconfirm" permission set may confirm a bug, and promote it to NEW state.
UNCONFIRMED
This bug has recently been added to the database. Nobody has validated that this bug is true. Users who have the "canconfirm" permission set may confirm this bug, changing its state to NEW. Or, it may be directly resolved and marked RESOLVED.
NEW
This bug has recently been added to the assignee's list of bugs and must be processed. Bugs in this state may be accepted, and become ASSIGNED, passed on to someone else, and remain NEW, or resolved and marked RESOLVED.
ASSIGNED
This bug is not yet resolved, but is assigned to the proper person. From here bugs can be given to another person and become NEW, or resolved and become RESOLVED.
I'm not holding up Apache's Bugzilla as a model of perfection -- it seems very basic; but maybe some of the ideas can be used? I don't have any experience with using a Jira database. Maybe someone could post a good exmaple of Jira.
Michael Kirby wrote:
We sort of simulate that with Duplicates. Many issues meld into one.
However, your comments make me think maybe we do indeed need a two layer
system: One layer is the 'external interface' to the users, where they
report issues as usual (just as the Tracker/newsgroups do today). The
other layer would be the 'internal interface' to the team, where user
issues are organized into team tasks/problems. Today, the Tracker plays
both roles, but not very effectively the 'internal interface' role.
So, if we had a more powerful team-centered internal system, then the
glue between the two layers could be the dedicated EAP members, who
guide the discussion in the external tracker and escalate issues to the
internal tracker, and also help clarify the internal issues for the team
members if necessary. This would separate the team members from
information overload, while keeping it easy and light-weight for the
casual user/bug-reporter. The designated EAP members would have more
control over what features should be focused on, etc., and since there
would be many of them (I'm imagining around 50 or so), I think it would
make their/our job not so difficult as to be discouraging.
--
Rob Harwood
Software Developer
JetBrains Inc.
http://www.jetbrains.com
"Develop with pleasure!"
Rob
>
That's exactly what I suggested: layer-1 = the (current) tracker,
layer-2 = JIRA, with bi-directionional sync and mirroring, when required
Alain
Note that it isn't just duplicate removal. It is coallessing external behavior into internal action (i.e. the problem might be a threading issue in xml output code, but we have an event in there indicating that user preferences aren't saved all the time, and another event indicating that in crash recovery, we sometimes dont' start where we left off (all of these events being indications of a file not being saved correctly due to the threading bug).
Don't leap so quickly to creating the "limited team". Instead, allow anyone to do the filtering, and encourage the community to police the quality. I suspect that a few comments from JetBrains individuals when things are not organized properly will result in a rapid alignment of the community.
If we find it is too difficult to control editorial standards, then we can migrate to the "nifty fifty" approach.
Mike
+1
It won't detract from the immediacy of the current system and will add a
tier for order.
Amnon