Issue tracking

0 votes
asked Jan 11, 2023 in Question / help by BenceSzalai (200 points)

Hi!

I am wondering few things:

- Why there are bugs accepted in this forum and on GitHub issues as well? Which gets more attention, which creates more overhead for development?

- Why there are lots of issues on Github that are fixed according to the comments, but never closed? (I can collect examples if someone may not see what I mean.)

I've opened several issues in Github recently which I've found while learning PlantUML and adapting my workflow to work with it. I prefer Github, because that acts like a central place for my contributions and issues towards most projects, but it is not important for the project, just saying...

However I'd like to better understand you process so I can be a better citizen when reporting my findings. So where issues should really go?

Also I am thinking that having 558 open issues in Github may give some prospective users a bad vibe about the project while my actual experience is that bugs are resolved quickly, just somehow there are many of them being left there unclosed indefinitely despite being resolved.

1 Answer

0 votes
answered Jan 12, 2023 by plantuml (295,000 points)
selected Jan 12, 2023 by BenceSzalai
 
Best answer

- Why there are bugs accepted in this forum and on GitHub issues as well? Which gets more attention, which creates more overhead for development?

This is for historical reason. This project was created in 2009 and GitHub in 2008, so at the very beginning, we did not even know GitHub :-)

We've created this forum to get feedback from users (and not only issues). It is also a good place for users to exchange about PlantUML (and this way we learn about what people are expecting).

Both gets 50% attention, which means that you can really use the one you like better.
- GitHub is easiest for people having already a github account
- The major advantage of the forum is that it can be used anonymously, so new users don't even have to create an account to give feedback.

Our secret wish is that in some future, some IA would be able to get all information on the forum (which is unstructured) and will generate some nice documentation about how to use PlantUML :-)


- Why there are lots of issues on Github that are fixed according to the comments, but never closed? (I can collect examples if someone may not see what I mean.)

I've opened several issues in Github recently which I've found while learning PlantUML and adapting my workflow to work with it. I prefer Github, because that acts like a central place for my contributions and issues towards most projects, but it is not important for the project, just saying...

However I'd like to better understand your process so I can be a better citizen when reporting my findings. So where issues should really go?

Ok, I'm going to explain our process.
First, I must say that your reports are perfect: you give some short examples about what is not working and you made some investigations to better understand what is working and what is not working. This is very nice because it really speeds up our resolution time :-)

Because the truth is that we have very limited time available.

So we have decided since the beginning that we will really focus on coding.

Everything else (documentation, issue closing...) is only secondary. Of course, we know this is important. We could have someone at full time dedicated to documentation, issue management and so on, be unfortunately we don't have this people. So we really do the minimum that works about these, and we are expecting the community to help us about it.

Our development process is totally chaotic:

1) We continuously take into account contributions from GitHub issues and the forum.

2) Many of them are minor issues that are easy to fix. We use 50% of our time on those issues because we don't like bugs and because it's easy. Once again, we focus on coding, which means that sometimes we don't even take the time to close the issue itself...

3) Some requests are new features requests. In that case, we wait to have several requests asking for something consistant and close, and we work on the new design needed for those new requests. A good example of these is the recent addition for EBNF and Regular Expression support. Let's says it's about 10% of our time.

4) The 40% time remaining is about major issues. Those issues are major in the sense that they are more "code design issues" than "coding issues". A good example of these is all issues about "namespace", "package" and "together". All those issues are linked together and caused by the fact that the initial design we've done in 2009 does not properly allow the management of namespaces. We have started a major refactoring about this some months ago to properly manage "namespace" but it really takes time because PlantUML code begins to be quite big and complex and because we don't want to break existing and working diagrams.


Also I am thinking that having 558 open issues in Github may give some prospective users a bad vibe about the project while my actual experience is that bugs are resolved quickly, just somehow there are many of them being left there unclosed indefinitely despite being resolved.

Well, any help is really welcome here.

Easy bugs are indeed quickly resolved. Some other may take more times.

Now, as I said, we don't have time to look at the 558 open issues so if you want to close some of them, please be our guest :-) I'm afraid it will be a boring job, though.

You can also edit the documentation itself here.

I realize that my answer will probably open some more questions... Sorry about that!

I will conclude by thanking you about your support :-)

commented Jan 12, 2023 by BenceSzalai (200 points)

Thanks for the detailed answer!

...you give some short examples about what is not working and you made some investigations to better understand what is working and what is not working.

I would even try to crunch out PRs for fixes myself, but Java is way to foreign to me unfortunately. But I am happy that you react this way to reports!

I’m making updates to the documentation whenever I find something interesting. However I'm only working with class diagram and probably will not really use any others for the foreseeable future, so my scope is limited.

So we have decided since the beginning that we will really focus on coding.

That is understandable and probably great for the tool.

Regarding the open issues, I think only the opener and the project maintainers can close them, so I cannot really help. On the other hand I can see myself sometimes crawling old issues to learn about the tools I'm using, limitations, solutions etc. Burning some time now and then to satisfy my OCPD tendencies. :D If I saw something that is definitely solved, I would close, but it involves governance, trust, authorization questions. I mean I'm not abusive, but you cannot know that for sure. ;)

commented Jan 12, 2023 by albert (3,520 points)

@plantuml

Thanks for the great tool and all the work you put into it.

As you wrote: "Because the truth is that we have very limited time available." and this is the case with a lot of software packages due to the limited resources and people actively working on a package.

Regarding the GitHub issues as written only a limited number of people will have the possibility to close issues, so only the submitter or the maintainer, collaborator and ... can close these.
As you wrote all issues are seen and therefore it would probably be relatively easy to assign a label to them (e.g. bug or enhancement) at that moment so that an indication is given (could also be some finer refinement but his would probably be more work). Also questions could be moved to discussion (not sure about it but e.g. https://github.com/plantuml/plantuml/issues/1259 as this has the word question already in the title but there might be a deeper problem, also seen the submitter, behind it).
At the moment that you make a fix for an issue you could also close the issue like you did for e.g https://github.com/plantuml/plantuml/issues/1216
The closing strategy is of course depending on the project, each project has a different strategy for it.
I think the labeling of the new issues (doing it for all the old issues is quite a task, but people could write in comments their findings so it might be easier for you to close / categorize an old issue) might already be an advantage and make maintenance easier.
This all would, as far as I can see, cost a little bit of time (and at startup a bit more as you have to get used to the new way of working) but it also would bring a lot.
My experiences with labeling of issues is good as for doxygen I try to look at all the issues as well and assign a label (I'm not the owner but just a "simple" collaborator there, so I only make suggestions on the issues) or ask for more information as quite often an issue is not 100% clear. Though this all takes a bit of time.
Just my 2 cents ...
commented Jan 12, 2023 by The-Lu (64,760 points)

Hello all,

- Why there are lots of issues on Github that are fixed according to the comments, but never closed?

I can help to manage and close (fixed) issue.
[Thank Plantuml for your invitation to be a collaborator on Plantuml repo.]

First I just close my fixed issues :)
Then, I can see for the other issues...

Regards.
Th.

commented Jan 12, 2023 by BenceSzalai (200 points)

Thanks for the added rights on GitHub! I'll try to see what I can do to help with the issue chaos.

I think the labeling of the new issues might already be an advantage and make maintenance easier.

I guess for this a set of standard tags need to be defined with the associated next action. For the cleanup effort I can see there may be need for a label for issues that are hard to decide if fixed or not for some reason. Because if the issue is clearly not fixed, well, that's an issue, so not much to do*. If it is clearly fixed, it can be closed. But maybe in some cases confirmation is needed. E.g. if someone was having an issue on a specific environment, we'll not be able to confirm that. So a label that expresses "likely fixed, needs confirmation" in a compact form could be added, pinging the reporter in a comment. If they do not report back, the issue can be closed anyway after a period, e.g. 30 days or so.

*: bug/feature/support/discussion can still be indicated, noting that support requests and discussions should probably redirected to this forum instead...

Another reasonable label can be that expresses that additional "information is needed" i.e. the report is not clear or issue cannot be reproduced.

Indeed wontfix is a common one too, however that also implies closing, so the label is of no real use.

commented Jan 13, 2023 by albert (3,520 points)

Upon initial evaluation there are also some labels that can be set like:

  • bug / enhancement: when the issue is clear
  • needinfo: when the issue is not clear here a very common problem is that no good test case is provided so a request for the a test case (and the used plantuml version) should be made. A little help might here be an "issue template" or "issue form". In case of no reaction issue can be closed in say 1 month.
  • won't fix: when the issue is clear (eventually after the user has provided additional information), though this should always have some text with reasoning as well. These issues can be closed directly
  • usage: when the user made a mistake, such issues could be immediately closed or after a little while (to give the user an opportunity to respond in case the evaluator didn't understand the intention of the writer), but a small explanation has to be given.
  • not a plantuml bug: when the problem lies outside of the scope / influence sphere of plantuml. In that case the problem should be reported upstream (preferably by the user)
  • documentation: when the documentation needs some update

Other labels might be useful:

  • solved in the mean time: When one tests an issue a small remark with which version the test is performed is of interest (and when the old version of plantuml is known against which the issue was submitted and the tester still has the that version available, or another more recent version which shows the same problem, it might also be useful information).
  • has pull request: when a pull request is available but this has not been integrated into the master branch yet.
  • specific labels signalling the parts of plantuml where the problem is, though this can lead to a lot of labels and bee contra-productive. A package that extensively uses labels is CGAL, see https://github.com/CGAL/cgal/issues

In case the submitter (or someone else) needs to verify the issue also a special label could be applied as well as pinging the submitter (or someone else) with the request to confirm that the problem is fixed and upon conformation or latest after around 1 or 3 months the issue could be closed (also here the labels and sorting of the labels can be useful).

Even labels on closed issues can be useful as they give some history  information (I explicitly don't imply here that all, old, closed issues have a label).

Also pull requests can have labels (e.g. bug / enhancement) to give some extra information.

I now see that GitHub even provides an initial / standard list (https://docs.github.com/en/issues/using-labels-and-milestones-to-track-work/managing-labels), maybe a good starting point to just use those.

commented Jan 13, 2023 by BenceSzalai (200 points)
Great writeup!

I'd recommend short tags, so for "not a plantuml bug" "3rd party".

Regarding "specific labels signalling the parts of plantuml where the problem is" one that might be fairly easy to categorise and could be useful as well is tag the diagram types the issue effects. But that could become messy, when an issue effects many diagram types, but not all. As for tagging for system component, it could probably only be done by the developers and in many cases only during the attempted fix, at which time it may not really be useful anymore.

The tags configured right now are listed here: https://github.com/plantuml/plantuml/labels

Of these is "Java" seems to be a bit strange to me, as 99.9% of the project is Java, so it adds no information aka. noise. But it is applied by dependabot only, so kind of like a given for GitHub, however if the tag was removed it may not be re-added. I'd try that.

I think it'd be good to make a document where we capture the planned tags, meaning and explanation when/where to use. Not sure what would be adequate form for that. Maybe even this topis is fine. Anyway, once the list is made, it can be configured in GitHub, and start to be applied.
commented Jan 13, 2023 by albert (3,520 points)
  • Short labels I agree and Camel case can give some extra information here. Problem might also be that a longer label like "not a plantuml bug" a label like "3rd party" is to restrictive.
  • "specific labels signalling the parts of plantuml where the problem is" indeed this might get messy quite fast when there are too many labels
  • small separate document, I agree. Note that labels can have a description as well and thus some explanation. Regarding the document I think best is that the discussion is placed on GitHub as is concerns GitHub. Maybe a document in the wiki part of plantuml (https://github.com/plantuml/plantuml/wiki) and discussing it in https://github.com/plantuml/plantuml/discussions The wiki part, unfortunately, has no "issues / pull request" possibility, so someone should evaluate the suggestions and incorporate them. A  different approach would be a separate, private, with just one document but at that moment there is a possibility of issues / pull requests. I'd prefer a private repository so that the owner can give a selected set of people access (as it is just for some "internal" discussions).
commented Jan 19, 2023 by The-Lu (64,760 points)

Hi all,

Step by step...
During this week, I close a lot of issues... currently we have now 458 issues... wink

About label:

During issue analyse (and number of issues related to), IMHO we needs, at minimum, for issue label:

  • documentation
  • enhancement 
  • or: wanted feature
  • graphviz related
  • or: graphviz 
  • or: layout related
  • or just: layout
  • smetana
  • elk
  • teoz
  • port
  • namespace
  • swimlane
  • ...

- Is it necessary or not to create label for all type of diagram?
[class, activity, JSON, YAML, state, nwdiag, ... in order to make filter...]

But in the end, we don't need too much otherwise we will be drowned in the number of labels...

- Who can create label?
Because I'm not allow to create such of labels.

Perhaps create discussion or project (for coordination) on:

Regards.

...